- Implemented loader in `modules/loader.py` to download HTML pages using curl and urllib.
- Created processor in `modules/processor.py` for processing the text of the Russian Criminal Code (УК РФ).
- Added README files for both loader and processor explaining their functionality and usage.
- Generated output files including original text, prepared text, and subject index in CSV and JSON formats.
- Implemented a new crawler module to extract text and links from legal documents.
- Added a loader module to handle HTML downloads with error handling and retries.
- Created a main script to initiate the crawling process on a specified URL.
- Defined functions for normalizing URLs, extracting links, checking terminal pages, and extracting text content.
- Updated project metadata with dependencies and versioning in pyproject.toml and uv.lock.
Dmitry