feat: Add data models for disciplinary actions, organizations, and training centers

- Implemented DisciplinaryAction model for tracking disciplinary measures.
- Created Organization model to represent auditing organizations with relevant details.
- Developed TrainingCenter model for educational and methodological centers.
- Added methods to convert models to dictionaries for easy export.

feat: Implement parsers for auditors and organizations

- Developed AuditorsParser to scrape and parse auditor registry data.
- Created OrganizationsParser for scraping auditing organizations' registry.
- Implemented GenericRegistryParser for handling generic tabular data.

chore: Set up base parser functionality

- Established BaseParser class with common methods for making requests and parsing HTML.
- Added pagination handling and detailed page parsing capabilities.

feat: Implement Excel export functionality

- Created ExcelExporter class for exporting data to Excel files with formatting options.
- Added support for exporting multiple sheets and organization data specifically.

chore: Add logging utilities

- Implemented setup_logger function for consistent logging across the application.
- Configured logging to output to both console and file.

chore: Update requirements and scripts

- Added necessary dependencies for parsing, Excel handling, and logging.
- Created run.sh and run.fish scripts for easy execution of the parser.
This commit is contained in:
ada
2025-10-25 20:31:45 +03:00
commit 12d5c6eaff
23 changed files with 2210 additions and 0 deletions
+18
View File
@@ -0,0 +1,18 @@
# Основные библиотеки для парсинга
requests>=2.31.0
beautifulsoup4>=4.12.0
lxml>=4.9.0
# Работа с Excel
openpyxl>=3.1.0
pandas>=2.0.0
# Selenium для динамического контента (опционально)
selenium>=4.15.0
# Утилиты
python-dateutil>=2.8.0
tqdm>=4.66.0
# Логирование
colorlog>=6.7.0