feat: Add data models for disciplinary actions, organizations, and training centers

- Implemented DisciplinaryAction model for tracking disciplinary measures.
- Created Organization model to represent auditing organizations with relevant details.
- Developed TrainingCenter model for educational and methodological centers.
- Added methods to convert models to dictionaries for easy export.

feat: Implement parsers for auditors and organizations

- Developed AuditorsParser to scrape and parse auditor registry data.
- Created OrganizationsParser for scraping auditing organizations' registry.
- Implemented GenericRegistryParser for handling generic tabular data.

chore: Set up base parser functionality

- Established BaseParser class with common methods for making requests and parsing HTML.
- Added pagination handling and detailed page parsing capabilities.

feat: Implement Excel export functionality

- Created ExcelExporter class for exporting data to Excel files with formatting options.
- Added support for exporting multiple sheets and organization data specifically.

chore: Add logging utilities

- Implemented setup_logger function for consistent logging across the application.
- Configured logging to output to both console and file.

chore: Update requirements and scripts

- Added necessary dependencies for parsing, Excel handling, and logging.
- Created run.sh and run.fish scripts for easy execution of the parser.
This commit is contained in:
ada
2025-10-25 20:31:45 +03:00
commit 12d5c6eaff
23 changed files with 2210 additions and 0 deletions
Executable
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env fish
# Скрипт быстрого запуска для Fish Shell
# Использование: ./run.fish
echo "=============================================="
echo " ПАРСЕР РЕЕСТРОВ СРО ААС"
echo "=============================================="
echo ""
# Проверка наличия Python
if not command -v python3 &> /dev/null
echo "❌ Python 3 не найден. Установите Python 3.8 или выше."
exit 1
end
echo "✅ Python найден: "(python3 --version)
echo ""
# Проверка виртуального окружения
if not test -d venv
echo "📦 Виртуальное окружение не найдено. Создание..."
python3 -m venv venv
echo "✅ Виртуальное окружение создано."
echo ""
end
# Активация виртуального окружения
echo "🔧 Активация виртуального окружения..."
source venv/bin/activate.fish
# Проверка зависимостей
if not python -c "import requests" 2>/dev/null
echo "📥 Установка зависимостей..."
pip install -q --upgrade pip
pip install -q -r requirements.txt
echo "✅ Зависимости установлены."
echo ""
end
# Создание необходимых директорий
mkdir -p data/exports
mkdir -p logs
# Запуск парсера
echo "🚀 Запуск парсера..."
echo ""
python main.py
# Деактивация виртуального окружения
deactivate