Score Scraper for https://hatinh.edu.vn/tracuudiemthihsg

A multithreaded Python tool designed to automatically look up and extract student results from the Hà Tĩnh Department of Education exam portal. It features automated CAPTCHA bypassing and an auto-resume mechanism to handle large batches of Student IDs safely.

---

Features

• Automated CAPTCHA Solving: Integrates ddddocr to bypass portal verification images locally on the fly.
• Multithreaded Execution: Utilizes a thread pool (default: 5 concurrent workers) to speed up lookups while safely syncing OCR processing.
• Smart Resuming: Checks your existing Excel spreadsheet before running, ensuring it only queries missing Student IDs (SBD).
• Robust Error Handling: Keeps trying up to 9,999 times per ID if the CAPTCHA resolution fails, logging true missing/failed items to a separate file.

---

Setup & Installation

1. Clone the repository and navigate to the project directory

cd thisisnotnam-ht-score-scraper

2. Install dependencies

Install the required packages using pip:

pip install -r requirements.txt

Note: ddddocr requires a compatible C++ runtime on Windows. If you face installation issues, make sure your build tools are up to date.

---

How to Use

Step 1: Prepare the Input File

Create a file named sbd_list.txt in the root directory. Add the target Student IDs (SBD), placing one ID per line:

100101
100102
100103

Step 2: Run the Scraper

Execute the Python script:

python scrape.py

Step 3: Respond to the Prompt

The script will calculate how many IDs are missing from your output file and ask:

Loaded 3 SBD(s) from 'sbd_list.txt'.
3 ID(s) not yet in saved file.
Fetch missing IDs? [y/n]: 

Type y and hit Enter to start.

---

Generated Files

Once the script runs, it generates or updates the following files in the project directory:

• scores_all.xlsx: The primary Excel workbook containing all successfully scraped candidate records.
• failed_ids.txt: A plain text file logging IDs that could not be found or failed entirely during the run. This file is reset automatically every time a new batch session starts.

---

Advanced: Changing Target Exams

The script currently targets the HSG 10 (2025-2026) exam context. To scrape a different exam, update the itemId value inside scrape.py with the appropriate hash token grabbed from the portal's network payload:

tokens = {
    'securityToken': get_val("securityToken"),
    'submitFormId':  "1016",
    'moduleId':      "1016",
    'itemId':        "YOUR_NEW_EXAM_ITEM_ID_HERE" 
}