An automated web scraper leveraging Azure Functions and Twilio to streamline the management and tracking of vehicle-related information and driver's license registrations.
- Automated Scraping: Efficiently extracts data related to vehicle information and driver's license appointments using Selenium and Undetected Chromedriver.
- Azure Functions Integration: Runs seamlessly as an Azure Function for scalability and cost-effectiveness.
- Twilio Notifications: Sends SMS notifications for important events like newly available test slots.
- Configurable Preferences: Easily customize search parameters (dates, centers, etc.) via
config.ini. - Intelligent Session Management: Handles license information changes and restarts failed browser sessions.
- Captcha Handling: Attempts to solve captchas (with Buster Chrome extension support).
- Firewall Bypass Attempts: Includes delays and refresh strategies.
- Error Logging: Reports errors for monitoring.
- Python
- Selenium
- Undetected Chromedriver
- Azure Functions
- Twilio
- ConfigParser
- requests
-
Clone the repository:
git clone [https://github.com/vectorc0de/car-license-scraper.git](https://github.com/vectorc0de/car-license-scraper.git) cd car-license-scraper -
Install dependencies:
pip install -r requirements.txt pip install undetected-chromedriver
-
Configure
config.ini:- Update Twilio
phone_numberin[twilio]. - Configure preferences in
[preferences]:licence: Your driving license number.booking: Your booking reference number.current_test_date,current_test_centre,current_test_error: Current test info.disabled_dates: Dates to exclude (e.g.,['2025-05-25', '2025-05-26']).centre: Preferred test centers (e.g.,['Test Centre 1', 'Another Centre']).before_date,after_date: Date range for earlier tests (e.g.,2025-06-01).auto_book_test:yesto enable auto-booking (use with caution!).formatted_current_test_date: Current test date inYYYY-MM-DDformat.
- Update Twilio
-
Set up Azure Function (if deploying to Azure):
- Create an Azure Function App.
- Deploy the Python code.
- Configure environment variables (Twilio SID, Auth Token).
- Ensure
chromedriveris accessible.
-
Running locally:
python your_scraper_script_name.py
(Replace
your_scraper_script_name.py). -
Running as Azure Function:
- Deployed function will execute based on triggers.
- Monitor Azure Function logs.
- Ethical Scraping: Use responsibly, respect
robots.txtand terms of service. - Website Changes: May break the scraper; updates might be needed.
- Captcha Limitations: Solving not guaranteed; manual intervention might be required.
- Auto Booking (Caution): Use
auto_book_testcarefully! - Driver Executable: Ensure
chromedriverin thechromedriverdirectory is compatible