This repository contains tooling for interacting with website robots directives and downloading resources. It is provided only for authorized testing and defensive research.
- Do not use this tool to access, download, or scrape content from websites where you lack explicit permission.
- Accessing disallowed paths or downloading content without authorization may violate laws and terms of service.
- Use responsibly: throttle requests, respect rate limits, and do not cause undue load on third‑party systems.
- The author disclaims any liability for unlawful or harmful use.