Initial Release of the Epstein Files Downloader.
This release provides a robust solution for archiving the full DOJ Epstein/Maxwell dataset. It specifically addresses issues with file access and directory organization found in other scraping methods.
Features in this release:
- Fixed DOJ Disclosures Parsing: Correctly handles the USWDS Accordion HTML structure. Files are now properly grouped into folders like "Epstein Files Transparency Act" and "Maxwell Proffer" rather than generic or incorrect folders.
- Security Bypass: Implemented
curl/7.68.0User-Agent spoofing to resolve HTTP 404 errors on valid links. - Hybrid Threading:
- Files >50MB trigger an 8-thread chunked downloader.
- Files <50MB utilize a concurrent queue for rapid processing.
- Full File Support: Automatically detects and downloads
.pdf,.zip,.mp4, and.wavfiles. - Live UI: Console dashboard shows real-time transfer speeds, active threads, and progress bars.
Download:
- [Source Code (zip)]
- [DojScraper-x64.exe (win-x64)]