This repository was archived by the owner on May 30, 2024. It is now read-only.
This repository was archived by the owner on May 30, 2024. It is now read-only.
Too Many Requests Error #1
Description
Running the script as cat file_with_the_url_list | bash ./archiveurl I bump into the issue that the scripts makes too many requests to archive.org and fails with a HTTP 429 error.
So, I usually run this script adding a pause,
{ echo -n; sleep 1; cat file_with_the_url_list; } | bash ./archiveurl
It helps a bit, but still not exactly a set and forget process.
How should we approach this issue?