compact install and update is rate limited

[aaronbassett]

Context & versions:

Attempting to install and update the compact cli triggers a GitHub API 403 (rate limit exceeded) error.

$ compact --version && compact list -i
compact 0.2.0
compact: installed versions

→ 0.25.0

$ sw_vers && uname -m && sysctl -n machdep.cpu.brand_string
ProductName:		macOS
ProductVersion:		14.5
BuildVersion:		23F79
arm64
Apple M3 Max

Steps to reproduce:

Run the compact install command:

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/midnightntwrk/compact/releases/latest/download/compact-installer.sh | sh

and then attempt to update it to the latest version

compact self update

Actual behavior:

Error: Failed to self update

Caused by:
    HTTP status client error (403 rate limit exceeded) for url (https://api.github.com/repos/midnightntwrk/compact/releases)

Expected behavior:

The compact CLI installs and updates to the latest version

Suggestions

The GitHub API has notoriously tight rate limits for unauthenticated requests (60/hour). I would suggest one of the following:

• allow the user to supply a GitHub API key to use with requests
• use the GitHub CLI to perform requests if available on the user's machine
• don't use the API

If repeated calls to the releases API is what is consuming all the API requests, then don't use the API to fetch releases. They're also available by downloading the Atom feed and AFAIK that's not rate limited, or if it is the limits are much higher.

✦ ❯ temp_file=$(mktemp)
echo "Making 100 requests and capturing status codes..."

for i in {1..100}; do
  status_code=$(curl -s -o /dev/null -w "%{http_code}" "https://github.com/midnightntwrk/compact/releases.atom")
  echo "$status_code" >> "$temp_file"

  if [ $((i % 10)) -eq 0 ]; then
    echo "Completed $i requests..."
    sleep 0.6  # Brief pause every 10 requests to spread them over the minute
  fi
done

echo -e "\nHTTP Status Code Summary:"
echo "========================"
sort "$temp_file" | uniq -c | sort -nr | while read count code; do
  echo "HTTP $code: $count requests"
done

echo -e "\nTotal requests: $(wc -l < "$temp_file")"
rm "$temp_file"
Making 100 requests and capturing status codes...
Completed 10 requests...
Completed 20 requests...
Completed 30 requests...
Completed 40 requests...
Completed 50 requests...
Completed 60 requests...
Completed 70 requests...
Completed 80 requests...
Completed 90 requests...
Completed 100 requests...

HTTP Status Code Summary:
========================
HTTP 200: 100 requests

Total requests:      100

[gilescope]

It will check for one of GITHUB_TOKEN and if present the rate limits are much higher as it's then authenticated downloads.
E.g. for your CI please do this: https://github.com/midnightntwrk/example-bboard/pull/1009/files

We're working on creating a setup-compact action to make this even slicker - it will appear here: https://github.com/midnightntwrk/setup-compact-action

[aaronbassett]

Hey @gilescope

That's great! So it already lets the user provide a GitHub API key. Perfect.

My only suggestion then would be to provide a better error message to the user on a 403.

Caution

Error: Failed to self update (GitHub API rate limit exceeded)

Set GITHUB_TOKEN in your environment to increase rate limits
https://github.com/settings/tokens