[BUG] Rate limit exceptions only seen when log level is DEBUG

[killermonkeys]

Describe the issue:

I'm using FastF1 in an ipython notebook (within VSCode).

When I hit the rate limit and my log level is set to WARNING or INFO, I typically see errors like this ValueError: Failed to load any schedule data.. If I set log level to DEBUG, I get a print of the error in the logger: fastf1.req.RateLimitExceededError: any API: 500 calls/h.

It seems like the FastF1 API is handling these fastf1.req.RateLimitExceededError internally, but the ValueError is very ambiguous. There are a bunch of different ways this could be handled but it would be nice to have a way to tell the end user that it's a rate limit, not the code.

Reproduce the code example:

import fastf1 as ff1
ff1.set_log_level('WARNING')
ff1.Cache.offline_mode(False)
race = ff1.get_session(2021, 22, 'R')
race.load(laps=True, telemetry=False, weather=False, messages=False)

Error message:

WITH LOG LEVEL WARNING

ValueError: Failed to load any schedule data.



WITH LOG LEVEL DEBUG

logger      WARNING 	Failed to load schedule from FastF1 backend!
logger        DEBUG 	Traceback for failure in FastF1 schedule
Traceback (most recent call last):
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/logger.py", line 151, in __wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/events.py", line 584, in _get_schedule_ff1
    response = Cache.requests_get(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 303, in requests_get
    return cls._cached_request('GET', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 347, in _cached_request
    response = func(url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 127, in get
    return self.request('GET', url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 183, in request
    return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 230, in send
    response = self._send_and_cache(request, actions, cached_response, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 254, in _send_and_cache
    response = super().send(request, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 134, in send
    lim.limit()
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 104, in limit
    raise RateLimitExceededError(self._info)
fastf1.req.RateLimitExceededError: any API: 500 calls/h
req            INFO 	No cached data found for season_schedule. Loading data...
_api           INFO 	Fetching season schedule...
logger      WARNING 	Failed to load schedule from F1 API backend!
logger        DEBUG 	Traceback for failure in F1 API schedule
Traceback (most recent call last):
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/logger.py", line 151, in __wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/events.py", line 644, in _get_schedule_from_f1_timing
    response = fastf1._api.season_schedule(f'/static/{year}/')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 479, in _cached_api_request
    data = func(api_path, **func_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/_api.py", line 1651, in season_schedule
    response = fetch_page(path, 'index')
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/_api.py", line 1725, in fetch_page
    r = Cache.requests_get(base_url + path + pages[name], headers=headers)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 303, in requests_get
    return cls._cached_request('GET', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 347, in _cached_request
    response = func(url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 127, in get
    return self.request('GET', url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 183, in request
    return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 230, in send
    response = self._send_and_cache(request, actions, cached_response, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 254, in _send_and_cache
    response = super().send(request, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 134, in send
    lim.limit()
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 104, in limit
    raise RateLimitExceededError(self._info)
fastf1.req.RateLimitExceededError: any API: 500 calls/h
logger      WARNING 	Failed to load schedule from Ergast API backend!
logger        DEBUG 	Traceback for failure in Ergast API Schedule
Traceback (most recent call last):
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/logger.py", line 151, in __wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/events.py", line 731, in _get_schedule_from_ergast
    season = fastf1.ergast.fetch_season(year)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/ergast/legacy.py", line 29, in fetch_season
    Cache.requests_get(url, headers=_headers))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 303, in requests_get
    return cls._cached_request('GET', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 347, in _cached_request
    response = func(url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 127, in get
    return self.request('GET', url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 183, in request
    return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 230, in send
    response = self._send_and_cache(request, actions, cached_response, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/requests_cache/session.py", line 254, in _send_and_cache
    response = super().send(request, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 134, in send
    lim.limit()
  File "/Users/u/.pyenv/versions/3.12.10/lib/python3.12/site-packages/fastf1/req.py", line 104, in limit
    raise RateLimitExceededError(self._info)
fastf1.req.RateLimitExceededError: any API: 500 calls/h

[theOehrly]

Thanks for the report. This is clearly a bug and those errors should always be raised to the user.

[killermonkeys]

By the way, I think there are cases where the requests cache is not actually caching requests when you call get_session and Session.load. I've tried clearing the cache and I've also tried adding some debug logging code to req.py, that seems to show this (if I got the cases right), however it was easier in my code to create my own in-memory cache of loaded sessions. I was seeing this particularly for getting the schedule, which happens a lot, but there may be other cases.

For reference, my project has feature extraction across many years and many sessions, so I am repeatedly using FP1, FP2, Q, R sessions with Laps=true from years 2019-2025, and without laps for a few years prior. My cache is currently 646 sessions. Because I've moved to an in-memory session cache, I'm no longer seeing the request cache issues myself so I'm not debugging it anymore. I'm surprised that other folks have noticed this, but perhaps they're not using the API that way.

[theOehrly]

The HTTP requests cache adheres to the cache-control headers. So I don't actually know how long responses are cached, as that is up to the server. It might be only a few minutes. I'd need to check the response headers. Could this be what you are seeing?

The livetiming API results are different in that they are cached separately after parsing. This is done to speed up data loading, as parsing all that JSON is fairly slow. That cache also never expires because we know that this data is never updated on the server. On the other hand, schedule data and Jolpica-F1/Ergast are updated frequently.