Skip to content

Add capability to block user agents #5749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
patphongs wants to merge 2 commits into
base: develop
Choose a base branch
from
Draft

Add capability to block user agents #5749

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6966bb9
Add capability to block user agents
patphongs Mar 8, 2024
3ab6b49
update to 403 and add check for user-agent first
patphongs Mar 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion webservices/rest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,13 @@ def handle_error(error):
RESTRICT_MESSAGE = "We apologize for the inconvenience, but we are temporarily " \
"blocking API traffic. Please contact apiinfo@fec.gov if this is an urgent issue."

# list of blocked user agent strings: ex: Googlebot, Bingbot, etc that will be result in the request
# being blocked if the user-agent header contains any of the specified strings
BLOCKED_USER_AGENTS = utils.split_env_var(env.get_credential('FEC_API_BLOCKED_USER_AGENTS', ''))


@app.before_request
def limit_remote_addr():
def limit_access_based_on_request():
"""
If `FEC_API_USE_PROXY` is set:
- Reject all requests that are not routed through the API Umbrella
Expand Down Expand Up @@ -176,6 +180,11 @@ def limit_remote_addr():
if request_api_key_id not in BYPASS_RESTRICTION_API_KEY_IDS:
# Service unavailable
abort(503, RESTRICT_MESSAGE)
user_agent = request.headers.get('User-Agent')
if user_agent and BLOCKED_USER_AGENTS:
for blocked_agent in BLOCKED_USER_AGENTS:
if len(blocked_agent) > 0 and blocked_agent in user_agent:
abort(403) # Too many requests


def get_cache_header(url):
Expand Down