getting bot detection html page rather than the flight results in a day

[seowcs]

bot returns googles consent/language selection page rather than flight data, heavily suggesting a bot detection problem. how to resolve?

[dosubot]

Hi @seowcs! I'm Dosu and I’m helping the flights team.

Google is showing the consent/language selection page or bot detection page because it detected automated scraping or needs regional consent. The flights scraper has several built-in ways to handle this:

• Use the Playwright fetch mode, which automates a real browser and can programmatically accept Google's consent page if it appears. This is the most robust built-in anti-bot measure and can be run headlessly or in Docker for isolation and flexibility (source, source).
• Try the Bright Data proxy mode, which proxies requests through Bright Data's API to avoid captchas and rate limiting (source, source).
• Use the cookies API to generate and set consent cookies, which can help bypass the EU consent screen (source, source).
• You can also set the display language and currency to match your region, which sometimes helps with consent and localization issues (source).

Switching to Playwright or Bright Data mode and ensuring consent cookies are set usually resolves this. If you’re running into persistent issues, rotating IPs and monitoring for changes in Google’s anti-bot behavior can also help.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[seowcs]

@dosu can u givve me a step by step on how to implement the playwright config

[dosubot]

Here's how to set up and use Playwright mode in the flights scraper:

1. Install the package with Playwright dependencies:

   pip install fast-flights[local]
   

2. Install the Playwright browser (Chromium is enough for most cases):

   python -m playwright install chromium
   

   Or, to install all supported browsers:

   python -m playwright install
   

3. In your code, enable Playwright mode by setting fetch_mode="local" in get_flights, or mode="local" in get_flights_from_filter. Example:

   get_flights(
       ...,
       fetch_mode="local"  # options: common, fallback, force-fallback, local, bright-data
   )
   # or
   get_flights_from_filter(
       filter,
       mode="local"
   )

   Reference

4. If you use the CLI or the example script, pass --fetch_mode local as an argument.

You can also use fetch_mode="fallback" or fetch_mode="force-fallback" to let the scraper automatically switch to Playwright if the normal method fails or to always use Playwright as a fallback reference.

No extra config files or environment variables are needed—just the pip install and browser setup. If you run into issues, make sure the browser is installed and your Python environment has access to it. Playwright mode will handle Google's consent page automatically by clicking "Accept all" reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[seowcs]

@dosu thanks it works now

[dosubot]

Happy to help! If you have any more questions or run into anything else, just let me know.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}