diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index f9eaacb..6c5ff1c 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -22,10 +22,10 @@ jobs: id-token: write steps: - name: Checkout code - uses: actions/checkout@v4 + uses: actions/checkout@v6 - name: Set up Python 3.12 - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: "3.12" diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..980cd54 --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,35 @@ +name: Test + +on: + push: + branches: + - dev + +permissions: + contents: read + +jobs: + test: + name: Run test suite + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Set up Python 3.12 + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install Poetry + run: | + curl -sSL https://install.python-poetry.org | python - -y + + - name: Update PATH + run: echo "$HOME/.local/bin" >> $GITHUB_PATH + + - name: Install dependencies + run: poetry install --no-interaction --with dev + + - name: Run tests + run: poetry run pytest tests/ -q diff --git a/.gitignore b/.gitignore index c705978..b3bbb52 100644 --- a/.gitignore +++ b/.gitignore @@ -1,13 +1,14 @@ .venv .archive +.claude build data notebooks +scripts/ads-no-subtype *.egg-info *__pycache__ -# Ignore test data +# Ignore test cache .pytest_cache -tests/__snapshots__/* diff --git a/README.md b/README.md index b1c0732..b30e052 100644 --- a/README.md +++ b/README.md @@ -8,53 +8,45 @@ and saving searches. It also includes a modular parser built on `BeautifulSoup` for decomposing a SERP into list of components with categorical classifications and position-based specifications. -## Recent Updates - -Below are some details about recent updates. For a longer list, see the [Update Log](#update-log). - -`0.6.6` -- Update packages with dependabot alerts (brotli, urllib3) - -`0.6.5` -- Add GitHub Actions section to README - -`0.6.0` -- Method for collecting data with selenium; requests no longer works without a redirect -- Pull request [#72](https://github.com/gitronald/WebSearcher/pull/72) - ## Table of Contents - [WebSearcher](#websearcher) - - [Tools for conducting and parsing web searches](#tools-for-conducting-and-parsing-web-searches) - - [Recent Updates](#recent-updates) - - [Table of Contents](#table-of-contents) - [Getting Started](#getting-started) - [Usage](#usage) - [Example Search Script](#example-search-script) - [Step by Step](#step-by-step) - - [1. Initialize Collector](#1-initialize-collector) - - [2. Conduct a Search](#2-conduct-a-search) - - [3. Parse Search Results](#3-parse-search-results) - - [4. Save HTML and Metadata](#4-save-html-and-metadata) - - [5. Save Parsed Results](#5-save-parsed-results) - [Localization](#localization) - [Contributing](#contributing) - - [Repair or Enhance a Parser](#repair-or-enhance-a-parser) - - [Add a Parser](#add-a-parser) - - [Testing](#testing) - [GitHub Actions](#github-actions) + - [Recent Updates](#recent-updates) - [Update Log](#update-log) - [Similar Packages](#similar-packages) - [License](#license) ---- +--- +## Recent Updates + +### 0.6.7 + +- Added `get_text_by_selectors()` to `webutils` -- centralizes multi-selector fallback pattern across 7 component parsers +- Added `perspectives`, `recent_posts`, and `latest_from` component classifiers +- Added `sub_type` to perspectives parser from header text +- Added CI test workflow on push to dev branch +- Added compressed test fixtures with `condense_fixtures.py` script +- Updated dependency lower bounds for security patches (protobuf, orjson) +- Updated GitHub Actions to checkout v6 and setup-python v6 + +--- ## Getting Started ```bash -# Install pip version +# Install from PyPI pip install WebSearcher -# Install Github development version - less stable, more fun! +# Or install with Poetry +poetry add WebSearcher + +# Install development version from GitHub pip install git+https://github.com/gitronald/WebSearcher@dev ``` @@ -229,45 +221,68 @@ Happy to have help! If you see a component that we aren't covering yet, please a 3. Add new parser to imports and catalogue in `/component_parsers/__init__.py` ### Testing + Run tests: -``` -pytest +```bash +poetry run pytest tests/ -q ``` Update snapshots: -``` -pytest --snapshot-update +```bash +poetry run pytest tests/ --snapshot-update ``` -Running pytest with the `-vv` flag will show a diff of the snapshots that have changed: -``` -pytest -vv +Show snapshot diffs with `-vv`: +```bash +poetry run pytest tests/ -vv ``` -With the `-k` flag you can run a test for a specific html file: +Run a specific snapshot test by serp_id prefix: +```bash +poetry run pytest tests/ -k "45b6e019bfa2" ``` -pytest -k "1684837514.html" + +### Test Fixtures + +Tests load from compressed fixtures in `tests/fixtures/`. To update fixtures after collecting new demo data: + +```bash +poetry run python scripts/condense_fixtures.py 0.6.7 +poetry run pytest tests/ --snapshot-update ``` --- ## GitHub Actions -This repository uses GitHub Actions for automated publishing: +**Test Workflow** (`.github/workflows/test.yml`) +Runs the test suite on every push to `dev`. **Release Workflow** (`.github/workflows/publish.yml`) -Automatically publishes to PyPI when a pull request is merged into `master`. The workflow: -- Triggers on merged PRs to `master` +Publishes to PyPI when a pull request is merged into `master`: - Builds the package using Poetry -- Publishes to PyPI using trusted publishing (no API tokens required) +- Publishes using trusted publishing (no API tokens required) To release a new version: -1. Update the version in `pyproject.toml` -2. Create a PR to `master` -3. Once merged, the package is automatically published to PyPI +1. Merge `dev` into `master` via PR +2. Once merged, the package is automatically published to PyPI --- ## Update Log +`0.6.7` +- Add `get_text_by_selectors()` utility, CI test workflow, compressed test fixtures +- Add `perspectives`, `recent_posts`, `latest_from` classifiers and `sub_type` for perspectives +- Update dependency bounds for security patches, GitHub Actions to v6 + +`0.6.6` +- Update packages with dependabot alerts (brotli, urllib3) + +`0.6.5` +- Add GitHub Actions section to README + +`0.6.0` +- Method for collecting data with selenium; requests no longer works without a redirect +- Pull request [#72](https://github.com/gitronald/WebSearcher/pull/72) `0.5.2` - Added support for Spanish component headers by text @@ -376,7 +391,7 @@ Many of the packages I've found for collecting web search data via python are no --- ## License -Copyright (C) 2017-2024 Ronald E. Robertson +Copyright (C) 2017-2026 Ronald E. Robertson This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/WebSearcher/__init__.py b/WebSearcher/__init__.py index a828f2f..b99df28 100644 --- a/WebSearcher/__init__.py +++ b/WebSearcher/__init__.py @@ -1,4 +1,4 @@ -__version__ = "0.6.6" +__version__ = "0.6.7" from .searchers import SearchEngine from .parsers import parse_serp, FeatureExtractor from .extractors import Extractor diff --git a/WebSearcher/classifiers/header_text.py b/WebSearcher/classifiers/header_text.py index b247af8..61593ac 100644 --- a/WebSearcher/classifiers/header_text.py +++ b/WebSearcher/classifiers/header_text.py @@ -28,8 +28,7 @@ def _classify_header(cmpt: bs4.element.Tag, level: int) -> str: header_list = [] header_list.extend(cmpt.find_all(f"h{level}", {"role":"heading"})) header_list.extend(cmpt.find_all(f"h{level}", {"class":["O3JH7", "q8U8x", "mfMhoc"]})) - header_list.extend(cmpt.find_all("div", {"aria-level":f"{level}", "role":"heading"})) - header_list.extend(cmpt.find_all("div", {"aria-level":f"{level}", "class":"XmmGVd"})) + header_list.extend(cmpt.find_all(attrs={"aria-level": f"{level}", "role": "heading"})) # Check header text for known title matches for header in filter(None, header_list): @@ -83,7 +82,6 @@ def _get_header_level_mapping(level) -> dict: "Artworks", "Obras de arte", "Songs", "Canciones", "Albums", "Álbumes", - "What people are saying", "About", "Información", "Profiles", "Perfiles"], "local_news": ["Local news", "Noticias Locales"], @@ -101,8 +99,9 @@ def _get_header_level_mapping(level) -> dict: "Hotel"], "omitted_notice": ["Notices about Filtered Results"], "people_also_ask": ["People also ask", "Más preguntas"], - "perspectives": ["Perspectives & opinions", - "Perspectives"], + "perspectives": ["Perspectives & opinions", + "Perspectives", + "What people are saying"], "searches_related": ["Additional searches", "More searches", "Ver más", "Other searches", @@ -117,7 +116,8 @@ def _get_header_level_mapping(level) -> dict: "News", "Noticias", "Market news"], - "recent_posts": ["Recent posts"], + "recent_posts": ["Recent posts", + "Latest posts from"], "twitter": ["Twitter Results"], "videos": ["Videos"] } diff --git a/WebSearcher/classifiers/main.py b/WebSearcher/classifiers/main.py index 616a539..4d39164 100644 --- a/WebSearcher/classifiers/main.py +++ b/WebSearcher/classifiers/main.py @@ -43,10 +43,10 @@ def classify(cmpt: bs4.element.Tag) -> str: @staticmethod def discussions_and_forums(cmpt: bs4.element.Tag) -> str: - conditions = [ - cmpt.find("div", {"class": "IFnjPb", "role": "heading"}), - ] - return 'discussions_and_forums' if all(conditions) else "unknown" + heading = cmpt.find("div", {"class": "IFnjPb", "role": "heading"}) + if heading and heading.get_text(strip=True).startswith("Discussions and forums"): + return 'discussions_and_forums' + return "unknown" @staticmethod def available_on(cmpt: bs4.element.Tag) -> str: diff --git a/WebSearcher/component_parsers/ads.py b/WebSearcher/component_parsers/ads.py index 26d7480..356e87f 100644 --- a/WebSearcher/component_parsers/ads.py +++ b/WebSearcher/component_parsers/ads.py @@ -10,43 +10,23 @@ """ +import bs4 from .. import webutils +from ..models.data import BaseResult, DetailsItem, DetailsList from .shopping_ads import parse_shopping_ads -import bs4 -PARSED = { - 'type': 'ad', - 'sub_type': '', - 'sub_rank': 0, - 'title': '', - 'url': '', - 'cite': '', - 'text': '', -} +SUB_TYPES = [ + "legacy", + "secondary", + "standard", + "shopping", + "carousel", +] -def parse_ads(cmpt: bs4.element.Tag) -> list: - """Parse ads from ad component""" - - parsed_list = [] - sub_type = classify_ad_type(cmpt) - - if sub_type == 'legacy': - subs = cmpt.find_all('li', {'class': 'ads-ad'}) - parsed_list = [parse_ad_legacy(sub, sub_rank) for sub_rank, sub in enumerate(subs)] - elif sub_type == 'secondary': - subs = cmpt.find_all('li', {'class': 'ads-fr'}) - parsed_list = [parse_ad_secondary(sub, sub_rank) for sub_rank, sub in enumerate(subs)] - elif sub_type == 'standard': - subs = webutils.find_all_divs(cmpt, 'div', {'class': ['uEierd', 'commercial-unit-desktop-top']}) - for sub_rank, sub in enumerate(subs): - sub_classes = sub.attrs.get("class", []) - if "commercial-unit-desktop-top" in sub_classes: - parsed_list.extend(parse_shopping_ads(sub)) - elif "uEierd" in sub_classes: - parsed_list.append(parse_ad(sub, sub_rank=sub_rank)) - elif sub_type == 'carousel': - parsed_list = parse_ad_carousel(cmpt, sub_type) - return parsed_list +AD_STANDARD_TEXT_SELECTORS = [ + ('div', {'class': 'yDYNvb'}), + ('div', {'class': 'Va3FIb'}), +] def classify_ad_type(cmpt: bs4.element.Tag) -> str: @@ -54,7 +34,8 @@ def classify_ad_type(cmpt: bs4.element.Tag) -> str: label_divs = { "legacy": webutils.find_all_divs(cmpt, 'div', {'class': 'ad_cclk'}), "secondary": webutils.find_all_divs(cmpt, 'div', {'class': 'd5oMvf'}), - "standard": webutils.find_all_divs(cmpt, 'div', {'class': ['uEierd', 'commercial-unit-desktop-top']}), + "shopping": webutils.find_all_divs(cmpt, 'div', {'class': 'commercial-unit-desktop-top'}), + "standard": webutils.find_all_divs(cmpt, 'div', {'class': 'uEierd'}), "carousel": webutils.find_all_divs(cmpt, 'g-scrolling-carousel'), } for label, divs in label_divs.items(): @@ -63,143 +44,223 @@ def classify_ad_type(cmpt: bs4.element.Tag) -> str: return 'unknown' -def parse_ad_carousel(cmpt: bs4.element.Tag, sub_type: str, filter_visible: bool = True) -> list: +def parse_ads(cmpt: bs4.element.Tag) -> list: + """Parse ads from ad component""" - def parse_ad_carousel_div(sub: bs4.element.Tag, sub_type: str, sub_rank: int) -> dict: - """Parse ad carousel div, seen 2025-02-06""" - parsed = PARSED.copy() - parsed['sub_type'] = sub_type - parsed['sub_rank'] = sub_rank - parsed['title'] = webutils.get_text(sub, 'div', {'class':'e7SMre'}) - parsed['url'] = webutils.get_link(sub) - parsed['text'] = webutils.get_text(sub, 'div', {"class":"vrAZpb"}) - parsed['cite'] = webutils.get_text(sub, 'div', {"class":"zpIwr"}) - parsed['visible'] = not (sub.has_attr('data-has-shown') and sub['data-has-shown'] == 'false') - return parsed - - def parse_ad_carousel_card(sub: bs4.element.Tag, sub_type: str, sub_rank: int) -> dict: - """Parse ad carousel card, seen 2024-09-21""" - parsed = PARSED.copy() - parsed['sub_type'] = sub_type - parsed['sub_rank'] = sub_rank - parsed['title'] = webutils.get_text(sub, 'div', {'class':'gCv54b'}) - parsed['url'] = webutils.get_link(sub, {"class": "KTsHxd"}) - parsed['text'] = webutils.get_text(sub, 'div', {"class":"VHpBje"}) - parsed['cite'] = webutils.get_text(sub, 'div', {"class":"j958Pd"}) - parsed['visible'] = not (sub.has_attr('data-viewurl') and sub['data-viewurl']) - return parsed + subtype_parsers = { + 'legacy': parse_ad_legacy, + 'secondary': parse_ad_secondary, + 'shopping': parse_ad_shopping, + 'standard': parse_ad_standard, + 'carousel': parse_ad_carousel, + } + parsed_list = [] + sub_type = classify_ad_type(cmpt) + if sub_type in subtype_parsers: + parser = subtype_parsers.get(sub_type) + parsed_list = parser(cmpt) + return parsed_list - ad_carousel_parsers = [ - {'find_kwargs': {'name': 'g-inner-card'}, - 'parser': parse_ad_carousel_card}, - {'find_kwargs': {'name': 'div', 'attrs': {'class': 'ZPze1e'}}, - 'parser': parse_ad_carousel_div} - ] +# ------------------------------------------------------------------------------ - output_list = [] - ad_carousel = cmpt.find('g-scrolling-carousel') - if ad_carousel: - for parser_details in ad_carousel_parsers: - parser_func = parser_details['parser'] - kwargs = parser_details['find_kwargs'] - sub_cmpts = webutils.find_all_divs(ad_carousel, **kwargs) - if sub_cmpts: - for sub_rank, sub in enumerate(sub_cmpts): - parsed = parser_func(sub, sub_type, sub_rank) - output_list.append(parsed) +def parse_ad_legacy(cmpt: bs4.element.Tag) -> list: - if filter_visible: - output_list = [{k:v for k,v in x.items() if k != 'visible'} for x in output_list if x['visible']] - return output_list + def _parse_ad_legacy(cmpt: bs4.element.Tag) -> list: + subs = cmpt.find_all('li', {'class': 'ads-ad'}) + return [_parse_ad_legacy_sub(sub, sub_rank) for sub_rank, sub in enumerate(subs)] + + def _parse_ad_legacy_sub(sub: bs4.element.Tag, sub_rank: int) -> dict: + header = sub.find('div', {'class': 'ad_cclk'}) + parsed = BaseResult( + type='ad', + sub_type='legacy', + sub_rank=sub_rank, + title=webutils.get_text(header, 'h3'), + url=webutils.get_text(header, 'cite'), + cite=None, + text=webutils.get_text(sub, 'div', {'class': 'ads-creative'}), + details=_parse_ad_legacy_sub_details(sub), + error=None + ).model_dump() + return parsed + + def _parse_ad_legacy_sub_details(sub: bs4.element.Tag) -> list: + details_list = DetailsList() + bottom_text = sub.find('ul') + if bottom_text: + for li in bottom_text.find_all('li'): + details_list.append(DetailsItem(text=li.get_text(separator=' '))) + return details_list.to_dicts() + return _parse_ad_legacy(cmpt) -def parse_ad(sub: bs4.element.Tag, sub_rank: int = 0) -> dict: - """Parse details of a single ad subcomponent, similar to general""" - parsed = PARSED.copy() - parsed["sub_type"] = "standard" - parsed["sub_rank"] = sub_rank +# ------------------------------------------------------------------------------ - parsed['title'] = webutils.get_text(sub, 'div', {'role':'heading'}) - parsed['url'] = webutils.get_link(sub, {"class":"sVXRqc"}) - parsed['cite'] = webutils.get_text(sub, 'span', {"role":"text"}) +def parse_ad_secondary(cmpt: bs4.element.Tag) -> list: + + def _parse_ad_secondary(cmpt: bs4.element.Tag) -> list: + subs = cmpt.find_all('li', {'class': 'ads-fr'}) + return [_parse_ad_secondary_sub(sub, sub_rank) for sub_rank, sub in enumerate(subs)] + + def _parse_ad_secondary_sub(sub: bs4.element.Tag, sub_rank: int) -> dict: + return BaseResult( + type='ad', + sub_type='secondary', + sub_rank=sub_rank, + title=webutils.get_text(sub, 'div', {'role': 'heading'}), + url=_parse_ad_secondary_sub_url(sub), + cite=webutils.get_text(sub, 'span', {'class': 'gBIQub'}), + text=_parse_ad_secondary_sub_text(sub), + details=_parse_ad_secondary_sub_details(sub), + error=None + ).model_dump() - name_attrs = [{"name":"div", "attrs":{"class":"yDYNvb"}}, - {"name":"div", "attrs":{"class":"Va3FIb"}}] - for kwargs in name_attrs: - text = webutils.get_text(sub, **kwargs) - if text: - break - label = webutils.get_text(sub, 'span', {'class':'mXsQRe'}) - parsed['text'] = f"{text} " if label else text + def _parse_ad_secondary_sub_url(sub: bs4.element.Tag) -> str: + url_div = webutils.get_div(sub, 'div', {'class': 'd5oMvf'}) + return webutils.get_link(url_div) + + def _parse_ad_secondary_sub_text(sub) -> str: + text_divs = sub.find_all('div', {'class': 'yDYNvb'}) + return '|'.join([d.text for d in text_divs]) if text_divs else '' + + def _parse_ad_secondary_sub_details(sub: bs4.element.Tag) -> list: + for selector in [{'role': 'list'}, {'class': 'bOeY0b'}]: + details_section = sub.find('div', selector) + if details_section: + urls = webutils.get_link_list(details_section) + if urls: + details_list = DetailsList() + for url in urls: + details_list.append(DetailsItem(url=url)) + return details_list.to_dicts() + return None + + return _parse_ad_secondary(cmpt) - submenu = parse_ad_menu(sub) - if submenu: - parsed['sub_type'] = 'submenu' - parsed['details'] = submenu - return parsed +# ------------------------------------------------------------------------------ + +def parse_ad_shopping(cmpt: bs4.element.Tag) -> list: + """Parse shopping ads from component""" + subs = webutils.find_all_divs(cmpt, 'div', {'class': 'commercial-unit-desktop-top'}) + parsed_list = [] + for sub in subs: + parsed_list.extend(parse_shopping_ads(sub)) + return parsed_list + +# ------------------------------------------------------------------------------ + +def parse_ad_standard(cmpt: bs4.element.Tag) -> list: + """Parse standard ads from component""" + + def _parse_ad_standard_sub(sub: bs4.element.Tag, sub_rank: int = 0) -> dict: + + def _parse_ad_standard_text(sub: bs4.element.Tag) -> str: + text = webutils.get_text_by_selectors(sub, AD_STANDARD_TEXT_SELECTORS) + label = webutils.get_text(sub, 'span', {'class': 'mXsQRe'}) + return f"{text} " if label else text + + submenu = parse_ad_menu(sub) + sub_type = 'submenu' if submenu else 'standard' + parsed = BaseResult( + type='ad', + sub_type=sub_type, + sub_rank=sub_rank, + title=webutils.get_text(sub, 'div', {'role': 'heading'}), + url=webutils.get_link(sub, {'class': 'sVXRqc'}), + cite=webutils.get_text(sub, 'span', {'role': 'text'}), + text=_parse_ad_standard_text(sub), + details=submenu, + error=None + ).model_dump() + return parsed + + subs = webutils.find_all_divs(cmpt, 'div', {'class': 'uEierd'}) + return [_parse_ad_standard_sub(sub, sub_rank) for sub_rank, sub in enumerate(subs)] def parse_ad_menu(sub: bs4.element.Tag) -> list: """Parse menu items for a large ad with additional subresults""" - parsed_items = [] - menu_items = sub.find_all('div', {'class':'MhgNwc'}) + parsed_items = DetailsList() + menu_items = sub.find_all('div', {'class': 'MhgNwc'}) for item in menu_items: - parsed_item = {} - item_divs = item.find_all('div', {'class':'MUxGbd'}) + parsed_item = DetailsItem() + item_divs = item.find_all('div', {'class': 'MUxGbd'}) for div in item_divs: if webutils.check_dict_value(div.attrs, 'role', 'listitem'): - parsed_item['url'] = webutils.get_link(div) - parsed_item['title'] = webutils.get_text(div) + parsed_item.url = webutils.get_link(div) or '' + parsed_item.title = webutils.get_text(div) or '' else: - parsed_item['text'] = webutils.get_text(div) + parsed_item.text = webutils.get_text(div) or '' parsed_items.append(parsed_item) - return parsed_items if parsed_items else None + return parsed_items.to_dicts() -def parse_ad_secondary(sub: bs4.element.Tag, sub_rank: int = 0) -> dict: - """Parse details of a single ad subcomponent, similar to general""" - parsed = PARSED.copy() - parsed["sub_type"] = "secondary" - parsed["sub_rank"] = sub_rank +# ------------------------------------------------------------------------------ - parsed['title'] = webutils.get_text(sub, 'div', {'role':'heading'}) - link_div = sub.find('div', {'class':'d5oMvf'}) - parsed['url'] = webutils.get_link(link_div) if link_div else '' - parsed['cite'] = webutils.get_text(sub, 'span', {'class':'gBIQub'}) +def parse_ad_carousel( + cmpt: bs4.element.Tag, + sub_type: str = 'carousel', + filter_visible: bool = True + ) -> list: - # Take the top div with this class, should be main result abstract - text_divs = sub.find_all('div', {'class':'yDYNvb'}) - parsed['text'] = '|'.join([d.text for d in text_divs]) if text_divs else '' - - bottom_section = sub.find('div', {'role':'list'}) - if bottom_section: - list_items = sub.find_all('div', {'role':'listitem'}) - if list_items: - parsed['details'] = [i.find('a')['href'] for i in list_items] - - elif sub.find('div', {'class':'bOeY0b'}): - bottom_alinks = sub.find('div', {'class':'bOeY0b'}).find_all('a') - if bottom_alinks: - parsed['details'] = [a.attrs['href'] for a in bottom_alinks] - - return parsed - -def parse_ad_legacy(sub: bs4.element.Tag, sub_rank: int = 0) -> dict: - """[legacy] Parse details of a single ad subcomponent, similar to general""" - parsed = PARSED.copy() - parsed["sub_type"] = "legacy" - parsed["sub_rank"] = sub_rank - - header = sub.find('div', {'class':'ad_cclk'}) - parsed['title'] = webutils.get_text(header, 'h3') - parsed['url'] = webutils.get_text(header, 'cite') - parsed['text'] = webutils.get_text(sub, 'div', {'class':'ads-creative'}) - - bottom_text = sub.find('ul') - if bottom_text: - bottom_li = bottom_text.find_all('li') - parsed['details'] = [li.get_text(separator=' ') for li in bottom_li] + def is_visible_div(sub: bs4.element.Tag) -> bool: + """Check if carousel div is visible""" + return not (sub.has_attr('data-has-shown') and sub['data-has-shown'] == 'false') + + def is_visible_card(sub: bs4.element.Tag) -> bool: + """Check if carousel card is visible""" + return not (sub.has_attr('data-viewurl') and sub['data-viewurl']) + + def parse_ad_carousel_div(sub: bs4.element.Tag, sub_type: str, sub_rank: int) -> dict: + """Parse ad carousel div, seen 2025-02-06""" + return BaseResult( + type='ad', + sub_type=sub_type, + sub_rank=sub_rank, + title=webutils.get_text(sub, 'div', {'class': 'e7SMre'}), + url=webutils.get_link(sub), + text=webutils.get_text(sub, 'div', {'class': 'vrAZpb'}), + cite=webutils.get_text(sub, 'div', {'class': 'zpIwr'}), + details=None, + error=None + ).model_dump() + + def parse_ad_carousel_card(sub: bs4.element.Tag, sub_type: str, sub_rank: int) -> dict: + """Parse ad carousel card, seen 2024-09-21""" + return BaseResult( + type='ad', + sub_type=sub_type, + sub_rank=sub_rank, + title=webutils.get_text(sub, 'div', {'class': 'gCv54b'}), + url=webutils.get_link(sub, {'class': 'KTsHxd'}), + text=webutils.get_text(sub, 'div', {'class': 'VHpBje'}), + cite=webutils.get_text(sub, 'div', {'class': 'j958Pd'}), + details=None, + error=None + ).model_dump() + + # Possible ad carousel item types + output_list = [] + ad_carousel = cmpt.find('g-scrolling-carousel') + if ad_carousel: + ad_carousel_types = { + 'carousel_card': webutils.find_all_divs(ad_carousel, name='g-inner-card'), + 'carousel_div': webutils.find_all_divs(ad_carousel, name='div', attrs={'class': 'ZPze1e'}) + } - return parsed + for ad_carousel_type, sub_cmpts in ad_carousel_types.items(): + if sub_cmpts: + for sub_rank, sub in enumerate(sub_cmpts): + if ad_carousel_type == 'carousel_card': + if filter_visible and not is_visible_card(sub): + continue + output = parse_ad_carousel_card(sub, sub_type, sub_rank) + elif ad_carousel_type == 'carousel_div': + if filter_visible and not is_visible_div(sub): + continue + output = parse_ad_carousel_div(sub, sub_type, sub_rank) + output_list.append(output) + + return output_list diff --git a/WebSearcher/component_parsers/available_on.py b/WebSearcher/component_parsers/available_on.py index 00044eb..dd843ee 100644 --- a/WebSearcher/component_parsers/available_on.py +++ b/WebSearcher/component_parsers/available_on.py @@ -1,26 +1,40 @@ +from ..models.data import DetailsItem, DetailsList + + def parse_available_on(cmpt, sub_rank=0) -> list: """Parse an available component These components contain a carousel of thumbnail images with links to - entertainment relevant to query - + entertainment relevant to query + Args: cmpt (bs4 object): An available on component - + Returns: dict : parsed component """ - parsed = {'type':'available_on', 'sub_rank':sub_rank} - - parsed['title'] = cmpt.find('span', {'class':'GzssTd'}).text - - details = [] - options = cmpt.find_all('div', {'class':'kno-fb-ctx'}) - for o in options: - option = {} - option['title'] = o.find('div', {'class':'i3LlFf'}).text - option['cost'] = o.find('div', {'class':'V8xno'}).text - option['url'] = o.find('a')['href'] - details.append(option) - parsed['details'] = details + parsed = {'type': 'available_on', 'sub_rank': sub_rank} + + parsed['title'] = cmpt.find('span', {'class': 'GzssTd'}).text + + details = DetailsList() + for o in cmpt.find_all('div', {'class': 'kno-fb-ctx'}): + details.append(parse_available_on_item(o)) + parsed['details'] = details.to_dicts() return [parsed] + + +def parse_available_on_item(sub) -> DetailsItem: + """Parse an available on item + + Args: + sub (bs4 object): An available on option element + + Returns: + DetailsItem : parsed item with title, url, and cost in misc + """ + return DetailsItem( + title=sub.find('div', {'class': 'i3LlFf'}).text, + url=sub.find('a')['href'], + misc={'cost': sub.find('div', {'class': 'V8xno'}).text}, + ) diff --git a/WebSearcher/component_parsers/discussions_and_forums.py b/WebSearcher/component_parsers/discussions_and_forums.py index 9547b8c..0768160 100644 --- a/WebSearcher/component_parsers/discussions_and_forums.py +++ b/WebSearcher/component_parsers/discussions_and_forums.py @@ -1,6 +1,16 @@ from .. import webutils import bs4 +TITLE_SELECTORS = [ + ('div', {'class': 'zNWc4c'}), + ('div', {'class': 'qyp6xb'}), +] + +CITE_SELECTORS = [ + ('div', {'class': 'LbKnXb'}), + ('div', {'class': 'VZGVuc'}), +] + def parse_discussions_and_forums(cmpt:bs4.element.Tag) -> list: """Parse a 'Discussions and forums' component""" @@ -14,9 +24,9 @@ def parse_discussions_and_forums_item(cmpt:bs4.element.Tag, sub_rank:int = 0) -> "type": "discussions_and_forums", "sub_type": None, "sub_rank": sub_rank, - "title": get_title(cmpt), + "title": webutils.get_text_by_selectors(cmpt, TITLE_SELECTORS), "url": get_url(cmpt), - "cite": get_cite(cmpt) + "cite": webutils.get_text_by_selectors(cmpt, CITE_SELECTORS) } def get_url(sub): @@ -26,16 +36,3 @@ def get_url(sub): url_list = [url for url in url_list if url] return url_list[0] if url_list else None -def get_title(sub): - """Get title from a subcomponent; try multiple, take first non-null""" - title_list = [webutils.get_text(sub, 'div', {'class':'zNWc4c'}), - webutils.get_text(sub, 'div', {'class':'qyp6xb'})] - title_list = [title for title in title_list if title] - return title_list[0] if title_list else None - -def get_cite(sub): - """Get cite from a subcomponent; try multiple, take first non-null""" - cite_list = [webutils.get_text(sub, 'div', {'class':'LbKnXb'}), - webutils.get_text(sub, 'div', {'class':'VZGVuc'})] - cite_list = [cite for cite in cite_list if cite] - return cite_list[0] if cite_list else None diff --git a/WebSearcher/component_parsers/footer.py b/WebSearcher/component_parsers/footer.py index e45a044..2b858a7 100644 --- a/WebSearcher/component_parsers/footer.py +++ b/WebSearcher/component_parsers/footer.py @@ -1,4 +1,5 @@ from .. import webutils +from ..models.data import DetailsItem, DetailsList class Footer: @@ -13,7 +14,10 @@ def parse_image_card(sub, sub_rank=0) -> dict: parsed['title'] = webutils.get_text(sub, "div", {'aria-level':"3", "role":"heading"}) images = sub.find_all('img') if images: - parsed['details'] = [{'text':i['alt'], 'url':i['src']} for i in images] + details = DetailsList() + for i in images: + details.append(DetailsItem(url=i['src'], text=i['alt'])) + parsed['details'] = details.to_dicts() return parsed @staticmethod diff --git a/WebSearcher/component_parsers/general.py b/WebSearcher/component_parsers/general.py index 45dfecc..6f36921 100644 --- a/WebSearcher/component_parsers/general.py +++ b/WebSearcher/component_parsers/general.py @@ -1,4 +1,5 @@ import re +from ..models.data import DetailsItem, DetailsList from ..webutils import get_text, get_link def parse_general_results(cmpt) -> list: @@ -7,38 +8,40 @@ def parse_general_results(cmpt) -> list: The ubiquitous blue title, green citation, and black text summary results. Sometimes grouped into components of multiple general results. The subcomponent general results tend to have a similar theme. - + Args: cmpt (bs4 object): A general component - + Returns: list : list of parsed subcomponent dictionaries """ + subs = find_subcomponents(cmpt) + return [parse_general_result(sub, sub_rank) for sub_rank, sub in enumerate(subs)] - # Legacy compatibility - subs = cmpt.find_all('div', {'class':'g'}) - # 2023.05.09 - finds subs +def find_subcomponents(cmpt) -> list: + """Find subcomponents within a general component, trying known formats""" + + # Standard format + subs = cmpt.find_all('div', {'class': 'g'}) + if subs: + parent_g = cmpt.find('div', {'class': 'g'}) + if parent_g and parent_g.find_all('div', {'class': 'g'}): + return [parent_g] # Nested .g dedup + return subs + + # Sub-results format (2023+) additional = cmpt.find_all('div', {'class': 'd4rhi'}) if additional: - # Catch general_subresult - # this means that there is a sub-element, with class d4rhi - # the first div child of the div.g is the first sub element - first = cmpt.find('div') - subs = [first] + additional - - # 2023.05.09 - handles duplicate .g tags within one component - if cmpt.find('div', {'class':'g'}): - parent_g = cmpt.find('div', {'class':'g'}) - if parent_g.find_all('div', {'class':'g'}): - # this means that there is a .g element inside of another .g element, - # and it would otherwise get double-counted - # we just want to keep the parent .g element in this case - subs = [parent_g] - subs = subs if subs else [cmpt] - - parsed_list = [parse_general_result(sub, sub_rank) for sub_rank, sub in enumerate(subs)] - return parsed_list + return [cmpt.find('div')] + additional + + # Video results + subs = cmpt.find_all('div', {'class': 'PmEWq'}) + if subs: + return subs + + # Fallback: treat entire component as single result + return [cmpt] def parse_general_result(sub, sub_rank=0) -> dict: @@ -72,12 +75,16 @@ def parse_general_result(sub, sub_rank=0) -> dict: return parsed -def parse_alink(a): - return {'text':a.text,'url':a.attrs['href']} +def parse_alink(a): + return DetailsItem(url=a.attrs['href'], text=a.text) def parse_alink_list(alinks): - return [parse_alink(a) for a in alinks if 'href' in a.attrs] + details = DetailsList() + for a in alinks: + if 'href' in a.attrs: + details.append(parse_alink(a)) + return details.to_dicts() def parse_subtype_details(sub, parsed) -> dict: diff --git a/WebSearcher/component_parsers/knowledge.py b/WebSearcher/component_parsers/knowledge.py index b9273fb..d5ea007 100644 --- a/WebSearcher/component_parsers/knowledge.py +++ b/WebSearcher/component_parsers/knowledge.py @@ -1,4 +1,5 @@ from .. import webutils +from ..models.data import DetailsItem, DetailsList from .general import parse_general_result @@ -30,11 +31,14 @@ def parse_knowledge_panel(cmpt, sub_rank=0) -> list: alinks = cmpt.find_all('a') if alinks: - details['urls'] = [ - parse_alink(a) - for a in alinks - if 'href' in a.attrs and a['href'] != '#' - ] + urls = DetailsList() + seen_urls = set() + for a in alinks: + if 'href' in a.attrs and a['href'] != '#': + if a['href'] not in seen_urls: + seen_urls.add(a['href']) + urls.append(parse_alink(a)) + details['urls'] = urls.to_dicts() # Get all text if cmpt.find("div", {"class": "Fzsovc"}): @@ -120,4 +124,4 @@ def get_text(div): return '|'.join([d.get_text(separator=' ') for d in div if d.text]) def parse_alink(a): - return {'url': a['href'], 'text': a.get_text('|')} + return DetailsItem(url=a['href'], text=a.get_text('|')) diff --git a/WebSearcher/component_parsers/knowledge_rhs.py b/WebSearcher/component_parsers/knowledge_rhs.py index bab65c1..d5d5173 100644 --- a/WebSearcher/component_parsers/knowledge_rhs.py +++ b/WebSearcher/component_parsers/knowledge_rhs.py @@ -1,3 +1,6 @@ +from ..models.data import DetailsItem, DetailsList + + def parse_knowledge_rhs(cmpt, sub_rank=0) -> list: """Parse the Right-Hand-Side Knowledge Panel @@ -72,9 +75,11 @@ def parse_knowledge_rhs_main(cmpt, sub_rank=0) -> list: if description.parent.previous_sibling: alinks += description.parent.previous_sibling.find_all('a') if len(alinks) > 1: # 1st match has main description - parsed['details']['urls'] = [ - parse_alink(a) for a in alinks[1:] if 'href' in a.attrs - ] + urls = DetailsList() + for a in alinks[1:]: + if 'href' in a.attrs: + urls.append(parse_alink(a)) + parsed['details']['urls'] = urls.to_dicts() if not len(parsed['details']): parsed['details'] = None @@ -100,10 +105,14 @@ def parse_knowledge_rhs_sub(sub, sub_rank=0) -> dict: alinks = sub.find_all('a') if alinks: - parsed['details'] = [parse_alink(a) for a in alinks if 'href' in a.attrs] + details = DetailsList() + for a in alinks: + if 'href' in a.attrs: + details.append(parse_alink(a)) + parsed['details'] = details.to_dicts() return parsed def parse_alink(a): - return {'url': a['href'], 'text': a.text} + return DetailsItem(url=a['href'], text=a.text) diff --git a/WebSearcher/component_parsers/local_results.py b/WebSearcher/component_parsers/local_results.py index 7cfb2d1..372f7f7 100644 --- a/WebSearcher/component_parsers/local_results.py +++ b/WebSearcher/component_parsers/local_results.py @@ -1,6 +1,12 @@ from .. import utils from .. import webutils +HEADER_SELECTORS = [ + ("h2", {"role": "heading"}), + ("div", {"aria-level": "2", "role": "heading"}), +] + + def parse_local_results(cmpt) -> list: """Parse a "Local Results" component @@ -19,13 +25,9 @@ def parse_local_results(cmpt) -> list: if parsed_list: # Set first non-empty header as sub_type (e.g. "Places" -> places) - header_list = [ - webutils.get_text(cmpt, "h2", {"role":"heading"}), - webutils.get_text(cmpt, 'div', {'aria-level':"2", "role":"heading"}), - ] - header_list = list(filter(None, header_list)) - if header_list: - sub_type = str(header_list[0]).lower().replace(" ", "_") + header = webutils.get_text_by_selectors(cmpt, HEADER_SELECTORS) + if header: + sub_type = header.lower().replace(" ", "_") for parsed in parsed_list: parsed.update({'sub_type':sub_type}) diff --git a/WebSearcher/component_parsers/map_results.py b/WebSearcher/component_parsers/map_results.py index c21572b..678e431 100644 --- a/WebSearcher/component_parsers/map_results.py +++ b/WebSearcher/component_parsers/map_results.py @@ -1,4 +1,9 @@ -from .. import webutils +from .. import webutils + +TITLE_SELECTORS = [ + ('div', {'class': 'aiAXrc'}), +] + def parse_map_results(cmpt, sub_rank=0) -> list: """Parse a "Map Results" component @@ -15,9 +20,5 @@ def parse_map_results(cmpt, sub_rank=0) -> list: return [{ 'type': 'map_results', 'sub_rank': sub_rank, - 'title': get_title(cmpt) + 'title': webutils.get_text_by_selectors(cmpt, TITLE_SELECTORS) }] - -def get_title(cmpt): - # return webutils.get_text(cmpt, 'div', {'class':'desktop-title-content'}) - return webutils.get_text(cmpt, 'div', {'class':'aiAXrc'}) \ No newline at end of file diff --git a/WebSearcher/component_parsers/people_also_ask.py b/WebSearcher/component_parsers/people_also_ask.py index 7397ce8..e1b1634 100644 --- a/WebSearcher/component_parsers/people_also_ask.py +++ b/WebSearcher/component_parsers/people_also_ask.py @@ -1,5 +1,14 @@ from .. import webutils +QUESTION_SELECTORS = [ + ('div', {'class': 'rc'}), + ('div', {'class': 'yuRUbf'}), + ('div', {'class': 'iDjcJe'}), # 2023-01-01 + ('div', {'class': 'JlqpRe'}), # 2023-11-16 + ('div', {'class': 'cbphWd'}), # 2021-01-09 +] + + def parse_people_also_ask(cmpt, sub_rank=0) -> list: """Parse a "People Also Ask" component @@ -31,19 +40,4 @@ def parse_people_also_ask(cmpt, sub_rank=0) -> list: def parse_question(question): """Parse an individual question in a "People Also Ask" component""" - - title_divs = [ - question.find('div', {'class':'rc'}), - question.find('div', {'class':'yuRUbf'}), - question.find('div', {'class':'iDjcJe'}), # 2023-01-01 - question.find('div', {'class':'JlqpRe'}), # 2023-11-16 - question.find('div', {'class':'cbphWd'}), # 2021-01-09 - ] - - # Return first valid text found - for title_div in filter(None, title_divs): - text = webutils.get_text(title_div, strip=True) - if text: - return text - - return None \ No newline at end of file + return webutils.get_text_by_selectors(question, QUESTION_SELECTORS, strip=True) \ No newline at end of file diff --git a/WebSearcher/component_parsers/perspectives.py b/WebSearcher/component_parsers/perspectives.py index 2029400..70f6d28 100644 --- a/WebSearcher/component_parsers/perspectives.py +++ b/WebSearcher/component_parsers/perspectives.py @@ -4,11 +4,20 @@ def parse_perspectives(cmpt): """Parse a "Perspectives & opinions" component These components are the same as Top Stories, but have a different heading. - + Args: cmpt (bs4 object): A latest from component - + Returns: dict : parsed result """ - return parse_top_stories(cmpt, ctype='perspectives') + # Extract header text as sub_type (e.g. "What people are saying" -> "what_people_are_saying") + header = cmpt.find(attrs={"aria-level": "2", "role": "heading"}) + if not header: + header = cmpt.find("h2", {"role": "heading"}) + sub_type = header.text.strip().lower().replace(" ", "_") if header else None + + results = parse_top_stories(cmpt, ctype='perspectives') + for result in results: + result['sub_type'] = sub_type + return results diff --git a/WebSearcher/component_parsers/searches_related.py b/WebSearcher/component_parsers/searches_related.py index 2c2d96c..817d1e8 100644 --- a/WebSearcher/component_parsers/searches_related.py +++ b/WebSearcher/component_parsers/searches_related.py @@ -1,5 +1,11 @@ from .. import webutils +HEADER_SELECTORS = [ + ("h2", {"role": "heading"}), + ("div", {"aria-level": "2", "role": "heading"}), +] + + def parse_searches_related(cmpt, sub_rank=0) -> list: """Parse a one or two column list of related search queries""" @@ -9,12 +15,8 @@ def parse_searches_related(cmpt, sub_rank=0) -> list: 'url': None} # Set first non-empty header as sub_type (e.g. "Additional searches" -> additional_searches) - header_list = [ - webutils.get_text(cmpt, "h2", {"role":"heading"}), - webutils.get_text(cmpt, 'div', {'aria-level':"2", "role":"heading"}), - ] - header_list = list(filter(None, header_list)) - parsed['sub_type'] = str(header_list[0]).lower().replace(" ", "_") if header_list else None + header = webutils.get_text_by_selectors(cmpt, HEADER_SELECTORS) + parsed['sub_type'] = header.lower().replace(" ", "_") if header else None output_list = [] diff --git a/WebSearcher/component_parsers/top_image_carousel.py b/WebSearcher/component_parsers/top_image_carousel.py index 4f70633..fcf37f7 100644 --- a/WebSearcher/component_parsers/top_image_carousel.py +++ b/WebSearcher/component_parsers/top_image_carousel.py @@ -1,4 +1,6 @@ from .. import webutils +from ..models.data import DetailsItem, DetailsList + def parse_top_image_carousel(cmpt, sub_rank=0) -> list: """parse image carousel that appears at top of page above search results @@ -23,17 +25,14 @@ def parse_top_image_carousel(cmpt, sub_rank=0) -> list: else: alinks = cmpt.find('g-scrolling-carousel').find_all('a') - parsed['details'] = [ - parse_alink(a) for a in alinks - if 'href' in a.attrs or 'data-url' in a.attrs - ] + details = DetailsList() + for a in alinks: + if 'href' in a.attrs or 'data-url' in a.attrs: + details.append(parse_alink(a)) + parsed['details'] = details.to_dicts() return [parsed] -def parse_alink(a): - parsed = {'text': a.get_text('|')} - if 'href' in a.attrs: - parsed['url'] = a['href'] - elif 'data-url' in a.attrs: - parsed['url'] = a['data-url'] - return parsed +def parse_alink(a): + url = a.attrs.get('href') or a.attrs.get('data-url', '') + return DetailsItem(url=url, text=a.get_text('|')) diff --git a/WebSearcher/component_parsers/top_stories.py b/WebSearcher/component_parsers/top_stories.py index 0df90a6..f54b692 100644 --- a/WebSearcher/component_parsers/top_stories.py +++ b/WebSearcher/component_parsers/top_stories.py @@ -1,4 +1,9 @@ -from ..webutils import find_all_divs, find_children, get_text, get_link +from ..webutils import find_all_divs, find_children, get_text, get_text_by_selectors, get_link + +TITLE_SELECTORS = [ + ('div', {'class': 'n0jPhd'}), # Top Stories + ('div', {'class': 'eAaXgc'}), # Perspectives +] def parse_top_stories(cmpt, ctype='top_stories') -> list: @@ -41,7 +46,7 @@ def parse_top_story(sub, ctype, sub_rank=0) -> dict: parsed = { 'type': ctype, 'sub_rank': sub_rank, - 'title': get_text(sub, 'div', {'class':'n0jPhd'}), + 'title': get_text_by_selectors(sub, TITLE_SELECTORS), 'url': get_link(sub, key='href'), 'text': get_text(sub, "div", {'class': "GI74Re"}), 'cite': get_cite(sub) @@ -49,6 +54,7 @@ def parse_top_story(sub, ctype, sub_rank=0) -> dict: return parsed + def get_cite(sub): div_cite = sub.find("div", {'class': 'Dx69l'}) diff --git a/WebSearcher/components.py b/WebSearcher/components.py index d757a65..1b13594 100644 --- a/WebSearcher/components.py +++ b/WebSearcher/components.py @@ -7,27 +7,43 @@ import bs4 import traceback -from typing import Dict +from collections.abc import Callable class Component: - def __init__(self, elem: bs4.element.Tag, section="unknown", type="unknown", cmpt_rank=None): - self.elem: bs4.element.Tag = elem - self.section: str = section + """A SERP component extracted from HTML""" + + def __init__( + self, + elem: bs4.element.Tag, + section: str = "unknown", + type: str = "unknown", + cmpt_rank: int | None = None + ) -> None: + """Initialize a Component + + Args: + elem: The BeautifulSoup Tag element containing the component HTML + section: The SERP section (header, main, footer, rhs) + type: The component type (e.g., general, ads, top_stories) + cmpt_rank: The component's rank position on the SERP + """ + self.elem = elem + self.section = section self.type = type self.cmpt_rank = cmpt_rank - self.result_list = [] + self.result_list: list[dict] = [] self.result_counter = 0 def __str__(self) -> str: return str(vars(self)) - def to_dict(self) -> Dict: + def to_dict(self) -> dict: return self.__dict__ - - def get_metadata(self, key_filter=["section", "cmpt_rank"]) -> Dict: + + def get_metadata(self, key_filter=["section", "cmpt_rank"]) -> dict: return {k:v for k,v in self.to_dict().items() if k in key_filter} - def classify_component(self, classify_type_func: callable = None): + def classify_component(self, classify_type_func: Callable | None = None): """Classify the component type""" if classify_type_func: self.type = classify_type_func(self.elem) @@ -41,7 +57,7 @@ def classify_component(self, classify_type_func: callable = None): elif self.section == "footer": self.type = ClassifyFooter.classify(self.elem) - def select_parser(self, parser_type_func: callable = None) -> callable: + def select_parser(self, parser_type_func: Callable | None = None) -> Callable: if parser_type_func: parser_func = parser_type_func else: @@ -57,7 +73,7 @@ def select_parser(self, parser_type_func: callable = None) -> callable: parser_func = parse_not_implemented return parser_func - def run_parser(self, parser_func: callable) -> list: + def run_parser(self, parser_func: Callable) -> list: log.debug(f"parsing: {self.cmpt_rank} | {self.section} | {self.type}") try: if parser_func in {parse_unknown, parse_not_implemented}: @@ -68,7 +84,7 @@ def run_parser(self, parser_func: callable) -> list: parsed_list = self.create_parsed_list_error("parsing exception", is_exception=True) return parsed_list - def parse_component(self, parser_type_func: callable = None): + def parse_component(self, parser_type_func: Callable | None = None): if not self.type: parsed_list = self.create_parsed_list_error("null component type") diff --git a/WebSearcher/locations.py b/WebSearcher/locations.py index c9fcbe1..469051c 100644 --- a/WebSearcher/locations.py +++ b/WebSearcher/locations.py @@ -5,7 +5,7 @@ import zipfile import requests from google.protobuf.internal import decoder, encoder # poetry add protobuf -from typing import Dict, Union, Any +from typing import Any from . import logger from . import webutils as wu @@ -23,7 +23,7 @@ def convert_canonical_name_to_uule(canon_name: str) -> str: return f'w+{encoded_string}' -def encode_protobuf_string(fields: Dict[int, Union[str, int]]) -> str: +def encode_protobuf_string(fields: dict[int, str | int]) -> str: """ Encode a dictionary of field numbers and values into a base64-encoded protobuf string. Args: fields: A dictionary where keys are protobuf field numbers and values are the data to encode @@ -47,7 +47,7 @@ def encode_protobuf_string(fields: Dict[int, Union[str, int]]) -> str: return base64.b64encode(bytes(encoded)).decode('utf-8') # Convert to base64 and decode to string -def decode_protobuf_string(encoded_string: str) -> Dict[int, Any]: +def decode_protobuf_string(encoded_string: str) -> dict[int, Any]: """ Decode a base64-encoded protobuf string into a dictionary of field numbers and values. Args: encoded_string: A base64-encoded protobuf message diff --git a/WebSearcher/logger.py b/WebSearcher/logger.py index 147000d..44fa771 100644 --- a/WebSearcher/logger.py +++ b/WebSearcher/logger.py @@ -2,7 +2,6 @@ """ import logging.config -from typing import Optional # Setting LOG_LEVEL_DEFAULT = 'INFO' @@ -98,6 +97,6 @@ def __init__(self, 'loggers': loggers } - def start(self, name: Optional[str] = __name__) -> logging.Logger: + def start(self, name: str | None = __name__) -> logging.Logger: logging.config.dictConfig(self.log_config) return logging.getLogger(name) \ No newline at end of file diff --git a/WebSearcher/models/cmpt_mappings.py b/WebSearcher/models/cmpt_mappings.py index 616bbad..404d299 100644 --- a/WebSearcher/models/cmpt_mappings.py +++ b/WebSearcher/models/cmpt_mappings.py @@ -109,7 +109,9 @@ "description": "Related questions that people search for", "sub_types": [], }, - "perspectives": {"description": "Opinion and perspective results", "sub_types": []}, + "perspectives": {"description": "Opinion and perspective results", "sub_types": [ + "perspectives", "perspectives_&_opinions", "what_people_are_saying", + ]}, "scholarly_articles": {"description": "Google Scholar results", "sub_types": []}, "searches_related": { "description": "Related search terms", diff --git a/WebSearcher/models/configs.py b/WebSearcher/models/configs.py index 5d6ea80..a408e56 100644 --- a/WebSearcher/models/configs.py +++ b/WebSearcher/models/configs.py @@ -1,7 +1,6 @@ import requests import subprocess from enum import Enum -from typing import Dict, Optional, Union from pydantic import BaseModel, Field, computed_field class BaseConfig(BaseModel): @@ -25,13 +24,13 @@ class LogConfig(BaseConfig): class SeleniumConfig(BaseConfig): headless: bool = False - version_main: int = 141 + version_main: int = 144 use_subprocess: bool = False driver_executable_path: str = "" class RequestsConfig(BaseConfig): model_config = {"arbitrary_types_allowed": True} - headers: Dict[str, str] = Field(default_factory=lambda: { + headers: dict[str, str] = Field(default_factory=lambda: { 'Host': 'www.google.com', 'Referer': 'https://www.google.com/', 'Accept': '*/*', @@ -39,7 +38,7 @@ class RequestsConfig(BaseConfig): 'Accept-Language': 'en-US,en;q=0.5', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/118.0', }) - ssh_tunnel: Optional[subprocess.Popen] = None + ssh_tunnel: subprocess.Popen | None = None unzip: bool = True @computed_field @@ -69,7 +68,7 @@ def create(cls, method=None): raise TypeError(f"Expected string or SearchMethod, got {type(method)}") class SearchConfig(BaseConfig): - method: Union[str, SearchMethod] = SearchMethod.SELENIUM + method: str | SearchMethod = SearchMethod.SELENIUM log: LogConfig = Field(default_factory=LogConfig) selenium: SeleniumConfig = Field(default_factory=SeleniumConfig) requests: RequestsConfig = Field(default_factory=RequestsConfig) diff --git a/WebSearcher/models/data.py b/WebSearcher/models/data.py index 45c4bef..a854228 100644 --- a/WebSearcher/models/data.py +++ b/WebSearcher/models/data.py @@ -1,5 +1,29 @@ from pydantic import BaseModel, Field -from typing import Any, Optional, List, Dict +from typing import Any +from dataclasses import asdict, dataclass, field + +@dataclass +class DetailsItem: + """Represents a details item within a search result.""" + url: str = '' + title: str = '' + text: str = '' + misc: dict = field(default_factory=dict) + + def to_dict(self) -> dict: + return asdict(self) + + +class DetailsList(list): + """A list of DetailsItem objects with conversion to dicts.""" + + def append(self, item: DetailsItem): + if not isinstance(item, DetailsItem): + raise TypeError(f"Expected DetailsItem, got {type(item).__name__}") + super().append(item) + + def to_dicts(self) -> list[dict]: + return [item.to_dict() for item in self] class BaseResult(BaseModel): @@ -11,13 +35,13 @@ class BaseResult(BaseModel): """ sub_rank: int = Field(0, description="Position within a results component") type: str = Field('unclassified', description="Result type (general, ad, etc.)") - sub_type: Optional[str] = Field(None, description="Result sub-type (e.g., header, item)") - title: Optional[str] = Field(None, description="Title of the search result") - url: Optional[str] = Field(None, description="URL of the search result") - text: Optional[str] = Field(None, description="Snippet text from the search result") - cite: Optional[str] = Field(None, description="Citation or source information") - details: Optional[Any] = Field(None, description="Additional structured details specific to result type") - error: Optional[str] = Field(None, description="Error message if result parsing failed") + sub_type: str | None = Field(None, description="Result sub-type (e.g., header, item)") + title: str | None = Field(None, description="Title of the search result") + url: str | None = Field(None, description="URL of the search result") + text: str | None = Field(None, description="Snippet text from the search result") + cite: str | None = Field(None, description="Citation or source information") + details: Any | None = Field(None, description="Additional structured details specific to result type") + error: str | None = Field(None, description="Error message if result parsing failed") class BaseSERP(BaseModel): @@ -28,8 +52,8 @@ class BaseSERP(BaseModel): raw HTML response, metadata about the request, and identifiers for tracking. """ qry: str = Field(..., description="Search query") - loc: Optional[str] = Field(None, description="Location if set, in Canonical Name format") - lang: Optional[str] = Field(None, description="Language code if set") + loc: str | None = Field(None, description="Location if set, in Canonical Name format") + lang: str | None = Field(None, description="Language code if set") url: str = Field(..., description="URL of the SERP") html: str = Field(..., description="Raw HTML of the SERP") timestamp: str = Field(..., description="ISO format timestamp of the crawl") diff --git a/WebSearcher/models/searches.py b/WebSearcher/models/searches.py index b213e3d..8335656 100644 --- a/WebSearcher/models/searches.py +++ b/WebSearcher/models/searches.py @@ -1,5 +1,5 @@ from pydantic import Field, computed_field -from typing import Dict, Optional, Any, List +from typing import Any from datetime import datetime from ..utils import hash_id @@ -11,15 +11,15 @@ class SearchParams(BaseConfig): """Contains parameters for a search request and utility methods for URL generation""" qry: str = Field('', description="The search query text") - num_results: Optional[int] = Field(None, description="Number of results to return") - lang: Optional[str] = Field(None, description="Language code (e.g., 'en')") - loc: Optional[str] = Field(None, description="Location in Canonical Name format") + num_results: int | None = Field(None, description="Number of results to return") + lang: str | None = Field(None, description="Language code (e.g., 'en')") + loc: str | None = Field(None, description="Location in Canonical Name format") base_url: str = Field("https://www.google.com/search", description="Base search engine URL") ai_expand: bool = Field(False, description="Expand AI overviews if present") - headers: Dict[str, str] = Field(default_factory=dict, description="Custom headers") - + headers: dict[str, str] = Field(default_factory=dict, description="Custom headers") + @computed_field - def url_params(self) -> Dict[str, Any]: + def url_params(self) -> dict[str, Any]: """Generates a dictionary of URL parameters based on the search parameters""" params = {'q': wu.encode_param_value(self.qry)} opt_params = { @@ -40,7 +40,7 @@ def url(self) -> str: def serp_id(self) -> str: return hash_id(f"{self.qry}{self.loc}{datetime.now().isoformat()}") - def to_serp_output(self) -> Dict[str, Any]: + def to_serp_output(self) -> dict[str, Any]: return { "qry": self.qry, "loc": self.loc, diff --git a/WebSearcher/parsers.py b/WebSearcher/parsers.py index 7f5fb0f..b42909c 100644 --- a/WebSearcher/parsers.py +++ b/WebSearcher/parsers.py @@ -5,22 +5,21 @@ import re from bs4 import BeautifulSoup -from typing import Union, List, Dict, Tuple def parse_serp( - serp: Union[str, BeautifulSoup], + serp: str | BeautifulSoup, extract_features: bool = False - ) -> Union[List[Dict], Tuple[List[Dict], Dict]]: + ) -> list[dict] | dict: """Parse a Search Engine Result Page (SERP) - + Args: - serp (Union[str, BeautifulSoup]): The HTML content of the SERP or a BeautifulSoup object - extract_features (bool, optional): Whether to also extract SERP features. Defaults to False. - + serp: The HTML content of the SERP or a BeautifulSoup object + extract_features: Whether to also extract SERP features. Defaults to False. + Returns: - Union[List[Dict], Tuple[List[Dict], Dict]]: If extract_features is False, returns a list of result components. - If extract_features is True, returns a tuple of (results, features). + If extract_features is False, returns a list of result components. + If extract_features is True, returns a dict with 'results' and 'features' keys. """ # Extract components soup = webutils.make_soup(serp) @@ -45,14 +44,14 @@ def parse_serp( class FeatureExtractor: @staticmethod - def extract_features(html_or_soup: Union[str, BeautifulSoup]) -> dict: + def extract_features(html_or_soup: str | BeautifulSoup) -> dict: """Extract SERP features from HTML or a BeautifulSoup object - + Args: - html_or_soup (Union[str, BeautifulSoup]): The HTML content or a BeautifulSoup object - + html_or_soup: The HTML content or a BeautifulSoup object + Returns: - dict: The extracted features + The extracted features """ output = {} diff --git a/WebSearcher/searchers.py b/WebSearcher/searchers.py index 2147b39..89bd0f1 100644 --- a/WebSearcher/searchers.py +++ b/WebSearcher/searchers.py @@ -11,28 +11,27 @@ import os import pandas as pd -from typing import Dict, Union from importlib import metadata WS_VERSION = metadata.version('WebSearcher') class SearchEngine: """Collect Search Engine Results Pages (SERPs)""" - def __init__(self, - method: Union[str, SearchMethod] = SearchMethod.SELENIUM, - log_config: Union[dict, LogConfig] = {}, - selenium_config: Union[dict, SeleniumConfig] = {}, - requests_config: Union[dict, RequestsConfig] = {}, + def __init__(self, + method: str | SearchMethod = SearchMethod.SELENIUM, + log_config: dict | LogConfig = {}, + selenium_config: dict | SeleniumConfig = {}, + requests_config: dict | RequestsConfig = {}, crawl_id: str = '', ) -> None: """Initialize the search engine - Args: - method (Union[str, SearchMethod], optional): The method to use for searching, either 'requests' or 'selenium'. Defaults to SearchMethod.SELENIUM. - log_config (Union[dict, LogConfig], optional): Common search configuration. Defaults to None. - selenium_config (Union[dict, SeleniumConfig], optional): Selenium-specific configuration. Defaults to None. - requests_config (Union[dict, RequestsConfig], optional): Requests-specific configuration. Defaults to None. - crawl_id (str, optional): A unique identifier for the crawl. Defaults to ''. + Args: + method: The method to use for searching, either 'requests' or 'selenium'. Defaults to SearchMethod.SELENIUM. + log_config: Common search configuration. Defaults to {}. + selenium_config: Selenium-specific configuration. Defaults to {}. + requests_config: Requests-specific configuration. Defaults to {}. + crawl_id: A unique identifier for the crawl. Defaults to ''. """ # Initialize config settings, log, and session data @@ -61,23 +60,23 @@ def __init__(self, self.search_params = SearchParams.create() self.parsed = {'results': [], 'features': {}} - def search(self, - qry: str, - location: str = None, - lang: str = None, - num_results: int = None, + def search(self, + qry: str, + location: str | None = None, + lang: str | None = None, + num_results: int | None = None, ai_expand: bool = False, - headers: Dict[str, str] = {}, + headers: dict[str, str] = {}, ): """Conduct a search and save HTML Args: - qry (str): The search query - location (str, optional): A location's Canonical Name - lang (str, optional): A language code (e.g., 'en') - num_results (int, optional): The number of results to return - ai_expand: (bool, optional): Whether to use selenium to expand AI overviews - headers (Dict[str, str], optional): Custom headers to include in the request + qry: The search query + location: A location's Canonical Name + lang: A language code (e.g., 'en') + num_results: The number of results to return + ai_expand: Whether to use selenium to expand AI overviews + headers: Custom headers to include in the request """ self.log.debug('starting search config') diff --git a/WebSearcher/webutils.py b/WebSearcher/webutils.py index f4db20f..563b1c0 100644 --- a/WebSearcher/webutils.py +++ b/WebSearcher/webutils.py @@ -1,10 +1,12 @@ -""" webutils (wu): A useful collection of web utilities +"""webutils (wu): A useful collection of web utilities Note on using socks5h, hostname resolution https://stackoverflow.com/questions/12601316/how-to-make-python-requests-work-via-socks-proxy """ + from . import utils from . import logger + log = logger.Logger().start(__name__) import os @@ -15,93 +17,122 @@ import subprocess import tldextract import urllib.parse as urlparse +from collections.abc import Iterable, Mapping, Sequence +from typing import Any from bs4 import BeautifulSoup +from bs4.element import NavigableString, Tag + +SoupElement = BeautifulSoup | Tag | NavigableString -def load_html(fp, zipped=False): +def load_html(fp: str | os.PathLike[str], zipped: bool = False) -> str | bytes: """Load html file, with option for brotli decompression""" read_func = lambda i: brotli.decompress(i.read()) if zipped else i.read() - read_type = 'rb' if zipped else 'r' + read_type = "rb" if zipped else "r" with open(fp, read_type) as infile: return read_func(infile) -def load_soup(fp, zipped=False): +def load_soup(fp: str | os.PathLike[str], zipped: bool = False) -> BeautifulSoup: return make_soup(load_html(fp, zipped)) -def start_sesh(headers=None, proxy_port=None): - protocols = ['http', 'https'] +def start_sesh( + headers: Mapping[str, str] | None = None, + proxy_port: int | None = None, +) -> requests.Session: + protocols = ["http", "https"] proxy_base = "socks5://127.0.0.1:" sesh = requests.Session() - if headers: # Add headers to all requests + if headers: # Add headers to all requests sesh.headers.update(headers) - if proxy_port: # Send all requests through an ssh tunnel - proxies = {p: f'{proxy_base}{p}' for p in protocols} + if proxy_port: # Send all requests through an ssh tunnel + proxies = {p: f"{proxy_base}{proxy_port}" for p in protocols} sesh.proxies.update(proxies) - for protocol in protocols: # Auto retry if random connection error + for protocol in protocols: # Auto retry if random connection error sesh.mount(protocol, requests.adapters.HTTPAdapter(max_retries=3)) return sesh + # Misc ------------------------------------------------------------------------- -def check_dict_value(d, key, value): + +def check_dict_value(d: Mapping[str, Any], key: str, value: Any) -> bool: """Check if a key exists in a dictionary and is equal to a input value""" return (d[key] == value) if key in d else False # Parsing ---------------------------------------------------------------------- -def strip_html_tags(string): + +def strip_html_tags(string: str) -> str: """Strips HTML """ - return re.sub('<[^<]+?>', '', string) + return re.sub("<[^<]+?>", "", string) + -def make_soup(html, parser='lxml'): +def make_soup(html: str | bytes | BeautifulSoup, parser: str = "lxml") -> BeautifulSoup: """Create soup object""" if isinstance(html, BeautifulSoup): return html else: return BeautifulSoup(html, parser) -def has_captcha(soup): + +def has_captcha(soup: BeautifulSoup) -> bool: """Boolean for 'CAPTCHA' appearance in soup""" - return True if soup.find(string=re.compile('CAPTCHA')) else False + return True if soup.find(string=re.compile("CAPTCHA")) else False + -def get_html_language(soup): +def get_html_language(soup: BeautifulSoup) -> str: try: - language = soup.html.attrs['lang'] + language = soup.html.attrs["lang"] except Exception: - language = '' + language = "" return language -def parse_hashtags(text): + +def parse_hashtags(text: str) -> list[str]: """Extract unique hashtags and strip surrounding punctuation""" hashtags = set([w for w in text.split() if w.startswith("#")]) - hashtags = [re.sub(r"(\W+)$", "", h, flags = re.UNICODE) for h in hashtags] + hashtags = [re.sub(r"(\W+)$", "", h, flags=re.UNICODE) for h in hashtags] return list(set(hashtags)) -def parse_lang(soup): +def parse_lang(soup: BeautifulSoup) -> str | None: """Parse language from html tags""" try: - return soup.find('html').attrs['lang'] + return soup.find("html").attrs["lang"] except Exception as e: - log.exception('Error while parsing language') + log.exception("Error while parsing language") return None # Get divs, links, and text ---------------------------------------------------- -def get_div(soup: BeautifulSoup, name: str, attrs: dict = {}) -> BeautifulSoup: + +def get_div( + soup: Tag | None, + name: str | None, + attrs: Mapping[str, Any] | None = None, +) -> SoupElement | None: """Utility for `soup.find(name)` with null attrs handling""" + if not soup: + return None return soup.find(name, attrs) if attrs else soup.find(name) -def get_text(soup: BeautifulSoup, name: str=None, attrs: dict={}, separator:str=" ", strip=False) -> str: + +def get_text( + soup: Tag | None, + name: str | None = None, + attrs: Mapping[str, Any] | None = None, + separator: str = " ", + strip: bool = False, +) -> str | None: """Utility for `soup.find(name).text` with null name handling""" if not soup: return None @@ -111,29 +142,71 @@ def get_text(soup: BeautifulSoup, name: str=None, attrs: dict={}, separator:str= text = div.get_text(separator=separator) return text.strip() if strip else text -def get_link(soup: BeautifulSoup, attrs: dict = {}, key: str = 'href') -> str: + +def get_link( + soup: Tag | None, attrs: Mapping[str, Any] | None = None, key: str = "href" +) -> str | None: """Utility for `soup.find('a')['href']` with null key handling""" - link = get_div(soup, 'a', attrs) + link = get_div(soup, "a", attrs) return link.attrs.get(key, None) if link else None -def get_link_list(soup: BeautifulSoup, attrs: dict = {}, key: str = 'href', filter_empty: bool = True) -> list: + +def get_link_list( + soup: Tag | None, + attrs: Mapping[str, Any] | None = None, + key: str = "href", + filter_empty: bool = True, +) -> list[str] | None: """Utility for `soup.find_all('a')['href']` with null key handling""" - links = find_all_divs(soup, 'a', attrs, filter_empty) + links = find_all_divs(soup, "a", attrs, filter_empty) return [link.attrs.get(key, None) for link in links] if links else None -def find_all_divs(soup: BeautifulSoup, name: str, attrs: dict = {}, filter_empty: bool = True) -> list: + +def get_text_by_selectors( + soup: Tag | None, + selectors: Sequence[tuple[str, Mapping[str, Any]]] | None = None, + strip: bool = False, +) -> str | None: + """Get text by trying multiple selectors, return first non-null""" + if not soup or not selectors: + return None + for name, attrs in selectors: + text = get_text(soup, name, attrs, strip=strip) + if text: + return text + return None + + +def find_all_divs( + soup: Tag | None, + name: str, + attrs: Mapping[str, Any] | None = None, + filter_empty: bool = True, +) -> list[SoupElement]: if not soup: return [] divs = soup.find_all(name, attrs) if attrs else soup.find_all(name) divs = filter_empty_divs(divs) if filter_empty else divs - return divs - -def filter_empty_divs(divs): - divs = [c for c in divs if c] - divs = [c for c in divs if c.text.strip() != ''] - return divs - -def find_children(soup, name: str, attrs: dict = {}, filter_empty: bool = False): + return list(divs) + + +def filter_empty_divs(divs: Iterable[SoupElement]) -> list[SoupElement]: + filtered: list[SoupElement] = [] + for candidate in divs: + if not candidate: + continue + text_content = candidate.text if hasattr(candidate, "text") else str(candidate) + if text_content.strip() != "": + filtered.append(candidate) + return filtered + + +def find_children( + soup: BeautifulSoup | None, + name: str, + attrs: Mapping[str, Any] | None = None, + filter_empty: bool = False, +) -> Iterable[SoupElement]: """Find all children of a div with a given name and attribute""" div = get_div(soup, name, attrs) divs = div.children if div else [] @@ -143,24 +216,28 @@ def find_children(soup, name: str, attrs: dict = {}, filter_empty: bool = False) # URLs ------------------------------------------------------------------------- -def join_url_quote(quote_dict): - return '&'.join([f'{k}={v}' for k, v in quote_dict.items()]) -def encode_param_value(value): +def join_url_quote(quote_dict: Mapping[str, str]) -> str: + return "&".join([f"{k}={v}" for k, v in quote_dict.items()]) + + +def encode_param_value(value: str) -> str: return urlparse.quote_plus(value) -def url_unquote(url): + +def url_unquote(url: str) -> str: return urlparse.unquote(url) -def get_domain(url): + +def get_domain(url: str | None) -> str: """Extract a full domain from a url, drop www""" if not url: - return '' + return "" domain = tldextract.extract(url) - without_subdomain = '.'.join([domain.domain, domain.suffix]) - with_subdomain = '.'.join([domain.subdomain, domain.domain, domain.suffix]) + without_subdomain = ".".join([domain.domain, domain.suffix]) + with_subdomain = ".".join([domain.subdomain, domain.domain, domain.suffix]) if domain.subdomain: - domain_str = without_subdomain if domain.subdomain=='www' else with_subdomain + domain_str = without_subdomain if domain.subdomain == "www" else with_subdomain else: domain_str = without_subdomain return domain_str @@ -168,64 +245,94 @@ def get_domain(url): # Misc ------------------------------------------------------------------------- -def extract_html_json(data_fp, extract_to, id_col): + +def extract_html_json( + data_fp: str | os.PathLike[str], + extract_to: str | os.PathLike[str], + id_col: str, +) -> None: """Save HTML to directory for viewing""" os.makedirs(extract_to, exist_ok=True) data = utils.read_lines(data_fp) for row in data: - fp = os.path.join(extract_to, row[id_col] + '.html') - with open(fp, 'wb') as outfile: - outfile.write(row['html']) + fp = os.path.join(extract_to, row[id_col] + ".html") + with open(fp, "wb") as outfile: + outfile.write(row["html"]) + + +def split_styles(soup: BeautifulSoup) -> list[str] | None: + """Extract embedded CSS""" -def split_styles(soup): - """Extract embedded CSS """ - def split_style(style): if style.string: - return style.string.replace('}', '}\n').split('\n') + return style.string.replace("}", "}\n").split("\n") else: return None - styles = soup.find_all('style') + styles = soup.find_all("style") if styles: - return sum(list(map(split_style, styles)), []) + style_chunks = [ + chunk for chunk in map(split_style, styles) if chunk is not None + ] + return sum(style_chunks, []) else: return None # SSH ------------------------------------------------------------------------- + class SSH: - """ Create SSH cmd and tunnel objects """ - def __init__(self, user='ubuntu', port=6000, ip='', keyfile=''): + """Create SSH cmd and tunnel objects""" + + def __init__( + self, + user: str = "ubuntu", + port: int = 6000, + ip: str = "", + keyfile: str = "", + ) -> None: self.user = user self.keyfile = keyfile self.port = port self.ip = ip - self.machine = f'{self.user}@{self.ip}' - self.cmd = ['ssh', - '-i', self.keyfile, - '-ND', f'127.0.0.1:{self.port}', - '-o','StrictHostKeyChecking=no', - self.machine + self.machine = f"{self.user}@{self.ip}" + self.cmd = [ + "ssh", + "-i", + self.keyfile, + "-ND", + f"127.0.0.1:{self.port}", + "-o", + "StrictHostKeyChecking=no", + self.machine, ] - self.cmd_str = ' '.join(self.cmd) - - def open_tunnel(self): + self.cmd_str = " ".join(self.cmd) + self.tunnel: subprocess.Popen[bytes] | None = None + + def open_tunnel(self) -> None: self.tunnel = subprocess.Popen(self.cmd, shell=False) -def generate_ssh_tunnels(ips, ports, keyfile): - """ Generate SSH tunnels for each (IP, port) combination""" - def generate_ssh_tunnel(ip, port, keyfile=keyfile): +def generate_ssh_tunnels( + ips: Sequence[str], + ports: Sequence[int], + keyfile: str, +) -> list[SSH]: + """Generate SSH tunnels for each (IP, port) combination""" + + def generate_ssh_tunnel(ip: str, port: int, keyfile: str = keyfile) -> SSH: ssh_tunnel = SSH(ip=ip, port=port, keyfile=keyfile) - subprocess.call(['chmod', '600', keyfile]) - log.info(f'{ssh_tunnel.cmd_str}') + subprocess.call(["chmod", "600", keyfile]) + log.info(f"{ssh_tunnel.cmd_str}") ssh_tunnel.open_tunnel() - atexit.register(exit_handler, ssh_tunnel) # Always kill tunnels on exit + atexit.register(exit_handler, ssh_tunnel) # Always kill tunnels on exit + return ssh_tunnel return [generate_ssh_tunnel(ip, port) for ip, port in zip(ips, ports)] -def exit_handler(ssh): - log.info(f'Killing: {ssh.machine} on port: {ssh.port}') - ssh.tunnel.kill() \ No newline at end of file + +def exit_handler(ssh: SSH) -> None: + log.info(f"Killing: {ssh.machine} on port: {ssh.port}") + if ssh.tunnel: + ssh.tunnel.kill() diff --git a/poetry.lock b/poetry.lock index af6b4a9..e57dcd9 100644 --- a/poetry.lock +++ b/poetry.lock @@ -41,6 +41,18 @@ files = [ astroid = ["astroid (>=2,<5)"] test = ["astroid (>=2,<5)", "pytest (<9.0)", "pytest-cov", "pytest-xdist"] +[[package]] +name = "async-generator" +version = "1.10" +description = "Async generators and context managers for Python 3.5+" +optional = false +python-versions = ">=3.5" +groups = ["main"] +files = [ + {file = "async_generator-1.10-py3-none-any.whl", hash = "sha256:01c7bf666359b4967d2cda0000cc2e4af16a0ae098cbffcb8472fb9e8ad6585b"}, + {file = "async_generator-1.10.tar.gz", hash = "sha256:6ebb3d106c12920aaae42ccb6f787ef5eefdcdd166ea3d628fa8476abe712144"}, +] + [[package]] name = "attrs" version = "25.4.0" @@ -188,14 +200,14 @@ files = [ [[package]] name = "certifi" -version = "2025.11.12" +version = "2026.1.4" description = "Python package for providing Mozilla's CA Bundle." optional = false python-versions = ">=3.7" groups = ["main"] files = [ - {file = "certifi-2025.11.12-py3-none-any.whl", hash = "sha256:97de8790030bbd5c2d96b7ec782fc2f7820ef8dba6db909ccf95449f2d062d4b"}, - {file = "certifi-2025.11.12.tar.gz", hash = "sha256:d8ab5478f2ecd78af242878415affce761ca6bc54a22a27e026d7c25357c3316"}, + {file = "certifi-2026.1.4-py3-none-any.whl", hash = "sha256:9943707519e4add1115f44c2bc244f782c0249876bf51b6599fee1ffbedd685c"}, + {file = "certifi-2026.1.4.tar.gz", hash = "sha256:ac726dd470482006e014ad384921ed6438c457018f4b3d204aea4281258b2120"}, ] [[package]] @@ -291,7 +303,7 @@ files = [ {file = "cffi-2.0.0-cp39-cp39-win_amd64.whl", hash = "sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9"}, {file = "cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529"}, ] -markers = {main = "os_name == \"nt\" and implementation_name != \"pypy\"", dev = "implementation_name == \"pypy\""} +markers = {main = "implementation_name != \"pypy\" and os_name == \"nt\"", dev = "implementation_name == \"pypy\""} [package.dependencies] pycparser = {version = "*", markers = "implementation_name != \"PyPy\""} @@ -441,7 +453,7 @@ description = "Cross-platform colored terminal text." optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" groups = ["dev"] -markers = "sys_platform == \"win32\" or platform_system == \"Windows\"" +markers = "platform_system == \"Windows\" or sys_platform == \"win32\"" files = [ {file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"}, {file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"}, @@ -464,42 +476,42 @@ test = ["pytest"] [[package]] name = "debugpy" -version = "1.8.17" +version = "1.8.20" description = "An implementation of the Debug Adapter Protocol for Python" optional = false python-versions = ">=3.8" groups = ["dev"] files = [ - {file = "debugpy-1.8.17-cp310-cp310-macosx_15_0_x86_64.whl", hash = "sha256:c41d2ce8bbaddcc0009cc73f65318eedfa3dbc88a8298081deb05389f1ab5542"}, - {file = "debugpy-1.8.17-cp310-cp310-manylinux_2_34_x86_64.whl", hash = "sha256:1440fd514e1b815edd5861ca394786f90eb24960eb26d6f7200994333b1d79e3"}, - {file = "debugpy-1.8.17-cp310-cp310-win32.whl", hash = "sha256:3a32c0af575749083d7492dc79f6ab69f21b2d2ad4cd977a958a07d5865316e4"}, - {file = "debugpy-1.8.17-cp310-cp310-win_amd64.whl", hash = "sha256:a3aad0537cf4d9c1996434be68c6c9a6d233ac6f76c2a482c7803295b4e4f99a"}, - {file = "debugpy-1.8.17-cp311-cp311-macosx_15_0_universal2.whl", hash = "sha256:d3fce3f0e3de262a3b67e69916d001f3e767661c6e1ee42553009d445d1cd840"}, - {file = "debugpy-1.8.17-cp311-cp311-manylinux_2_34_x86_64.whl", hash = "sha256:c6bdf134457ae0cac6fb68205776be635d31174eeac9541e1d0c062165c6461f"}, - {file = "debugpy-1.8.17-cp311-cp311-win32.whl", hash = "sha256:e79a195f9e059edfe5d8bf6f3749b2599452d3e9380484cd261f6b7cd2c7c4da"}, - {file = "debugpy-1.8.17-cp311-cp311-win_amd64.whl", hash = "sha256:b532282ad4eca958b1b2d7dbcb2b7218e02cb934165859b918e3b6ba7772d3f4"}, - {file = "debugpy-1.8.17-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:f14467edef672195c6f6b8e27ce5005313cb5d03c9239059bc7182b60c176e2d"}, - {file = "debugpy-1.8.17-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:24693179ef9dfa20dca8605905a42b392be56d410c333af82f1c5dff807a64cc"}, - {file = "debugpy-1.8.17-cp312-cp312-win32.whl", hash = "sha256:6a4e9dacf2cbb60d2514ff7b04b4534b0139facbf2abdffe0639ddb6088e59cf"}, - {file = "debugpy-1.8.17-cp312-cp312-win_amd64.whl", hash = "sha256:e8f8f61c518952fb15f74a302e068b48d9c4691768ade433e4adeea961993464"}, - {file = "debugpy-1.8.17-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:857c1dd5d70042502aef1c6d1c2801211f3ea7e56f75e9c335f434afb403e464"}, - {file = "debugpy-1.8.17-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:3bea3b0b12f3946e098cce9b43c3c46e317b567f79570c3f43f0b96d00788088"}, - {file = "debugpy-1.8.17-cp313-cp313-win32.whl", hash = "sha256:e34ee844c2f17b18556b5bbe59e1e2ff4e86a00282d2a46edab73fd7f18f4a83"}, - {file = "debugpy-1.8.17-cp313-cp313-win_amd64.whl", hash = "sha256:6c5cd6f009ad4fca8e33e5238210dc1e5f42db07d4b6ab21ac7ffa904a196420"}, - {file = "debugpy-1.8.17-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:045290c010bcd2d82bc97aa2daf6837443cd52f6328592698809b4549babcee1"}, - {file = "debugpy-1.8.17-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:b69b6bd9dba6a03632534cdf67c760625760a215ae289f7489a452af1031fe1f"}, - {file = "debugpy-1.8.17-cp314-cp314-win32.whl", hash = "sha256:5c59b74aa5630f3a5194467100c3b3d1c77898f9ab27e3f7dc5d40fc2f122670"}, - {file = "debugpy-1.8.17-cp314-cp314-win_amd64.whl", hash = "sha256:893cba7bb0f55161de4365584b025f7064e1f88913551bcd23be3260b231429c"}, - {file = "debugpy-1.8.17-cp38-cp38-macosx_15_0_x86_64.whl", hash = "sha256:8deb4e31cd575c9f9370042876e078ca118117c1b5e1f22c32befcfbb6955f0c"}, - {file = "debugpy-1.8.17-cp38-cp38-manylinux_2_34_x86_64.whl", hash = "sha256:b75868b675949a96ab51abc114c7163f40ff0d8f7d6d5fd63f8932fd38e9c6d7"}, - {file = "debugpy-1.8.17-cp38-cp38-win32.whl", hash = "sha256:17e456da14848d618662354e1dccfd5e5fb75deec3d1d48dc0aa0baacda55860"}, - {file = "debugpy-1.8.17-cp38-cp38-win_amd64.whl", hash = "sha256:e851beb536a427b5df8aa7d0c7835b29a13812f41e46292ff80b2ef77327355a"}, - {file = "debugpy-1.8.17-cp39-cp39-macosx_15_0_x86_64.whl", hash = "sha256:f2ac8055a0c4a09b30b931100996ba49ef334c6947e7ae365cdd870416d7513e"}, - {file = "debugpy-1.8.17-cp39-cp39-manylinux_2_34_x86_64.whl", hash = "sha256:eaa85bce251feca8e4c87ce3b954aba84b8c645b90f0e6a515c00394a9f5c0e7"}, - {file = "debugpy-1.8.17-cp39-cp39-win32.whl", hash = "sha256:b13eea5587e44f27f6c48588b5ad56dcb74a4f3a5f89250443c94587f3eb2ea1"}, - {file = "debugpy-1.8.17-cp39-cp39-win_amd64.whl", hash = "sha256:bb1bbf92317e1f35afcf3ef0450219efb3afe00be79d8664b250ac0933b9015f"}, - {file = "debugpy-1.8.17-py2.py3-none-any.whl", hash = "sha256:60c7dca6571efe660ccb7a9508d73ca14b8796c4ed484c2002abba714226cfef"}, - {file = "debugpy-1.8.17.tar.gz", hash = "sha256:fd723b47a8c08892b1a16b2c6239a8b96637c62a59b94bb5dab4bac592a58a8e"}, + {file = "debugpy-1.8.20-cp310-cp310-macosx_15_0_x86_64.whl", hash = "sha256:157e96ffb7f80b3ad36d808646198c90acb46fdcfd8bb1999838f0b6f2b59c64"}, + {file = "debugpy-1.8.20-cp310-cp310-manylinux_2_34_x86_64.whl", hash = "sha256:c1178ae571aff42e61801a38b007af504ec8e05fde1c5c12e5a7efef21009642"}, + {file = "debugpy-1.8.20-cp310-cp310-win32.whl", hash = "sha256:c29dd9d656c0fbd77906a6e6a82ae4881514aa3294b94c903ff99303e789b4a2"}, + {file = "debugpy-1.8.20-cp310-cp310-win_amd64.whl", hash = "sha256:3ca85463f63b5dd0aa7aaa933d97cbc47c174896dcae8431695872969f981893"}, + {file = "debugpy-1.8.20-cp311-cp311-macosx_15_0_universal2.whl", hash = "sha256:eada6042ad88fa1571b74bd5402ee8b86eded7a8f7b827849761700aff171f1b"}, + {file = "debugpy-1.8.20-cp311-cp311-manylinux_2_34_x86_64.whl", hash = "sha256:7de0b7dfeedc504421032afba845ae2a7bcc32ddfb07dae2c3ca5442f821c344"}, + {file = "debugpy-1.8.20-cp311-cp311-win32.whl", hash = "sha256:773e839380cf459caf73cc533ea45ec2737a5cc184cf1b3b796cd4fd98504fec"}, + {file = "debugpy-1.8.20-cp311-cp311-win_amd64.whl", hash = "sha256:1f7650546e0eded1902d0f6af28f787fa1f1dbdbc97ddabaf1cd963a405930cb"}, + {file = "debugpy-1.8.20-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:4ae3135e2089905a916909ef31922b2d733d756f66d87345b3e5e52b7a55f13d"}, + {file = "debugpy-1.8.20-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:88f47850a4284b88bd2bfee1f26132147d5d504e4e86c22485dfa44b97e19b4b"}, + {file = "debugpy-1.8.20-cp312-cp312-win32.whl", hash = "sha256:4057ac68f892064e5f98209ab582abfee3b543fb55d2e87610ddc133a954d390"}, + {file = "debugpy-1.8.20-cp312-cp312-win_amd64.whl", hash = "sha256:a1a8f851e7cf171330679ef6997e9c579ef6dd33c9098458bd9986a0f4ca52e3"}, + {file = "debugpy-1.8.20-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:5dff4bb27027821fdfcc9e8f87309a28988231165147c31730128b1c983e282a"}, + {file = "debugpy-1.8.20-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:84562982dd7cf5ebebfdea667ca20a064e096099997b175fe204e86817f64eaf"}, + {file = "debugpy-1.8.20-cp313-cp313-win32.whl", hash = "sha256:da11dea6447b2cadbf8ce2bec59ecea87cc18d2c574980f643f2d2dfe4862393"}, + {file = "debugpy-1.8.20-cp313-cp313-win_amd64.whl", hash = "sha256:eb506e45943cab2efb7c6eafdd65b842f3ae779f020c82221f55aca9de135ed7"}, + {file = "debugpy-1.8.20-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:9c74df62fc064cd5e5eaca1353a3ef5a5d50da5eb8058fcef63106f7bebe6173"}, + {file = "debugpy-1.8.20-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:077a7447589ee9bc1ff0cdf443566d0ecf540ac8aa7333b775ebcb8ce9f4ecad"}, + {file = "debugpy-1.8.20-cp314-cp314-win32.whl", hash = "sha256:352036a99dd35053b37b7803f748efc456076f929c6a895556932eaf2d23b07f"}, + {file = "debugpy-1.8.20-cp314-cp314-win_amd64.whl", hash = "sha256:a98eec61135465b062846112e5ecf2eebb855305acc1dfbae43b72903b8ab5be"}, + {file = "debugpy-1.8.20-cp38-cp38-macosx_15_0_x86_64.whl", hash = "sha256:b773eb026a043e4d9c76265742bc846f2f347da7e27edf7fe97716ea19d6bfc5"}, + {file = "debugpy-1.8.20-cp38-cp38-manylinux_2_34_x86_64.whl", hash = "sha256:20d6e64ea177ab6732bffd3ce8fc6fb8879c60484ce14c3b3fe183b1761459ca"}, + {file = "debugpy-1.8.20-cp38-cp38-win32.whl", hash = "sha256:0dfd9adb4b3c7005e9c33df430bcdd4e4ebba70be533e0066e3a34d210041b66"}, + {file = "debugpy-1.8.20-cp38-cp38-win_amd64.whl", hash = "sha256:60f89411a6c6afb89f18e72e9091c3dfbcfe3edc1066b2043a1f80a3bbb3e11f"}, + {file = "debugpy-1.8.20-cp39-cp39-macosx_15_0_x86_64.whl", hash = "sha256:bff8990f040dacb4c314864da95f7168c5a58a30a66e0eea0fb85e2586a92cd6"}, + {file = "debugpy-1.8.20-cp39-cp39-manylinux_2_34_x86_64.whl", hash = "sha256:70ad9ae09b98ac307b82c16c151d27ee9d68ae007a2e7843ba621b5ce65333b5"}, + {file = "debugpy-1.8.20-cp39-cp39-win32.whl", hash = "sha256:9eeed9f953f9a23850c85d440bf51e3c56ed5d25f8560eeb29add815bd32f7ee"}, + {file = "debugpy-1.8.20-cp39-cp39-win_amd64.whl", hash = "sha256:760813b4fff517c75bfe7923033c107104e76acfef7bda011ffea8736e9a66f8"}, + {file = "debugpy-1.8.20-py2.py3-none-any.whl", hash = "sha256:5be9bed9ae3be00665a06acaa48f8329d2b9632f15fd09f6a9a8c8d9907e54d7"}, + {file = "debugpy-1.8.20.tar.gz", hash = "sha256:55bc8701714969f1ab89a6d5f2f3d40c36f91b2cbe2f65d98bf8196f6a6a2c33"}, ] [[package]] @@ -587,6 +599,30 @@ files = [ [package.extras] all = ["flake8 (>=7.1.1)", "mypy (>=1.11.2)", "pytest (>=8.3.2)", "ruff (>=0.6.2)"] +[[package]] +name = "importlib-metadata" +version = "8.7.1" +description = "Read metadata from Python packages" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151"}, + {file = "importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb"}, +] + +[package.dependencies] +zipp = ">=3.20" + +[package.extras] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\""] +cover = ["pytest-cov"] +doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-lint"] +enabler = ["pytest-enabler (>=3.4)"] +perf = ["ipython"] +test = ["flufl.flake8", "jaraco.test (>=5.4)", "packaging", "pyfakefs", "pytest (>=6,!=8.1.*)", "pytest-perf (>=0.9.2)"] +type = ["mypy (<1.19) ; platform_python_implementation == \"PyPy\"", "pytest-mypy (>=1.0.1)"] + [[package]] name = "iniconfig" version = "2.3.0" @@ -635,15 +671,15 @@ test = ["flaky", "ipyparallel", "pre-commit", "pytest (>=7.0,<9)", "pytest-async [[package]] name = "ipython" -version = "8.37.0" +version = "8.38.0" description = "IPython: Productive Interactive Computing" optional = false python-versions = ">=3.10" groups = ["dev"] markers = "python_version == \"3.10\"" files = [ - {file = "ipython-8.37.0-py3-none-any.whl", hash = "sha256:ed87326596b878932dbcb171e3e698845434d8c61b8d8cd474bf663041a9dcf2"}, - {file = "ipython-8.37.0.tar.gz", hash = "sha256:ca815841e1a41a1e6b73a0b08f3038af9b2252564d01fc405356d34033012216"}, + {file = "ipython-8.38.0-py3-none-any.whl", hash = "sha256:750162629d800ac65bb3b543a14e7a74b0e88063eac9b92124d4b2aa3f6d8e86"}, + {file = "ipython-8.38.0.tar.gz", hash = "sha256:9cfea8c903ce0867cc2f23199ed8545eb741f3a69420bfcf3743ad1cec856d39"}, ] [package.dependencies] @@ -675,15 +711,15 @@ test-extra = ["curio", "ipython[test]", "jupyter_ai", "matplotlib (!=3.2.0)", "n [[package]] name = "ipython" -version = "9.8.0" +version = "9.10.0" description = "IPython: Productive Interactive Computing" optional = false python-versions = ">=3.11" groups = ["dev"] markers = "python_version >= \"3.11\"" files = [ - {file = "ipython-9.8.0-py3-none-any.whl", hash = "sha256:ebe6d1d58d7d988fbf23ff8ff6d8e1622cfdb194daf4b7b73b792c4ec3b85385"}, - {file = "ipython-9.8.0.tar.gz", hash = "sha256:8e4ce129a627eb9dd221c41b1d2cdaed4ef7c9da8c17c63f6f578fe231141f83"}, + {file = "ipython-9.10.0-py3-none-any.whl", hash = "sha256:c6ab68cc23bba8c7e18e9b932797014cc61ea7fd6f19de180ab9ba73e65ee58d"}, + {file = "ipython-9.10.0.tar.gz", hash = "sha256:cd9e656be97618a0676d058134cd44e6dc7012c0e5cb36a9ce96a8c904adaf77"}, ] [package.dependencies] @@ -700,7 +736,7 @@ traitlets = ">=5.13.0" typing_extensions = {version = ">=4.6", markers = "python_version < \"3.12\""} [package.extras] -all = ["ipython[doc,matplotlib,test,test-extra]"] +all = ["argcomplete (>=3.0)", "ipython[doc,matplotlib,terminal,test,test-extra]"] black = ["black"] doc = ["docrepr", "exceptiongroup", "intersphinx_registry", "ipykernel", "ipython[matplotlib,test]", "setuptools (>=70.0)", "sphinx (>=8.0)", "sphinx-rtd-theme (>=0.1.8)", "sphinx_toml (==0.0.4)", "typing_extensions"] matplotlib = ["matplotlib (>3.9)"] @@ -745,26 +781,27 @@ testing = ["Django", "attrs", "colorama", "docopt", "pytest (<9.0.0)"] [[package]] name = "jupyter-client" -version = "8.6.3" +version = "8.8.0" description = "Jupyter protocol implementation and client libraries" optional = false -python-versions = ">=3.8" +python-versions = ">=3.10" groups = ["dev"] files = [ - {file = "jupyter_client-8.6.3-py3-none-any.whl", hash = "sha256:e8a19cc986cc45905ac3362915f410f3af85424b4c0905e94fa5f2cb08e8f23f"}, - {file = "jupyter_client-8.6.3.tar.gz", hash = "sha256:35b3a0947c4a6e9d589eb97d7d4cd5e90f910ee73101611f01283732bd6d9419"}, + {file = "jupyter_client-8.8.0-py3-none-any.whl", hash = "sha256:f93a5b99c5e23a507b773d3a1136bd6e16c67883ccdbd9a829b0bbdb98cd7d7a"}, + {file = "jupyter_client-8.8.0.tar.gz", hash = "sha256:d556811419a4f2d96c869af34e854e3f059b7cc2d6d01a9cd9c85c267691be3e"}, ] [package.dependencies] -jupyter-core = ">=4.12,<5.0.dev0 || >=5.1.dev0" +jupyter-core = ">=5.1" python-dateutil = ">=2.8.2" -pyzmq = ">=23.0" -tornado = ">=6.2" +pyzmq = ">=25.0" +tornado = ">=6.4.1" traitlets = ">=5.3" [package.extras] docs = ["ipykernel", "myst-parser", "pydata-sphinx-theme", "sphinx (>=4)", "sphinx-autodoc-typehints", "sphinxcontrib-github-alt", "sphinxcontrib-spelling"] -test = ["coverage", "ipykernel (>=6.14)", "mypy", "paramiko ; sys_platform == \"win32\"", "pre-commit", "pytest (<8.2.0)", "pytest-cov", "pytest-jupyter[client] (>=0.4.1)", "pytest-timeout"] +orjson = ["orjson"] +test = ["anyio", "coverage", "ipykernel (>=6.14)", "msgpack", "mypy ; platform_python_implementation != \"PyPy\"", "paramiko ; sys_platform == \"win32\"", "pre-commit", "pytest", "pytest-cov", "pytest-jupyter[client] (>=0.6.2)", "pytest-timeout"] [[package]] name = "jupyter-core" @@ -996,6 +1033,18 @@ files = [ {file = "mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba"}, ] +[[package]] +name = "mypy-extensions" +version = "1.1.0" +description = "Type system extensions for programs checked with the mypy type checker." +optional = false +python-versions = ">=3.8" +groups = ["main"] +files = [ + {file = "mypy_extensions-1.1.0-py3-none-any.whl", hash = "sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505"}, + {file = "mypy_extensions-1.1.0.tar.gz", hash = "sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558"}, +] + [[package]] name = "nest-asyncio" version = "1.6.0" @@ -1076,184 +1125,169 @@ files = [ [[package]] name = "numpy" -version = "2.3.5" +version = "2.4.2" description = "Fundamental package for array computing in Python" optional = false python-versions = ">=3.11" groups = ["main"] markers = "python_version >= \"3.11\"" files = [ - {file = "numpy-2.3.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:de5672f4a7b200c15a4127042170a694d4df43c992948f5e1af57f0174beed10"}, - {file = "numpy-2.3.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:acfd89508504a19ed06ef963ad544ec6664518c863436306153e13e94605c218"}, - {file = "numpy-2.3.5-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:ffe22d2b05504f786c867c8395de703937f934272eb67586817b46188b4ded6d"}, - {file = "numpy-2.3.5-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:872a5cf366aec6bb1147336480fef14c9164b154aeb6542327de4970282cd2f5"}, - {file = "numpy-2.3.5-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3095bdb8dd297e5920b010e96134ed91d852d81d490e787beca7e35ae1d89cf7"}, - {file = "numpy-2.3.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8cba086a43d54ca804ce711b2a940b16e452807acebe7852ff327f1ecd49b0d4"}, - {file = "numpy-2.3.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:6cf9b429b21df6b99f4dee7a1218b8b7ffbbe7df8764dc0bd60ce8a0708fed1e"}, - {file = "numpy-2.3.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:396084a36abdb603546b119d96528c2f6263921c50df3c8fd7cb28873a237748"}, - {file = "numpy-2.3.5-cp311-cp311-win32.whl", hash = "sha256:b0c7088a73aef3d687c4deef8452a3ac7c1be4e29ed8bf3b366c8111128ac60c"}, - {file = "numpy-2.3.5-cp311-cp311-win_amd64.whl", hash = "sha256:a414504bef8945eae5f2d7cb7be2d4af77c5d1cb5e20b296c2c25b61dff2900c"}, - {file = "numpy-2.3.5-cp311-cp311-win_arm64.whl", hash = "sha256:0cd00b7b36e35398fa2d16af7b907b65304ef8bb4817a550e06e5012929830fa"}, - {file = "numpy-2.3.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:74ae7b798248fe62021dbf3c914245ad45d1a6b0cb4a29ecb4b31d0bfbc4cc3e"}, - {file = "numpy-2.3.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ee3888d9ff7c14604052b2ca5535a30216aa0a58e948cdd3eeb8d3415f638769"}, - {file = "numpy-2.3.5-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:612a95a17655e213502f60cfb9bf9408efdc9eb1d5f50535cc6eb365d11b42b5"}, - {file = "numpy-2.3.5-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:3101e5177d114a593d79dd79658650fe28b5a0d8abeb8ce6f437c0e6df5be1a4"}, - {file = "numpy-2.3.5-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8b973c57ff8e184109db042c842423ff4f60446239bd585a5131cc47f06f789d"}, - {file = "numpy-2.3.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0d8163f43acde9a73c2a33605353a4f1bc4798745a8b1d73183b28e5b435ae28"}, - {file = "numpy-2.3.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:51c1e14eb1e154ebd80e860722f9e6ed6ec89714ad2db2d3aa33c31d7c12179b"}, - {file = "numpy-2.3.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:b46b4ec24f7293f23adcd2d146960559aaf8020213de8ad1909dba6c013bf89c"}, - {file = "numpy-2.3.5-cp312-cp312-win32.whl", hash = "sha256:3997b5b3c9a771e157f9aae01dd579ee35ad7109be18db0e85dbdbe1de06e952"}, - {file = "numpy-2.3.5-cp312-cp312-win_amd64.whl", hash = "sha256:86945f2ee6d10cdfd67bcb4069c1662dd711f7e2a4343db5cecec06b87cf31aa"}, - {file = "numpy-2.3.5-cp312-cp312-win_arm64.whl", hash = "sha256:f28620fe26bee16243be2b7b874da327312240a7cdc38b769a697578d2100013"}, - {file = "numpy-2.3.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:d0f23b44f57077c1ede8c5f26b30f706498b4862d3ff0a7298b8411dd2f043ff"}, - {file = "numpy-2.3.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:aa5bc7c5d59d831d9773d1170acac7893ce3a5e130540605770ade83280e7188"}, - {file = "numpy-2.3.5-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:ccc933afd4d20aad3c00bcef049cb40049f7f196e0397f1109dba6fed63267b0"}, - {file = "numpy-2.3.5-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:afaffc4393205524af9dfa400fa250143a6c3bc646c08c9f5e25a9f4b4d6a903"}, - {file = "numpy-2.3.5-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c75442b2209b8470d6d5d8b1c25714270686f14c749028d2199c54e29f20b4d"}, - {file = "numpy-2.3.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:11e06aa0af8c0f05104d56450d6093ee639e15f24ecf62d417329d06e522e017"}, - {file = "numpy-2.3.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ed89927b86296067b4f81f108a2271d8926467a8868e554eaf370fc27fa3ccaf"}, - {file = "numpy-2.3.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51c55fe3451421f3a6ef9a9c1439e82101c57a2c9eab9feb196a62b1a10b58ce"}, - {file = "numpy-2.3.5-cp313-cp313-win32.whl", hash = "sha256:1978155dd49972084bd6ef388d66ab70f0c323ddee6f693d539376498720fb7e"}, - {file = "numpy-2.3.5-cp313-cp313-win_amd64.whl", hash = "sha256:00dc4e846108a382c5869e77c6ed514394bdeb3403461d25a829711041217d5b"}, - {file = "numpy-2.3.5-cp313-cp313-win_arm64.whl", hash = "sha256:0472f11f6ec23a74a906a00b48a4dcf3849209696dff7c189714511268d103ae"}, - {file = "numpy-2.3.5-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:414802f3b97f3c1eef41e530aaba3b3c1620649871d8cb38c6eaff034c2e16bd"}, - {file = "numpy-2.3.5-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5ee6609ac3604fa7780e30a03e5e241a7956f8e2fcfe547d51e3afa5247ac47f"}, - {file = "numpy-2.3.5-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:86d835afea1eaa143012a2d7a3f45a3adce2d7adc8b4961f0b362214d800846a"}, - {file = "numpy-2.3.5-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:30bc11310e8153ca664b14c5f1b73e94bd0503681fcf136a163de856f3a50139"}, - {file = "numpy-2.3.5-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1062fde1dcf469571705945b0f221b73928f34a20c904ffb45db101907c3454e"}, - {file = "numpy-2.3.5-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ce581db493ea1a96c0556360ede6607496e8bf9b3a8efa66e06477267bc831e9"}, - {file = "numpy-2.3.5-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:cc8920d2ec5fa99875b670bb86ddeb21e295cb07aa331810d9e486e0b969d946"}, - {file = "numpy-2.3.5-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:9ee2197ef8c4f0dfe405d835f3b6a14f5fee7782b5de51ba06fb65fc9b36e9f1"}, - {file = "numpy-2.3.5-cp313-cp313t-win32.whl", hash = "sha256:70b37199913c1bd300ff6e2693316c6f869c7ee16378faf10e4f5e3275b299c3"}, - {file = "numpy-2.3.5-cp313-cp313t-win_amd64.whl", hash = "sha256:b501b5fa195cc9e24fe102f21ec0a44dffc231d2af79950b451e0d99cea02234"}, - {file = "numpy-2.3.5-cp313-cp313t-win_arm64.whl", hash = "sha256:a80afd79f45f3c4a7d341f13acbe058d1ca8ac017c165d3fa0d3de6bc1a079d7"}, - {file = "numpy-2.3.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:bf06bc2af43fa8d32d30fae16ad965663e966b1a3202ed407b84c989c3221e82"}, - {file = "numpy-2.3.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:052e8c42e0c49d2575621c158934920524f6c5da05a1d3b9bab5d8e259e045f0"}, - {file = "numpy-2.3.5-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:1ed1ec893cff7040a02c8aa1c8611b94d395590d553f6b53629a4461dc7f7b63"}, - {file = "numpy-2.3.5-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:2dcd0808a421a482a080f89859a18beb0b3d1e905b81e617a188bd80422d62e9"}, - {file = "numpy-2.3.5-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:727fd05b57df37dc0bcf1a27767a3d9a78cbbc92822445f32cc3436ba797337b"}, - {file = "numpy-2.3.5-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fffe29a1ef00883599d1dc2c51aa2e5d80afe49523c261a74933df395c15c520"}, - {file = "numpy-2.3.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8f7f0e05112916223d3f438f293abf0727e1181b5983f413dfa2fefc4098245c"}, - {file = "numpy-2.3.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2e2eb32ddb9ccb817d620ac1d8dae7c3f641c1e5f55f531a33e8ab97960a75b8"}, - {file = "numpy-2.3.5-cp314-cp314-win32.whl", hash = "sha256:66f85ce62c70b843bab1fb14a05d5737741e74e28c7b8b5a064de10142fad248"}, - {file = "numpy-2.3.5-cp314-cp314-win_amd64.whl", hash = "sha256:e6a0bc88393d65807d751a614207b7129a310ca4fe76a74e5c7da5fa5671417e"}, - {file = "numpy-2.3.5-cp314-cp314-win_arm64.whl", hash = "sha256:aeffcab3d4b43712bb7a60b65f6044d444e75e563ff6180af8f98dd4b905dfd2"}, - {file = "numpy-2.3.5-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:17531366a2e3a9e30762c000f2c43a9aaa05728712e25c11ce1dbe700c53ad41"}, - {file = "numpy-2.3.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d21644de1b609825ede2f48be98dfde4656aefc713654eeee280e37cadc4e0ad"}, - {file = "numpy-2.3.5-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:c804e3a5aba5460c73955c955bdbd5c08c354954e9270a2c1565f62e866bdc39"}, - {file = "numpy-2.3.5-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:cc0a57f895b96ec78969c34f682c602bf8da1a0270b09bc65673df2e7638ec20"}, - {file = "numpy-2.3.5-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:900218e456384ea676e24ea6a0417f030a3b07306d29d7ad843957b40a9d8d52"}, - {file = "numpy-2.3.5-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:09a1bea522b25109bf8e6f3027bd810f7c1085c64a0c7ce050c1676ad0ba010b"}, - {file = "numpy-2.3.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:04822c00b5fd0323c8166d66c701dc31b7fbd252c100acd708c48f763968d6a3"}, - {file = "numpy-2.3.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:d6889ec4ec662a1a37eb4b4fb26b6100841804dac55bd9df579e326cdc146227"}, - {file = "numpy-2.3.5-cp314-cp314t-win32.whl", hash = "sha256:93eebbcf1aafdf7e2ddd44c2923e2672e1010bddc014138b229e49725b4d6be5"}, - {file = "numpy-2.3.5-cp314-cp314t-win_amd64.whl", hash = "sha256:c8a9958e88b65c3b27e22ca2a076311636850b612d6bbfb76e8d156aacde2aaf"}, - {file = "numpy-2.3.5-cp314-cp314t-win_arm64.whl", hash = "sha256:6203fdf9f3dc5bdaed7319ad8698e685c7a3be10819f41d32a0723e611733b42"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:f0963b55cdd70fad460fa4c1341f12f976bb26cb66021a5580329bd498988310"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:f4255143f5160d0de972d28c8f9665d882b5f61309d8362fdd3e103cf7bf010c"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:a4b9159734b326535f4dd01d947f919c6eefd2d9827466a696c44ced82dfbc18"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:2feae0d2c91d46e59fcd62784a3a83b3fb677fead592ce51b5a6fbb4f95965ff"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ffac52f28a7849ad7576293c0cb7b9f08304e8f7d738a8cb8a90ec4c55a998eb"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:63c0e9e7eea69588479ebf4a8a270d5ac22763cc5854e9a7eae952a3908103f7"}, - {file = "numpy-2.3.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:f16417ec91f12f814b10bafe79ef77e70113a2f5f7018640e7425ff979253425"}, - {file = "numpy-2.3.5.tar.gz", hash = "sha256:784db1dcdab56bf0517743e746dfb0f885fc68d948aba86eeec2cba234bdf1c0"}, + {file = "numpy-2.4.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:e7e88598032542bd49af7c4747541422884219056c268823ef6e5e89851c8825"}, + {file = "numpy-2.4.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7edc794af8b36ca37ef5fcb5e0d128c7e0595c7b96a2318d1badb6fcd8ee86b1"}, + {file = "numpy-2.4.2-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:6e9f61981ace1360e42737e2bae58b27bf28a1b27e781721047d84bd754d32e7"}, + {file = "numpy-2.4.2-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:cb7bbb88aa74908950d979eeaa24dbdf1a865e3c7e45ff0121d8f70387b55f73"}, + {file = "numpy-2.4.2-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4f069069931240b3fc703f1e23df63443dbd6390614c8c44a87d96cd0ec81eb1"}, + {file = "numpy-2.4.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c02ef4401a506fb60b411467ad501e1429a3487abca4664871d9ae0b46c8ba32"}, + {file = "numpy-2.4.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2653de5c24910e49c2b106499803124dde62a5a1fe0eedeaecf4309a5f639390"}, + {file = "numpy-2.4.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1ae241bbfc6ae276f94a170b14785e561cb5e7f626b6688cf076af4110887413"}, + {file = "numpy-2.4.2-cp311-cp311-win32.whl", hash = "sha256:df1b10187212b198dd45fa943d8985a3c8cf854aed4923796e0e019e113a1bda"}, + {file = "numpy-2.4.2-cp311-cp311-win_amd64.whl", hash = "sha256:b9c618d56a29c9cb1c4da979e9899be7578d2e0b3c24d52079c166324c9e8695"}, + {file = "numpy-2.4.2-cp311-cp311-win_arm64.whl", hash = "sha256:47c5a6ed21d9452b10227e5e8a0e1c22979811cad7dcc19d8e3e2fb8fa03f1a3"}, + {file = "numpy-2.4.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:21982668592194c609de53ba4933a7471880ccbaadcc52352694a59ecc860b3a"}, + {file = "numpy-2.4.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40397bda92382fcec844066efb11f13e1c9a3e2a8e8f318fb72ed8b6db9f60f1"}, + {file = "numpy-2.4.2-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:b3a24467af63c67829bfaa61eecf18d5432d4f11992688537be59ecd6ad32f5e"}, + {file = "numpy-2.4.2-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:805cc8de9fd6e7a22da5aed858e0ab16be5a4db6c873dde1d7451c541553aa27"}, + {file = "numpy-2.4.2-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6d82351358ffbcdcd7b686b90742a9b86632d6c1c051016484fa0b326a0a1548"}, + {file = "numpy-2.4.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9e35d3e0144137d9fdae62912e869136164534d64a169f86438bc9561b6ad49f"}, + {file = "numpy-2.4.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:adb6ed2ad29b9e15321d167d152ee909ec73395901b70936f029c3bc6d7f4460"}, + {file = "numpy-2.4.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:8906e71fd8afcb76580404e2a950caef2685df3d2a57fe82a86ac8d33cc007ba"}, + {file = "numpy-2.4.2-cp312-cp312-win32.whl", hash = "sha256:ec055f6dae239a6299cace477b479cca2fc125c5675482daf1dd886933a1076f"}, + {file = "numpy-2.4.2-cp312-cp312-win_amd64.whl", hash = "sha256:209fae046e62d0ce6435fcfe3b1a10537e858249b3d9b05829e2a05218296a85"}, + {file = "numpy-2.4.2-cp312-cp312-win_arm64.whl", hash = "sha256:fbde1b0c6e81d56f5dccd95dd4a711d9b95df1ae4009a60887e56b27e8d903fa"}, + {file = "numpy-2.4.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:25f2059807faea4b077a2b6837391b5d830864b3543627f381821c646f31a63c"}, + {file = "numpy-2.4.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bd3a7a9f5847d2fb8c2c6d1c862fa109c31a9abeca1a3c2bd5a64572955b2979"}, + {file = "numpy-2.4.2-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:8e4549f8a3c6d13d55041925e912bfd834285ef1dd64d6bc7d542583355e2e98"}, + {file = "numpy-2.4.2-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:aea4f66ff44dfddf8c2cffd66ba6538c5ec67d389285292fe428cb2c738c8aef"}, + {file = "numpy-2.4.2-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c3cd545784805de05aafe1dde61752ea49a359ccba9760c1e5d1c88a93bbf2b7"}, + {file = "numpy-2.4.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d0d9b7c93578baafcbc5f0b83eaf17b79d345c6f36917ba0c67f45226911d499"}, + {file = "numpy-2.4.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f74f0f7779cc7ae07d1810aab8ac6b1464c3eafb9e283a40da7309d5e6e48fbb"}, + {file = "numpy-2.4.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:c7ac672d699bf36275c035e16b65539931347d68b70667d28984c9fb34e07fa7"}, + {file = "numpy-2.4.2-cp313-cp313-win32.whl", hash = "sha256:8e9afaeb0beff068b4d9cd20d322ba0ee1cecfb0b08db145e4ab4dd44a6b5110"}, + {file = "numpy-2.4.2-cp313-cp313-win_amd64.whl", hash = "sha256:7df2de1e4fba69a51c06c28f5a3de36731eb9639feb8e1cf7e4a7b0daf4cf622"}, + {file = "numpy-2.4.2-cp313-cp313-win_arm64.whl", hash = "sha256:0fece1d1f0a89c16b03442eae5c56dc0be0c7883b5d388e0c03f53019a4bfd71"}, + {file = "numpy-2.4.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5633c0da313330fd20c484c78cdd3f9b175b55e1a766c4a174230c6b70ad8262"}, + {file = "numpy-2.4.2-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:d9f64d786b3b1dd742c946c42d15b07497ed14af1a1f3ce840cce27daa0ce913"}, + {file = "numpy-2.4.2-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:b21041e8cb6a1eb5312dd1d2f80a94d91efffb7a06b70597d44f1bd2dfc315ab"}, + {file = "numpy-2.4.2-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:00ab83c56211a1d7c07c25e3217ea6695e50a3e2f255053686b081dc0b091a82"}, + {file = "numpy-2.4.2-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fb882da679409066b4603579619341c6d6898fc83a8995199d5249f986e8e8f"}, + {file = "numpy-2.4.2-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:66cb9422236317f9d44b67b4d18f44efe6e9c7f8794ac0462978513359461554"}, + {file = "numpy-2.4.2-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:0f01dcf33e73d80bd8dc0f20a71303abbafa26a19e23f6b68d1aa9990af90257"}, + {file = "numpy-2.4.2-cp313-cp313t-win32.whl", hash = "sha256:52b913ec40ff7ae845687b0b34d8d93b60cb66dcee06996dd5c99f2fc9328657"}, + {file = "numpy-2.4.2-cp313-cp313t-win_amd64.whl", hash = "sha256:5eea80d908b2c1f91486eb95b3fb6fab187e569ec9752ab7d9333d2e66bf2d6b"}, + {file = "numpy-2.4.2-cp313-cp313t-win_arm64.whl", hash = "sha256:fd49860271d52127d61197bb50b64f58454e9f578cb4b2c001a6de8b1f50b0b1"}, + {file = "numpy-2.4.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:444be170853f1f9d528428eceb55f12918e4fda5d8805480f36a002f1415e09b"}, + {file = "numpy-2.4.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:d1240d50adff70c2a88217698ca844723068533f3f5c5fa6ee2e3220e3bdb000"}, + {file = "numpy-2.4.2-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:7cdde6de52fb6664b00b056341265441192d1291c130e99183ec0d4b110ff8b1"}, + {file = "numpy-2.4.2-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:cda077c2e5b780200b6b3e09d0b42205a3d1c68f30c6dceb90401c13bff8fe74"}, + {file = "numpy-2.4.2-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d30291931c915b2ab5717c2974bb95ee891a1cf22ebc16a8006bd59cd210d40a"}, + {file = "numpy-2.4.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bba37bc29d4d85761deed3954a1bc62be7cf462b9510b51d367b769a8c8df325"}, + {file = "numpy-2.4.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b2f0073ed0868db1dcd86e052d37279eef185b9c8db5bf61f30f46adac63c909"}, + {file = "numpy-2.4.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7f54844851cdb630ceb623dcec4db3240d1ac13d4990532446761baede94996a"}, + {file = "numpy-2.4.2-cp314-cp314-win32.whl", hash = "sha256:12e26134a0331d8dbd9351620f037ec470b7c75929cb8a1537f6bfe411152a1a"}, + {file = "numpy-2.4.2-cp314-cp314-win_amd64.whl", hash = "sha256:068cdb2d0d644cdb45670810894f6a0600797a69c05f1ac478e8d31670b8ee75"}, + {file = "numpy-2.4.2-cp314-cp314-win_arm64.whl", hash = "sha256:6ed0be1ee58eef41231a5c943d7d1375f093142702d5723ca2eb07db9b934b05"}, + {file = "numpy-2.4.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:98f16a80e917003a12c0580f97b5f875853ebc33e2eaa4bccfc8201ac6869308"}, + {file = "numpy-2.4.2-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:20abd069b9cda45874498b245c8015b18ace6de8546bf50dfa8cea1696ed06ef"}, + {file = "numpy-2.4.2-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:e98c97502435b53741540a5717a6749ac2ada901056c7db951d33e11c885cc7d"}, + {file = "numpy-2.4.2-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:da6cad4e82cb893db4b69105c604d805e0c3ce11501a55b5e9f9083b47d2ffe8"}, + {file = "numpy-2.4.2-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9e4424677ce4b47fe73c8b5556d876571f7c6945d264201180db2dc34f676ab5"}, + {file = "numpy-2.4.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2b8f157c8a6f20eb657e240f8985cc135598b2b46985c5bccbde7616dc9c6b1e"}, + {file = "numpy-2.4.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5daf6f3914a733336dab21a05cdec343144600e964d2fcdabaac0c0269874b2a"}, + {file = "numpy-2.4.2-cp314-cp314t-win32.whl", hash = "sha256:8c50dd1fc8826f5b26a5ee4d77ca55d88a895f4e4819c7ecc2a9f5905047a443"}, + {file = "numpy-2.4.2-cp314-cp314t-win_amd64.whl", hash = "sha256:fcf92bee92742edd401ba41135185866f7026c502617f422eb432cfeca4fe236"}, + {file = "numpy-2.4.2-cp314-cp314t-win_arm64.whl", hash = "sha256:1f92f53998a17265194018d1cc321b2e96e900ca52d54c7c77837b71b9465181"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:89f7268c009bc492f506abd6f5265defa7cb3f7487dc21d357c3d290add45082"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:e6dee3bb76aa4009d5a912180bf5b2de012532998d094acee25d9cb8dee3e44a"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:cd2bd2bbed13e213d6b55dc1d035a4f91748a7d3edc9480c13898b0353708920"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:cf28c0c1d4c4bf00f509fa7eb02c58d7caf221b50b467bcb0d9bbf1584d5c821"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e04ae107ac591763a47398bb45b568fc38f02dbc4aa44c063f67a131f99346cb"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:602f65afdef699cda27ec0b9224ae5dc43e328f4c24c689deaf77133dbee74d0"}, + {file = "numpy-2.4.2-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:be71bf1edb48ebbbf7f6337b5bfd2f895d1902f6335a5830b20141fc126ffba0"}, + {file = "numpy-2.4.2.tar.gz", hash = "sha256:659a6107e31a83c4e33f763942275fd278b21d095094044eb35569e86a21ddae"}, ] [[package]] name = "orjson" -version = "3.11.4" +version = "3.11.7" description = "Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy" optional = false -python-versions = ">=3.9" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "orjson-3.11.4-cp310-cp310-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:e3aa2118a3ece0d25489cbe48498de8a5d580e42e8d9979f65bf47900a15aba1"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a69ab657a4e6733133a3dca82768f2f8b884043714e8d2b9ba9f52b6efef5c44"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3740bffd9816fc0326ddc406098a3a8f387e42223f5f455f2a02a9f834ead80c"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:65fd2f5730b1bf7f350c6dc896173d3460d235c4be007af73986d7cd9a2acd23"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9fdc3ae730541086158d549c97852e2eea6820665d4faf0f41bf99df41bc11ea"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e10b4d65901da88845516ce9f7f9736f9638d19a1d483b3883dc0182e6e5edba"}, - {file = "orjson-3.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fb6a03a678085f64b97f9d4a9ae69376ce91a3a9e9b56a82b1580d8e1d501aff"}, - {file = "orjson-3.11.4-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:2c82e4f0b1c712477317434761fbc28b044c838b6b1240d895607441412371ac"}, - {file = "orjson-3.11.4-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:d58c166a18f44cc9e2bad03a327dc2d1a3d2e85b847133cfbafd6bfc6719bd79"}, - {file = "orjson-3.11.4-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:94f206766bf1ea30e1382e4890f763bd1eefddc580e08fec1ccdc20ddd95c827"}, - {file = "orjson-3.11.4-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:41bf25fb39a34cf8edb4398818523277ee7096689db352036a9e8437f2f3ee6b"}, - {file = "orjson-3.11.4-cp310-cp310-win32.whl", hash = "sha256:fa9627eba4e82f99ca6d29bc967f09aba446ee2b5a1ea728949ede73d313f5d3"}, - {file = "orjson-3.11.4-cp310-cp310-win_amd64.whl", hash = "sha256:23ef7abc7fca96632d8174ac115e668c1e931b8fe4dde586e92a500bf1914dcc"}, - {file = "orjson-3.11.4-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:5e59d23cd93ada23ec59a96f215139753fbfe3a4d989549bcb390f8c00370b39"}, - {file = "orjson-3.11.4-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:5c3aedecfc1beb988c27c79d52ebefab93b6c3921dbec361167e6559aba2d36d"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da9e5301f1c2caa2a9a4a303480d79c9ad73560b2e7761de742ab39fe59d9175"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8873812c164a90a79f65368f8f96817e59e35d0cc02786a5356f0e2abed78040"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5d7feb0741ebb15204e748f26c9638e6665a5fa93c37a2c73d64f1669b0ddc63"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:01ee5487fefee21e6910da4c2ee9eef005bee568a0879834df86f888d2ffbdd9"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3d40d46f348c0321df01507f92b95a377240c4ec31985225a6668f10e2676f9a"}, - {file = "orjson-3.11.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95713e5fc8af84d8edc75b785d2386f653b63d62b16d681687746734b4dfc0be"}, - {file = "orjson-3.11.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ad73ede24f9083614d6c4ca9a85fe70e33be7bf047ec586ee2363bc7418fe4d7"}, - {file = "orjson-3.11.4-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:842289889de515421f3f224ef9c1f1efb199a32d76d8d2ca2706fa8afe749549"}, - {file = "orjson-3.11.4-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:3b2427ed5791619851c52a1261b45c233930977e7de8cf36de05636c708fa905"}, - {file = "orjson-3.11.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:3c36e524af1d29982e9b190573677ea02781456b2e537d5840e4538a5ec41907"}, - {file = "orjson-3.11.4-cp311-cp311-win32.whl", hash = "sha256:87255b88756eab4a68ec61837ca754e5d10fa8bc47dc57f75cedfeaec358d54c"}, - {file = "orjson-3.11.4-cp311-cp311-win_amd64.whl", hash = "sha256:e2d5d5d798aba9a0e1fede8d853fa899ce2cb930ec0857365f700dffc2c7af6a"}, - {file = "orjson-3.11.4-cp311-cp311-win_arm64.whl", hash = "sha256:6bb6bb41b14c95d4f2702bce9975fda4516f1db48e500102fc4d8119032ff045"}, - {file = "orjson-3.11.4-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:d4371de39319d05d3f482f372720b841c841b52f5385bd99c61ed69d55d9ab50"}, - {file = "orjson-3.11.4-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:e41fd3b3cac850eaae78232f37325ed7d7436e11c471246b87b2cd294ec94853"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:600e0e9ca042878c7fdf189cf1b028fe2c1418cc9195f6cb9824eb6ed99cb938"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7bbf9b333f1568ef5da42bc96e18bf30fd7f8d54e9ae066d711056add508e415"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4806363144bb6e7297b8e95870e78d30a649fdc4e23fc84daa80c8ebd366ce44"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ad355e8308493f527d41154e9053b86a5be892b3b359a5c6d5d95cda23601cb2"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c8a7517482667fb9f0ff1b2f16fe5829296ed7a655d04d68cd9711a4d8a4e708"}, - {file = "orjson-3.11.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97eb5942c7395a171cbfecc4ef6701fc3c403e762194683772df4c54cfbb2210"}, - {file = "orjson-3.11.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:149d95d5e018bdd822e3f38c103b1a7c91f88d38a88aada5c4e9b3a73a244241"}, - {file = "orjson-3.11.4-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:624f3951181eb46fc47dea3d221554e98784c823e7069edb5dbd0dc826ac909b"}, - {file = "orjson-3.11.4-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:03bfa548cf35e3f8b3a96c4e8e41f753c686ff3d8e182ce275b1751deddab58c"}, - {file = "orjson-3.11.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:525021896afef44a68148f6ed8a8bf8375553d6066c7f48537657f64823565b9"}, - {file = "orjson-3.11.4-cp312-cp312-win32.whl", hash = "sha256:b58430396687ce0f7d9eeb3dd47761ca7d8fda8e9eb92b3077a7a353a75efefa"}, - {file = "orjson-3.11.4-cp312-cp312-win_amd64.whl", hash = "sha256:c6dbf422894e1e3c80a177133c0dda260f81428f9de16d61041949f6a2e5c140"}, - {file = "orjson-3.11.4-cp312-cp312-win_arm64.whl", hash = "sha256:d38d2bc06d6415852224fcc9c0bfa834c25431e466dc319f0edd56cca81aa96e"}, - {file = "orjson-3.11.4-cp313-cp313-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:2d6737d0e616a6e053c8b4acc9eccea6b6cce078533666f32d140e4f85002534"}, - {file = "orjson-3.11.4-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:afb14052690aa328cc118a8e09f07c651d301a72e44920b887c519b313d892ff"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:38aa9e65c591febb1b0aed8da4d469eba239d434c218562df179885c94e1a3ad"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f2cf4dfaf9163b0728d061bebc1e08631875c51cd30bf47cb9e3293bfbd7dcd5"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:89216ff3dfdde0e4070932e126320a1752c9d9a758d6a32ec54b3b9334991a6a"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9daa26ca8e97fae0ce8aa5d80606ef8f7914e9b129b6b5df9104266f764ce436"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5c8b2769dc31883c44a9cd126560327767f848eb95f99c36c9932f51090bfce9"}, - {file = "orjson-3.11.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1469d254b9884f984026bd9b0fa5bbab477a4bfe558bba6848086f6d43eb5e73"}, - {file = "orjson-3.11.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:68e44722541983614e37117209a194e8c3ad07838ccb3127d96863c95ec7f1e0"}, - {file = "orjson-3.11.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:8e7805fda9672c12be2f22ae124dcd7b03928d6c197544fe12174b86553f3196"}, - {file = "orjson-3.11.4-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:04b69c14615fb4434ab867bf6f38b2d649f6f300af30a6705397e895f7aec67a"}, - {file = "orjson-3.11.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:639c3735b8ae7f970066930e58cf0ed39a852d417c24acd4a25fc0b3da3c39a6"}, - {file = "orjson-3.11.4-cp313-cp313-win32.whl", hash = "sha256:6c13879c0d2964335491463302a6ca5ad98105fc5db3565499dcb80b1b4bd839"}, - {file = "orjson-3.11.4-cp313-cp313-win_amd64.whl", hash = "sha256:09bf242a4af98732db9f9a1ec57ca2604848e16f132e3f72edfd3c5c96de009a"}, - {file = "orjson-3.11.4-cp313-cp313-win_arm64.whl", hash = "sha256:a85f0adf63319d6c1ba06fb0dbf997fced64a01179cf17939a6caca662bf92de"}, - {file = "orjson-3.11.4-cp314-cp314-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:42d43a1f552be1a112af0b21c10a5f553983c2a0938d2bbb8ecd8bc9fb572803"}, - {file = "orjson-3.11.4-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:26a20f3fbc6c7ff2cb8e89c4c5897762c9d88cf37330c6a117312365d6781d54"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6e3f20be9048941c7ffa8fc523ccbd17f82e24df1549d1d1fe9317712d19938e"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:aac364c758dc87a52e68e349924d7e4ded348dedff553889e4d9f22f74785316"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d5c54a6d76e3d741dcc3f2707f8eeb9ba2a791d3adbf18f900219b62942803b1"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f28485bdca8617b79d44627f5fb04336897041dfd9fa66d383a49d09d86798bc"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bfc2a484cad3585e4ba61985a6062a4c2ed5c7925db6d39f1fa267c9d166487f"}, - {file = "orjson-3.11.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e34dbd508cb91c54f9c9788923daca129fe5b55c5b4eebe713bf5ed3791280cf"}, - {file = "orjson-3.11.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b13c478fa413d4b4ee606ec8e11c3b2e52683a640b006bb586b3041c2ca5f606"}, - {file = "orjson-3.11.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:724ca721ecc8a831b319dcd72cfa370cc380db0bf94537f08f7edd0a7d4e1780"}, - {file = "orjson-3.11.4-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:977c393f2e44845ce1b540e19a786e9643221b3323dae190668a98672d43fb23"}, - {file = "orjson-3.11.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:1e539e382cf46edec157ad66b0b0872a90d829a6b71f17cb633d6c160a223155"}, - {file = "orjson-3.11.4-cp314-cp314-win32.whl", hash = "sha256:d63076d625babab9db5e7836118bdfa086e60f37d8a174194ae720161eb12394"}, - {file = "orjson-3.11.4-cp314-cp314-win_amd64.whl", hash = "sha256:0a54d6635fa3aaa438ae32e8570b9f0de36f3f6562c308d2a2a452e8b0592db1"}, - {file = "orjson-3.11.4-cp314-cp314-win_arm64.whl", hash = "sha256:78b999999039db3cf58f6d230f524f04f75f129ba3d1ca2ed121f8657e575d3d"}, - {file = "orjson-3.11.4-cp39-cp39-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:405261b0a8c62bcbd8e2931c26fdc08714faf7025f45531541e2b29e544b545b"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:af02ff34059ee9199a3546f123a6ab4c86caf1708c79042caf0820dc290a6d4f"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0b2eba969ea4203c177c7b38b36c69519e6067ee68c34dc37081fac74c796e10"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0baa0ea43cfa5b008a28d3c07705cf3ada40e5d347f0f44994a64b1b7b4b5350"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:80fd082f5dcc0e94657c144f1b2a3a6479c44ad50be216cf0c244e567f5eae19"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1e3704d35e47d5bee811fb1cbd8599f0b4009b14d451c4c57be5a7e25eb89a13"}, - {file = "orjson-3.11.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:caa447f2b5356779d914658519c874cf3b7629e99e63391ed519c28c8aea4919"}, - {file = "orjson-3.11.4-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:bba5118143373a86f91dadb8df41d9457498226698ebdf8e11cbb54d5b0e802d"}, - {file = "orjson-3.11.4-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:622463ab81d19ef3e06868b576551587de8e4d518892d1afab71e0fbc1f9cffc"}, - {file = "orjson-3.11.4-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:3e0a700c4b82144b72946b6629968df9762552ee1344bfdb767fecdd634fbd5a"}, - {file = "orjson-3.11.4-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:6e18a5c15e764e5f3fc569b47872450b4bcea24f2a6354c0a0e95ad21045d5a9"}, - {file = "orjson-3.11.4-cp39-cp39-win32.whl", hash = "sha256:fb1c37c71cad991ef4d89c7a634b5ffb4447dbd7ae3ae13e8f5ee7f1775e7ab1"}, - {file = "orjson-3.11.4-cp39-cp39-win_amd64.whl", hash = "sha256:e2985ce8b8c42d00492d0ed79f2bd2b6460d00f2fa671dfde4bf2e02f49bf5c6"}, - {file = "orjson-3.11.4.tar.gz", hash = "sha256:39485f4ab4c9b30a3943cfe99e1a213c4776fb69e8abd68f66b83d5a0b0fdc6d"}, + {file = "orjson-3.11.7-cp310-cp310-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:a02c833f38f36546ba65a452127633afce4cf0dd7296b753d3bb54e55e5c0174"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b63c6e6738d7c3470ad01601e23376aa511e50e1f3931395b9f9c722406d1a67"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:043d3006b7d32c7e233b8cfb1f01c651013ea079e08dcef7189a29abd8befe11"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:57036b27ac8a25d81112eb0cc9835cd4833c5b16e1467816adc0015f59e870dc"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:733ae23ada68b804b222c44affed76b39e30806d38660bf1eb200520d259cc16"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5fdfad2093bdd08245f2e204d977facd5f871c88c4a71230d5bcbd0e43bf6222"}, + {file = "orjson-3.11.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cededd6738e1c153530793998e31c05086582b08315db48ab66649768f326baa"}, + {file = "orjson-3.11.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:14f440c7268c8f8633d1b3d443a434bd70cb15686117ea6beff8fdc8f5917a1e"}, + {file = "orjson-3.11.7-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:3a2479753bbb95b0ebcf7969f562cdb9668e6d12416a35b0dda79febf89cdea2"}, + {file = "orjson-3.11.7-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:71924496986275a737f38e3f22b4e0878882b3f7a310d2ff4dc96e812789120c"}, + {file = "orjson-3.11.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:b4a9eefdc70bf8bf9857f0290f973dec534ac84c35cd6a7f4083be43e7170a8f"}, + {file = "orjson-3.11.7-cp310-cp310-win32.whl", hash = "sha256:ae9e0b37a834cef7ce8f99de6498f8fad4a2c0bf6bfc3d02abd8ed56aa15b2de"}, + {file = "orjson-3.11.7-cp310-cp310-win_amd64.whl", hash = "sha256:d772afdb22555f0c58cfc741bdae44180122b3616faa1ecadb595cd526e4c993"}, + {file = "orjson-3.11.7-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:9487abc2c2086e7c8eb9a211d2ce8855bae0e92586279d0d27b341d5ad76c85c"}, + {file = "orjson-3.11.7-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:79cacb0b52f6004caf92405a7e1f11e6e2de8bdf9019e4f76b44ba045125cd6b"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c2e85fe4698b6a56d5e2ebf7ae87544d668eb6bde1ad1226c13f44663f20ec9e"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b8d14b71c0b12963fe8a62aac87119f1afdf4cb88a400f61ca5ae581449efcb5"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:91c81ef070c8f3220054115e1ef468b1c9ce8497b4e526cb9f68ab4dc0a7ac62"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:411ebaf34d735e25e358a6d9e7978954a9c9d58cfb47bc6683cdc3964cd2f910"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a16bcd08ab0bcdfc7e8801d9c4a9cc17e58418e4d48ddc6ded4e9e4b1a94062b"}, + {file = "orjson-3.11.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9c0b51672e466fd7e56230ffbae7f1639e18d0ce023351fb75da21b71bc2c960"}, + {file = "orjson-3.11.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:136dcd6a2e796dfd9ffca9fc027d778567b0b7c9968d092842d3c323cef88aa8"}, + {file = "orjson-3.11.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:7ba61079379b0ae29e117db13bda5f28d939766e410d321ec1624afc6a0b0504"}, + {file = "orjson-3.11.7-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:0527a4510c300e3b406591b0ba69b5dc50031895b0a93743526a3fc45f59d26e"}, + {file = "orjson-3.11.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a709e881723c9b18acddcfb8ba357322491ad553e277cf467e1e7e20e2d90561"}, + {file = "orjson-3.11.7-cp311-cp311-win32.whl", hash = "sha256:c43b8b5bab288b6b90dac410cca7e986a4fa747a2e8f94615aea407da706980d"}, + {file = "orjson-3.11.7-cp311-cp311-win_amd64.whl", hash = "sha256:6543001328aa857187f905308a028935864aefe9968af3848401b6fe80dbb471"}, + {file = "orjson-3.11.7-cp311-cp311-win_arm64.whl", hash = "sha256:1ee5cc7160a821dfe14f130bc8e63e7611051f964b463d9e2a3a573204446a4d"}, + {file = "orjson-3.11.7-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:bd03ea7606833655048dab1a00734a2875e3e86c276e1d772b2a02556f0d895f"}, + {file = "orjson-3.11.7-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:89e440ebc74ce8ab5c7bc4ce6757b4a6b1041becb127df818f6997b5c71aa60b"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5ede977b5fe5ac91b1dffc0a517ca4542d2ec8a6a4ff7b2652d94f640796342a"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b7b1dae39230a393df353827c855a5f176271c23434cfd2db74e0e424e693e10"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ed46f17096e28fb28d2975834836a639af7278aa87c84f68ab08fbe5b8bd75fa"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3726be79e36e526e3d9c1aceaadbfb4a04ee80a72ab47b3f3c17fefb9812e7b8"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0724e265bc548af1dedebd9cb3d24b4e1c1e685a343be43e87ba922a5c5fff2f"}, + {file = "orjson-3.11.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e7745312efa9e11c17fbd3cb3097262d079da26930ae9ae7ba28fb738367cbad"}, + {file = "orjson-3.11.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f904c24bdeabd4298f7a977ef14ca2a022ca921ed670b92ecd16ab6f3d01f867"}, + {file = "orjson-3.11.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:b9fc4d0f81f394689e0814617aadc4f2ea0e8025f38c226cbf22d3b5ddbf025d"}, + {file = "orjson-3.11.7-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:849e38203e5be40b776ed2718e587faf204d184fc9a008ae441f9442320c0cab"}, + {file = "orjson-3.11.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4682d1db3bcebd2b64757e0ddf9e87ae5f00d29d16c5cdf3a62f561d08cc3dd2"}, + {file = "orjson-3.11.7-cp312-cp312-win32.whl", hash = "sha256:f4f7c956b5215d949a1f65334cf9d7612dde38f20a95f2315deef167def91a6f"}, + {file = "orjson-3.11.7-cp312-cp312-win_amd64.whl", hash = "sha256:bf742e149121dc5648ba0a08ea0871e87b660467ef168a3a5e53bc1fbd64bb74"}, + {file = "orjson-3.11.7-cp312-cp312-win_arm64.whl", hash = "sha256:26c3b9132f783b7d7903bf1efb095fed8d4a3a85ec0d334ee8beff3d7a4749d5"}, + {file = "orjson-3.11.7-cp313-cp313-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:1d98b30cc1313d52d4af17d9c3d307b08389752ec5f2e5febdfada70b0f8c733"}, + {file = "orjson-3.11.7-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:d897e81f8d0cbd2abb82226d1860ad2e1ab3ff16d7b08c96ca00df9d45409ef4"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:814be4b49b228cfc0b3c565acf642dd7d13538f966e3ccde61f4f55be3e20785"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d06e5c5fed5caedd2e540d62e5b1c25e8c82431b9e577c33537e5fa4aa909539"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:31c80ce534ac4ea3739c5ee751270646cbc46e45aea7576a38ffec040b4029a1"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f50979824bde13d32b4320eedd513431c921102796d86be3eee0b58e58a3ecd1"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9e54f3808e2b6b945078c41aa8d9b5834b28c50843846e97807e5adb75fa9705"}, + {file = "orjson-3.11.7-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a12b80df61aab7b98b490fe9e4879925ba666fccdfcd175252ce4d9035865ace"}, + {file = "orjson-3.11.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:996b65230271f1a97026fd0e6a753f51fbc0c335d2ad0c6201f711b0da32693b"}, + {file = "orjson-3.11.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:ab49d4b2a6a1d415ddb9f37a21e02e0d5dbfe10b7870b21bf779fc21e9156157"}, + {file = "orjson-3.11.7-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:390a1dce0c055ddf8adb6aa94a73b45a4a7d7177b5c584b8d1c1947f2ba60fb3"}, + {file = "orjson-3.11.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1eb80451a9c351a71dfaf5b7ccc13ad065405217726b59fdbeadbcc544f9d223"}, + {file = "orjson-3.11.7-cp313-cp313-win32.whl", hash = "sha256:7477aa6a6ec6139c5cb1cc7b214643592169a5494d200397c7fc95d740d5fcf3"}, + {file = "orjson-3.11.7-cp313-cp313-win_amd64.whl", hash = "sha256:b9f95dcdea9d4f805daa9ddf02617a89e484c6985fa03055459f90e87d7a0757"}, + {file = "orjson-3.11.7-cp313-cp313-win_arm64.whl", hash = "sha256:800988273a014a0541483dc81021247d7eacb0c845a9d1a34a422bc718f41539"}, + {file = "orjson-3.11.7-cp314-cp314-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:de0a37f21d0d364954ad5de1970491d7fbd0fb1ef7417d4d56a36dc01ba0c0a0"}, + {file = "orjson-3.11.7-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:c2428d358d85e8da9d37cba18b8c4047c55222007a84f97156a5b22028dfbfc0"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3c4bc6c6ac52cdaa267552544c73e486fecbd710b7ac09bc024d5a78555a22f6"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd0d68edd7dfca1b2eca9361a44ac9f24b078de3481003159929a0573f21a6bf"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:623ad1b9548ef63886319c16fa317848e465a21513b31a6ad7b57443c3e0dcf5"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6e776b998ac37c0396093d10290e60283f59cfe0fc3fccbd0ccc4bd04dd19892"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:652c6c3af76716f4a9c290371ba2e390ede06f6603edb277b481daf37f6f464e"}, + {file = "orjson-3.11.7-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a56df3239294ea5964adf074c54bcc4f0ccd21636049a2cf3ca9cf03b5d03cf1"}, + {file = "orjson-3.11.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:bda117c4148e81f746655d5a3239ae9bd00cb7bc3ca178b5fc5a5997e9744183"}, + {file = "orjson-3.11.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:23d6c20517a97a9daf1d48b580fcdc6f0516c6f4b5038823426033690b4d2650"}, + {file = "orjson-3.11.7-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:8ff206156006da5b847c9304b6308a01e8cdbc8cce824e2779a5ba71c3def141"}, + {file = "orjson-3.11.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:962d046ee1765f74a1da723f4b33e3b228fe3a48bd307acce5021dfefe0e29b2"}, + {file = "orjson-3.11.7-cp314-cp314-win32.whl", hash = "sha256:89e13dd3f89f1c38a9c9eba5fbf7cdc2d1feca82f5f290864b4b7a6aac704576"}, + {file = "orjson-3.11.7-cp314-cp314-win_amd64.whl", hash = "sha256:845c3e0d8ded9c9271cd79596b9b552448b885b97110f628fb687aee2eed11c1"}, + {file = "orjson-3.11.7-cp314-cp314-win_arm64.whl", hash = "sha256:4a2e9c5be347b937a2e0203866f12bba36082e89b402ddb9e927d5822e43088d"}, + {file = "orjson-3.11.7.tar.gz", hash = "sha256:9b1a67243945819ce55d24a30b59d6a168e86220452d2c96f4d1f093e71c0c49"}, ] [[package]] @@ -1273,14 +1307,14 @@ attrs = ">=19.2.0" [[package]] name = "packaging" -version = "25.0" +version = "26.0" description = "Core utilities for Python packages" optional = false python-versions = ">=3.8" -groups = ["dev"] +groups = ["main", "dev"] files = [ - {file = "packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484"}, - {file = "packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f"}, + {file = "packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529"}, + {file = "packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4"}, ] [[package]] @@ -1290,6 +1324,7 @@ description = "Powerful data structures for data analysis, time series, and stat optional = false python-versions = ">=3.9" groups = ["main"] +markers = "python_version == \"3.10\"" files = [ {file = "pandas-2.3.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:376c6446ae31770764215a6c937f72d917f214b43560603cd60da6408f183b6c"}, {file = "pandas-2.3.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e19d192383eab2f4ceb30b412b22ea30690c9e618f78870357ae1d682912015a"}, @@ -1349,11 +1384,7 @@ files = [ ] [package.dependencies] -numpy = [ - {version = ">=1.22.4", markers = "python_version < \"3.11\""}, - {version = ">=1.23.2", markers = "python_version == \"3.11\""}, - {version = ">=1.26.0", markers = "python_version >= \"3.12\""}, -] +numpy = {version = ">=1.22.4", markers = "python_version < \"3.11\""} python-dateutil = ">=2.8.2" pytz = ">=2020.1" tzdata = ">=2022.7" @@ -1383,6 +1414,99 @@ sql-other = ["SQLAlchemy (>=2.0.0)", "adbc-driver-postgresql (>=0.8.0)", "adbc-d test = ["hypothesis (>=6.46.1)", "pytest (>=7.3.2)", "pytest-xdist (>=2.2.0)"] xml = ["lxml (>=4.9.2)"] +[[package]] +name = "pandas" +version = "3.0.0" +description = "Powerful data structures for data analysis, time series, and statistics" +optional = false +python-versions = ">=3.11" +groups = ["main"] +markers = "python_version >= \"3.11\"" +files = [ + {file = "pandas-3.0.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:d64ce01eb9cdca96a15266aa679ae50212ec52757c79204dbc7701a222401850"}, + {file = "pandas-3.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:613e13426069793aa1ec53bdcc3b86e8d32071daea138bbcf4fa959c9cdaa2e2"}, + {file = "pandas-3.0.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0192fee1f1a8e743b464a6607858ee4b071deb0b118eb143d71c2a1d170996d5"}, + {file = "pandas-3.0.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f0b853319dec8d5e0c8b875374c078ef17f2269986a78168d9bd57e49bf650ae"}, + {file = "pandas-3.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:707a9a877a876c326ae2cb640fbdc4ef63b0a7b9e2ef55c6df9942dcee8e2af9"}, + {file = "pandas-3.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:afd0aa3d0b5cda6e0b8ffc10dbcca3b09ef3cbcd3fe2b27364f85fdc04e1989d"}, + {file = "pandas-3.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:113b4cca2614ff7e5b9fee9b6f066618fe73c5a83e99d721ffc41217b2bf57dd"}, + {file = "pandas-3.0.0-cp311-cp311-win_arm64.whl", hash = "sha256:c14837eba8e99a8da1527c0280bba29b0eb842f64aa94982c5e21227966e164b"}, + {file = "pandas-3.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:9803b31f5039b3c3b10cc858c5e40054adb4b29b4d81cb2fd789f4121c8efbcd"}, + {file = "pandas-3.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:14c2a4099cd38a1d18ff108168ea417909b2dea3bd1ebff2ccf28ddb6a74d740"}, + {file = "pandas-3.0.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d257699b9a9960e6125686098d5714ac59d05222bef7a5e6af7a7fd87c650801"}, + {file = "pandas-3.0.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:69780c98f286076dcafca38d8b8eee1676adf220199c0a39f0ecbf976b68151a"}, + {file = "pandas-3.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4a66384f017240f3858a4c8a7cf21b0591c3ac885cddb7758a589f0f71e87ebb"}, + {file = "pandas-3.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:be8c515c9bc33989d97b89db66ea0cececb0f6e3c2a87fcc8b69443a6923e95f"}, + {file = "pandas-3.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:a453aad8c4f4e9f166436994a33884442ea62aa8b27d007311e87521b97246e1"}, + {file = "pandas-3.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:da768007b5a33057f6d9053563d6b74dd6d029c337d93c6d0d22a763a5c2ecc0"}, + {file = "pandas-3.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:b78d646249b9a2bc191040988c7bb524c92fa8534fb0898a0741d7e6f2ffafa6"}, + {file = "pandas-3.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bc9cba7b355cb4162442a88ce495e01cb605f17ac1e27d6596ac963504e0305f"}, + {file = "pandas-3.0.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c9a1a149aed3b6c9bf246033ff91e1b02d529546c5d6fb6b74a28fea0cf4c70"}, + {file = "pandas-3.0.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:95683af6175d884ee89471842acfca29172a85031fccdabc35e50c0984470a0e"}, + {file = "pandas-3.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1fbbb5a7288719e36b76b4f18d46ede46e7f916b6c8d9915b756b0a6c3f792b3"}, + {file = "pandas-3.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8e8b9808590fa364416b49b2a35c1f4cf2785a6c156935879e57f826df22038e"}, + {file = "pandas-3.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:98212a38a709feb90ae658cb6227ea3657c22ba8157d4b8f913cd4c950de5e7e"}, + {file = "pandas-3.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:177d9df10b3f43b70307a149d7ec49a1229a653f907aa60a48f1877d0e6be3be"}, + {file = "pandas-3.0.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2713810ad3806767b89ad3b7b69ba153e1c6ff6d9c20f9c2140379b2a98b6c98"}, + {file = "pandas-3.0.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:15d59f885ee5011daf8335dff47dcb8a912a27b4ad7826dc6cbe809fd145d327"}, + {file = "pandas-3.0.0-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:24e6547fb64d2c92665dd2adbfa4e85fa4fd70a9c070e7cfb03b629a0bbab5eb"}, + {file = "pandas-3.0.0-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:48ee04b90e2505c693d3f8e8f524dab8cb8aaf7ddcab52c92afa535e717c4812"}, + {file = "pandas-3.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:66f72fb172959af42a459e27a8d8d2c7e311ff4c1f7db6deb3b643dbc382ae08"}, + {file = "pandas-3.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:4a4a400ca18230976724a5066f20878af785f36c6756e498e94c2a5e5d57779c"}, + {file = "pandas-3.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:940eebffe55528074341a5a36515f3e4c5e25e958ebbc764c9502cfc35ba3faa"}, + {file = "pandas-3.0.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:597c08fb9fef0edf1e4fa2f9828dd27f3d78f9b8c9b4a748d435ffc55732310b"}, + {file = "pandas-3.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:447b2d68ac5edcbf94655fe909113a6dba6ef09ad7f9f60c80477825b6c489fe"}, + {file = "pandas-3.0.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:debb95c77ff3ed3ba0d9aa20c3a2f19165cc7956362f9873fce1ba0a53819d70"}, + {file = "pandas-3.0.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fedabf175e7cd82b69b74c30adbaa616de301291a5231138d7242596fc296a8d"}, + {file = "pandas-3.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:412d1a89aab46889f3033a386912efcdfa0f1131c5705ff5b668dda88305e986"}, + {file = "pandas-3.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e979d22316f9350c516479dd3a92252be2937a9531ed3a26ec324198a99cdd49"}, + {file = "pandas-3.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:083b11415b9970b6e7888800c43c82e81a06cd6b06755d84804444f0007d6bb7"}, + {file = "pandas-3.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:5db1e62cb99e739fa78a28047e861b256d17f88463c76b8dafc7c1338086dca8"}, + {file = "pandas-3.0.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:697b8f7d346c68274b1b93a170a70974cdc7d7354429894d5927c1effdcccd73"}, + {file = "pandas-3.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8cb3120f0d9467ed95e77f67a75e030b67545bcfa08964e349252d674171def2"}, + {file = "pandas-3.0.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:33fd3e6baa72899746b820c31e4b9688c8e1b7864d7aec2de7ab5035c285277a"}, + {file = "pandas-3.0.0-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8942e333dc67ceda1095227ad0febb05a3b36535e520154085db632c40ad084"}, + {file = "pandas-3.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:783ac35c4d0fe0effdb0d67161859078618b1b6587a1af15928137525217a721"}, + {file = "pandas-3.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:125eb901e233f155b268bbef9abd9afb5819db74f0e677e89a61b246228c71ac"}, + {file = "pandas-3.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b86d113b6c109df3ce0ad5abbc259fe86a1bd4adfd4a31a89da42f84f65509bb"}, + {file = "pandas-3.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:1c39eab3ad38f2d7a249095f0a3d8f8c22cc0f847e98ccf5bbe732b272e2d9fa"}, + {file = "pandas-3.0.0.tar.gz", hash = "sha256:0facf7e87d38f721f0af46fe70d97373a37701b1c09f7ed7aeeb292ade5c050f"}, +] + +[package.dependencies] +numpy = [ + {version = ">=1.26.0", markers = "python_version < \"3.14\""}, + {version = ">=2.3.3", markers = "python_version >= \"3.14\""}, +] +python-dateutil = ">=2.8.2" +tzdata = {version = "*", markers = "sys_platform == \"win32\" or sys_platform == \"emscripten\""} + +[package.extras] +all = ["PyQt5 (>=5.15.9)", "SQLAlchemy (>=2.0.36)", "adbc-driver-postgresql (>=1.2.0)", "adbc-driver-sqlite (>=1.2.0)", "beautifulsoup4 (>=4.12.3)", "bottleneck (>=1.4.2)", "fastparquet (>=2024.11.0)", "fsspec (>=2024.10.0)", "gcsfs (>=2024.10.0)", "html5lib (>=1.1)", "hypothesis (>=6.116.0)", "jinja2 (>=3.1.5)", "lxml (>=5.3.0)", "matplotlib (>=3.9.3)", "numba (>=0.60.0)", "numexpr (>=2.10.2)", "odfpy (>=1.4.1)", "openpyxl (>=3.1.5)", "psycopg2 (>=2.9.10)", "pyarrow (>=13.0.0)", "pyiceberg (>=0.8.1)", "pymysql (>=1.1.1)", "pyreadstat (>=1.2.8)", "pytest (>=8.3.4)", "pytest-xdist (>=3.6.1)", "python-calamine (>=0.3.0)", "pytz (>=2024.2)", "pyxlsb (>=1.0.10)", "qtpy (>=2.4.2)", "s3fs (>=2024.10.0)", "scipy (>=1.14.1)", "tables (>=3.10.1)", "tabulate (>=0.9.0)", "xarray (>=2024.10.0)", "xlrd (>=2.0.1)", "xlsxwriter (>=3.2.0)", "zstandard (>=0.23.0)"] +aws = ["s3fs (>=2024.10.0)"] +clipboard = ["PyQt5 (>=5.15.9)", "qtpy (>=2.4.2)"] +compression = ["zstandard (>=0.23.0)"] +computation = ["scipy (>=1.14.1)", "xarray (>=2024.10.0)"] +excel = ["odfpy (>=1.4.1)", "openpyxl (>=3.1.5)", "python-calamine (>=0.3.0)", "pyxlsb (>=1.0.10)", "xlrd (>=2.0.1)", "xlsxwriter (>=3.2.0)"] +feather = ["pyarrow (>=13.0.0)"] +fss = ["fsspec (>=2024.10.0)"] +gcp = ["gcsfs (>=2024.10.0)"] +hdf5 = ["tables (>=3.10.1)"] +html = ["beautifulsoup4 (>=4.12.3)", "html5lib (>=1.1)", "lxml (>=5.3.0)"] +iceberg = ["pyiceberg (>=0.8.1)"] +mysql = ["SQLAlchemy (>=2.0.36)", "pymysql (>=1.1.1)"] +output-formatting = ["jinja2 (>=3.1.5)", "tabulate (>=0.9.0)"] +parquet = ["pyarrow (>=13.0.0)"] +performance = ["bottleneck (>=1.4.2)", "numba (>=0.60.0)", "numexpr (>=2.10.2)"] +plot = ["matplotlib (>=3.9.3)"] +postgresql = ["SQLAlchemy (>=2.0.36)", "adbc-driver-postgresql (>=1.2.0)", "psycopg2 (>=2.9.10)"] +pyarrow = ["pyarrow (>=13.0.0)"] +spss = ["pyreadstat (>=1.2.8)"] +sql-other = ["SQLAlchemy (>=2.0.36)", "adbc-driver-postgresql (>=1.2.0)", "adbc-driver-sqlite (>=1.2.0)"] +test = ["hypothesis (>=6.116.0)", "pytest (>=8.3.4)", "pytest-xdist (>=3.6.1)"] +timezone = ["pytz (>=2024.2)"] +xml = ["lxml (>=5.3.0)"] + [[package]] name = "parso" version = "0.8.5" @@ -1448,6 +1572,69 @@ files = [ dev = ["pre-commit", "tox"] testing = ["coverage", "pytest", "pytest-benchmark"] +[[package]] +name = "polars" +version = "1.38.1" +description = "Blazingly fast DataFrame library" +optional = false +python-versions = ">=3.10" +groups = ["dev"] +files = [ + {file = "polars-1.38.1-py3-none-any.whl", hash = "sha256:a29479c48fed4984d88b656486d221f638cba45d3e961631a50ee5fdde38cb2c"}, + {file = "polars-1.38.1.tar.gz", hash = "sha256:803a2be5344ef880ad625addfb8f641995cfd777413b08a10de0897345778239"}, +] + +[package.dependencies] +polars-runtime-32 = "1.38.1" + +[package.extras] +adbc = ["adbc-driver-manager[dbapi]", "adbc-driver-sqlite[dbapi]"] +all = ["polars[async,cloudpickle,database,deltalake,excel,fsspec,graph,iceberg,numpy,pandas,plot,pyarrow,pydantic,style,timezone]"] +async = ["gevent"] +calamine = ["fastexcel (>=0.9)"] +cloudpickle = ["cloudpickle"] +connectorx = ["connectorx (>=0.3.2)"] +database = ["polars[adbc,connectorx,sqlalchemy]"] +deltalake = ["deltalake (>=1.0.0)"] +excel = ["polars[calamine,openpyxl,xlsx2csv,xlsxwriter]"] +fsspec = ["fsspec"] +gpu = ["cudf-polars-cu12"] +graph = ["matplotlib"] +iceberg = ["pyiceberg (>=0.7.1)"] +numpy = ["numpy (>=1.16.0)"] +openpyxl = ["openpyxl (>=3.0.0)"] +pandas = ["pandas", "polars[pyarrow]"] +plot = ["altair (>=5.4.0)"] +polars-cloud = ["polars_cloud (>=0.4.0)"] +pyarrow = ["pyarrow (>=7.0.0)"] +pydantic = ["pydantic"] +rt64 = ["polars-runtime-64 (==1.38.1)"] +rtcompat = ["polars-runtime-compat (==1.38.1)"] +sqlalchemy = ["polars[pandas]", "sqlalchemy"] +style = ["great-tables (>=0.8.0)"] +timezone = ["tzdata ; platform_system == \"Windows\""] +xlsx2csv = ["xlsx2csv (>=0.8.0)"] +xlsxwriter = ["xlsxwriter"] + +[[package]] +name = "polars-runtime-32" +version = "1.38.1" +description = "Blazingly fast DataFrame library" +optional = false +python-versions = ">=3.10" +groups = ["dev"] +files = [ + {file = "polars_runtime_32-1.38.1-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:18154e96044724a0ac38ce155cf63aa03c02dd70500efbbf1a61b08cadd269ef"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:c49acac34cc4049ed188f1eb67d6ff3971a39b4af7f7b734b367119970f313ac"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fef2ef2626a954e010e006cc8e4de467ecf32d08008f130cea1c78911f545323"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e8a5f7a8125e2d50e2e060296551c929aec09be23a9edcb2b12ca923f555a5ba"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:10d19cd9863e129273b18b7fcaab625b5c8143c2d22b3e549067b78efa32e4fa"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:61e8d73c614b46a00d2f853625a7569a2e4a0999333e876354ac81d1bf1bb5e2"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-win_amd64.whl", hash = "sha256:08c2b3b93509c1141ac97891294ff5c5b0c548a373f583eaaea873a4bf506437"}, + {file = "polars_runtime_32-1.38.1-cp310-abi3-win_arm64.whl", hash = "sha256:6d07d0cc832bfe4fb54b6e04218c2c27afcfa6b9498f9f6bbf262a00d58cc7c4"}, + {file = "polars_runtime_32-1.38.1.tar.gz", hash = "sha256:04f20ed1f5c58771f34296a27029dc755a9e4b1390caeaef8f317e06fdfce2ec"}, +] + [[package]] name = "prompt-toolkit" version = "3.0.52" @@ -1465,56 +1652,58 @@ wcwidth = "*" [[package]] name = "protobuf" -version = "6.33.1" +version = "6.33.5" description = "" optional = false python-versions = ">=3.9" groups = ["main"] files = [ - {file = "protobuf-6.33.1-cp310-abi3-win32.whl", hash = "sha256:f8d3fdbc966aaab1d05046d0240dd94d40f2a8c62856d41eaa141ff64a79de6b"}, - {file = "protobuf-6.33.1-cp310-abi3-win_amd64.whl", hash = "sha256:923aa6d27a92bf44394f6abf7ea0500f38769d4b07f4be41cb52bd8b1123b9ed"}, - {file = "protobuf-6.33.1-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:fe34575f2bdde76ac429ec7b570235bf0c788883e70aee90068e9981806f2490"}, - {file = "protobuf-6.33.1-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:f8adba2e44cde2d7618996b3fc02341f03f5bc3f2748be72dc7b063319276178"}, - {file = "protobuf-6.33.1-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:0f4cf01222c0d959c2b399142deb526de420be8236f22c71356e2a544e153c53"}, - {file = "protobuf-6.33.1-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:8fd7d5e0eb08cd5b87fd3df49bc193f5cfd778701f47e11d127d0afc6c39f1d1"}, - {file = "protobuf-6.33.1-cp39-cp39-win32.whl", hash = "sha256:023af8449482fa884d88b4563d85e83accab54138ae098924a985bcbb734a213"}, - {file = "protobuf-6.33.1-cp39-cp39-win_amd64.whl", hash = "sha256:df051de4fd7e5e4371334e234c62ba43763f15ab605579e04c7008c05735cd82"}, - {file = "protobuf-6.33.1-py3-none-any.whl", hash = "sha256:d595a9fd694fdeb061a62fbe10eb039cc1e444df81ec9bb70c7fc59ebcb1eafa"}, - {file = "protobuf-6.33.1.tar.gz", hash = "sha256:97f65757e8d09870de6fd973aeddb92f85435607235d20b2dfed93405d00c85b"}, + {file = "protobuf-6.33.5-cp310-abi3-win32.whl", hash = "sha256:d71b040839446bac0f4d162e758bea99c8251161dae9d0983a3b88dee345153b"}, + {file = "protobuf-6.33.5-cp310-abi3-win_amd64.whl", hash = "sha256:3093804752167bcab3998bec9f1048baae6e29505adaf1afd14a37bddede533c"}, + {file = "protobuf-6.33.5-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:a5cb85982d95d906df1e2210e58f8e4f1e3cdc088e52c921a041f9c9a0386de5"}, + {file = "protobuf-6.33.5-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:9b71e0281f36f179d00cbcb119cb19dec4d14a81393e5ea220f64b286173e190"}, + {file = "protobuf-6.33.5-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:8afa18e1d6d20af15b417e728e9f60f3aa108ee76f23c3b2c07a2c3b546d3afd"}, + {file = "protobuf-6.33.5-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:cbf16ba3350fb7b889fca858fb215967792dc125b35c7976ca4818bee3521cf0"}, + {file = "protobuf-6.33.5-cp39-cp39-win32.whl", hash = "sha256:a3157e62729aafb8df6da2c03aa5c0937c7266c626ce11a278b6eb7963c4e37c"}, + {file = "protobuf-6.33.5-cp39-cp39-win_amd64.whl", hash = "sha256:8f04fa32763dcdb4973d537d6b54e615cc61108c7cb38fe59310c3192d29510a"}, + {file = "protobuf-6.33.5-py3-none-any.whl", hash = "sha256:69915a973dd0f60f31a08b8318b73eab2bd6a392c79184b3612226b0a3f8ec02"}, + {file = "protobuf-6.33.5.tar.gz", hash = "sha256:6ddcac2a081f8b7b9642c09406bc6a4290128fce5f471cddd165960bb9119e5c"}, ] [[package]] name = "psutil" -version = "7.1.3" +version = "7.2.2" description = "Cross-platform lib for process and system monitoring." optional = false python-versions = ">=3.6" groups = ["dev"] files = [ - {file = "psutil-7.1.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0005da714eee687b4b8decd3d6cc7c6db36215c9e74e5ad2264b90c3df7d92dc"}, - {file = "psutil-7.1.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:19644c85dcb987e35eeeaefdc3915d059dac7bd1167cdcdbf27e0ce2df0c08c0"}, - {file = "psutil-7.1.3-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:95ef04cf2e5ba0ab9eaafc4a11eaae91b44f4ef5541acd2ee91d9108d00d59a7"}, - {file = "psutil-7.1.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1068c303be3a72f8e18e412c5b2a8f6d31750fb152f9cb106b54090296c9d251"}, - {file = "psutil-7.1.3-cp313-cp313t-win_amd64.whl", hash = "sha256:18349c5c24b06ac5612c0428ec2a0331c26443d259e2a0144a9b24b4395b58fa"}, - {file = "psutil-7.1.3-cp313-cp313t-win_arm64.whl", hash = "sha256:c525ffa774fe4496282fb0b1187725793de3e7c6b29e41562733cae9ada151ee"}, - {file = "psutil-7.1.3-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:b403da1df4d6d43973dc004d19cee3b848e998ae3154cc8097d139b77156c353"}, - {file = "psutil-7.1.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ad81425efc5e75da3f39b3e636293360ad8d0b49bed7df824c79764fb4ba9b8b"}, - {file = "psutil-7.1.3-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8f33a3702e167783a9213db10ad29650ebf383946e91bc77f28a5eb083496bc9"}, - {file = "psutil-7.1.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fac9cd332c67f4422504297889da5ab7e05fd11e3c4392140f7370f4208ded1f"}, - {file = "psutil-7.1.3-cp314-cp314t-win_amd64.whl", hash = "sha256:3792983e23b69843aea49c8f5b8f115572c5ab64c153bada5270086a2123c7e7"}, - {file = "psutil-7.1.3-cp314-cp314t-win_arm64.whl", hash = "sha256:31d77fcedb7529f27bb3a0472bea9334349f9a04160e8e6e5020f22c59893264"}, - {file = "psutil-7.1.3-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:2bdbcd0e58ca14996a42adf3621a6244f1bb2e2e528886959c72cf1e326677ab"}, - {file = "psutil-7.1.3-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:bc31fa00f1fbc3c3802141eede66f3a2d51d89716a194bf2cd6fc68310a19880"}, - {file = "psutil-7.1.3-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3bb428f9f05c1225a558f53e30ccbad9930b11c3fc206836242de1091d3e7dd3"}, - {file = "psutil-7.1.3-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:56d974e02ca2c8eb4812c3f76c30e28836fffc311d55d979f1465c1feeb2b68b"}, - {file = "psutil-7.1.3-cp37-abi3-win_amd64.whl", hash = "sha256:f39c2c19fe824b47484b96f9692932248a54c43799a84282cfe58d05a6449efd"}, - {file = "psutil-7.1.3-cp37-abi3-win_arm64.whl", hash = "sha256:bd0d69cee829226a761e92f28140bec9a5ee9d5b4fb4b0cc589068dbfff559b1"}, - {file = "psutil-7.1.3.tar.gz", hash = "sha256:6c86281738d77335af7aec228328e944b30930899ea760ecf33a4dba66be5e74"}, + {file = "psutil-7.2.2-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2edccc433cbfa046b980b0df0171cd25bcaeb3a68fe9022db0979e7aa74a826b"}, + {file = "psutil-7.2.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e78c8603dcd9a04c7364f1a3e670cea95d51ee865e4efb3556a3a63adef958ea"}, + {file = "psutil-7.2.2-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a571f2330c966c62aeda00dd24620425d4b0cc86881c89861fbc04549e5dc63"}, + {file = "psutil-7.2.2-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:917e891983ca3c1887b4ef36447b1e0873e70c933afc831c6b6da078ba474312"}, + {file = "psutil-7.2.2-cp313-cp313t-win_amd64.whl", hash = "sha256:ab486563df44c17f5173621c7b198955bd6b613fb87c71c161f827d3fb149a9b"}, + {file = "psutil-7.2.2-cp313-cp313t-win_arm64.whl", hash = "sha256:ae0aefdd8796a7737eccea863f80f81e468a1e4cf14d926bd9b6f5f2d5f90ca9"}, + {file = "psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00"}, + {file = "psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9"}, + {file = "psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a"}, + {file = "psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf"}, + {file = "psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1"}, + {file = "psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841"}, + {file = "psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486"}, + {file = "psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979"}, + {file = "psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9"}, + {file = "psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e"}, + {file = "psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8"}, + {file = "psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc"}, + {file = "psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988"}, + {file = "psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee"}, + {file = "psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372"}, ] [package.extras] -dev = ["abi3audit", "black", "check-manifest", "colorama ; os_name == \"nt\"", "coverage", "packaging", "pylint", "pyperf", "pypinfo", "pyreadline ; os_name == \"nt\"", "pytest", "pytest-cov", "pytest-instafail", "pytest-subtests", "pytest-xdist", "pywin32 ; os_name == \"nt\" and platform_python_implementation != \"PyPy\"", "requests", "rstcheck", "ruff", "setuptools", "sphinx", "sphinx_rtd_theme", "toml-sort", "twine", "validate-pyproject[all]", "virtualenv", "vulture", "wheel", "wheel ; os_name == \"nt\" and platform_python_implementation != \"PyPy\"", "wmi ; os_name == \"nt\" and platform_python_implementation != \"PyPy\""] -test = ["pytest", "pytest-instafail", "pytest-subtests", "pytest-xdist", "pywin32 ; os_name == \"nt\" and platform_python_implementation != \"PyPy\"", "setuptools", "wheel ; os_name == \"nt\" and platform_python_implementation != \"PyPy\"", "wmi ; os_name == \"nt\" and platform_python_implementation != \"PyPy\""] +dev = ["abi3audit", "black", "check-manifest", "colorama ; os_name == \"nt\"", "coverage", "packaging", "psleak", "pylint", "pyperf", "pypinfo", "pyreadline3 ; os_name == \"nt\"", "pytest", "pytest-cov", "pytest-instafail", "pytest-xdist", "pywin32 ; os_name == \"nt\" and implementation_name != \"pypy\"", "requests", "rstcheck", "ruff", "setuptools", "sphinx", "sphinx_rtd_theme", "toml-sort", "twine", "validate-pyproject[all]", "virtualenv", "vulture", "wheel", "wheel ; os_name == \"nt\" and implementation_name != \"pypy\"", "wmi ; os_name == \"nt\" and implementation_name != \"pypy\""] +test = ["psleak", "pytest", "pytest-instafail", "pytest-xdist", "pywin32 ; os_name == \"nt\" and implementation_name != \"pypy\"", "setuptools", "wheel ; os_name == \"nt\" and implementation_name != \"pypy\"", "wmi ; os_name == \"nt\" and implementation_name != \"pypy\""] [[package]] name = "ptyprocess" @@ -1544,18 +1733,78 @@ files = [ [package.extras] tests = ["pytest"] +[[package]] +name = "pyarrow" +version = "23.0.0" +description = "Python library for Apache Arrow" +optional = false +python-versions = ">=3.10" +groups = ["dev"] +files = [ + {file = "pyarrow-23.0.0-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:cbdc2bf5947aa4d462adcf8453cf04aee2f7932653cb67a27acd96e5e8528a67"}, + {file = "pyarrow-23.0.0-cp310-cp310-macosx_12_0_x86_64.whl", hash = "sha256:4d38c836930ce15cd31dce20114b21ba082da231c884bdc0a7b53e1477fe7f07"}, + {file = "pyarrow-23.0.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:4222ff8f76919ecf6c716175a0e5fddb5599faeed4c56d9ea41a2c42be4998b2"}, + {file = "pyarrow-23.0.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:87f06159cbe38125852657716889296c83c37b4d09a5e58f3d10245fd1f69795"}, + {file = "pyarrow-23.0.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:1675c374570d8b91ea6d4edd4608fa55951acd44e0c31bd146e091b4005de24f"}, + {file = "pyarrow-23.0.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:247374428fde4f668f138b04031a7e7077ba5fa0b5b1722fdf89a017bf0b7ee0"}, + {file = "pyarrow-23.0.0-cp310-cp310-win_amd64.whl", hash = "sha256:de53b1bd3b88a2ee93c9af412c903e57e738c083be4f6392288294513cd8b2c1"}, + {file = "pyarrow-23.0.0-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:5574d541923efcbfdf1294a2746ae3b8c2498a2dc6cd477882f6f4e7b1ac08d3"}, + {file = "pyarrow-23.0.0-cp311-cp311-macosx_12_0_x86_64.whl", hash = "sha256:2ef0075c2488932e9d3c2eb3482f9459c4be629aa673b725d5e3cf18f777f8e4"}, + {file = "pyarrow-23.0.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:65666fc269669af1ef1c14478c52222a2aa5c907f28b68fb50a203c777e4f60c"}, + {file = "pyarrow-23.0.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:4d85cb6177198f3812db4788e394b757223f60d9a9f5ad6634b3e32be1525803"}, + {file = "pyarrow-23.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1a9ff6fa4141c24a03a1a434c63c8fa97ce70f8f36bccabc18ebba905ddf0f17"}, + {file = "pyarrow-23.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:84839d060a54ae734eb60a756aeacb62885244aaa282f3c968f5972ecc7b1ecc"}, + {file = "pyarrow-23.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:a149a647dbfe928ce8830a713612aa0b16e22c64feac9d1761529778e4d4eaa5"}, + {file = "pyarrow-23.0.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:5961a9f646c232697c24f54d3419e69b4261ba8a8b66b0ac54a1851faffcbab8"}, + {file = "pyarrow-23.0.0-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:632b3e7c3d232f41d64e1a4a043fb82d44f8a349f339a1188c6a0dd9d2d47d8a"}, + {file = "pyarrow-23.0.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:76242c846db1411f1d6c2cc3823be6b86b40567ee24493344f8226ba34a81333"}, + {file = "pyarrow-23.0.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:b73519f8b52ae28127000986bf228fda781e81d3095cd2d3ece76eb5cf760e1b"}, + {file = "pyarrow-23.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:068701f6823449b1b6469120f399a1239766b117d211c5d2519d4ed5861f75de"}, + {file = "pyarrow-23.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:1801ba947015d10e23bca9dd6ef5d0e9064a81569a89b6e9a63b59224fd060df"}, + {file = "pyarrow-23.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:52265266201ec25b6839bf6bd4ea918ca6d50f31d13e1cf200b4261cd11dc25c"}, + {file = "pyarrow-23.0.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:ad96a597547af7827342ffb3c503c8316e5043bb09b47a84885ce39394c96e00"}, + {file = "pyarrow-23.0.0-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:b9edf990df77c2901e79608f08c13fbde60202334a4fcadb15c1f57bf7afee43"}, + {file = "pyarrow-23.0.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:36d1b5bc6ddcaff0083ceec7e2561ed61a51f49cce8be079ee8ed406acb6fdef"}, + {file = "pyarrow-23.0.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:4292b889cd224f403304ddda8b63a36e60f92911f89927ec8d98021845ea21be"}, + {file = "pyarrow-23.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:dfd9e133e60eaa847fd80530a1b89a052f09f695d0b9c34c235ea6b2e0924cf7"}, + {file = "pyarrow-23.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:832141cc09fac6aab1cd3719951d23301396968de87080c57c9a7634e0ecd068"}, + {file = "pyarrow-23.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:7a7d067c9a88faca655c71bcc30ee2782038d59c802d57950826a07f60d83c4c"}, + {file = "pyarrow-23.0.0-cp313-cp313t-macosx_12_0_arm64.whl", hash = "sha256:ce9486e0535a843cf85d990e2ec5820a47918235183a5c7b8b97ed7e92c2d47d"}, + {file = "pyarrow-23.0.0-cp313-cp313t-macosx_12_0_x86_64.whl", hash = "sha256:075c29aeaa685fd1182992a9ed2499c66f084ee54eea47da3eb76e125e06064c"}, + {file = "pyarrow-23.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:799965a5379589510d888be3094c2296efd186a17ca1cef5b77703d4d5121f53"}, + {file = "pyarrow-23.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:ef7cac8fe6fccd8b9e7617bfac785b0371a7fe26af59463074e4882747145d40"}, + {file = "pyarrow-23.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:15a414f710dc927132dd67c361f78c194447479555af57317066ee5116b90e9e"}, + {file = "pyarrow-23.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:3e0d2e6915eca7d786be6a77bf227fbc06d825a75b5b5fe9bcbef121dec32685"}, + {file = "pyarrow-23.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:4b317ea6e800b5704e5e5929acb6e2dc13e9276b708ea97a39eb8b345aa2658b"}, + {file = "pyarrow-23.0.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:20b187ed9550d233a872074159f765f52f9d92973191cd4b93f293a19efbe377"}, + {file = "pyarrow-23.0.0-cp314-cp314-macosx_12_0_x86_64.whl", hash = "sha256:18ec84e839b493c3886b9b5e06861962ab4adfaeb79b81c76afbd8d84c7d5fda"}, + {file = "pyarrow-23.0.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:e438dd3f33894e34fd02b26bd12a32d30d006f5852315f611aa4add6c7fab4bc"}, + {file = "pyarrow-23.0.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:a244279f240c81f135631be91146d7fa0e9e840e1dfed2aba8483eba25cd98e6"}, + {file = "pyarrow-23.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c4692e83e42438dba512a570c6eaa42be2f8b6c0f492aea27dec54bdc495103a"}, + {file = "pyarrow-23.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae7f30f898dfe44ea69654a35c93e8da4cef6606dc4c72394068fd95f8e9f54a"}, + {file = "pyarrow-23.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:5b86bb649e4112fb0614294b7d0a175c7513738876b89655605ebb87c804f861"}, + {file = "pyarrow-23.0.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:ebc017d765d71d80a3f8584ca0566b53e40464586585ac64176115baa0ada7d3"}, + {file = "pyarrow-23.0.0-cp314-cp314t-macosx_12_0_x86_64.whl", hash = "sha256:0800cc58a6d17d159df823f87ad66cefebf105b982493d4bad03ee7fab84b993"}, + {file = "pyarrow-23.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:3a7c68c722da9bb5b0f8c10e3eae71d9825a4b429b40b32709df5d1fa55beb3d"}, + {file = "pyarrow-23.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:bd5556c24622df90551063ea41f559b714aa63ca953db884cfb958559087a14e"}, + {file = "pyarrow-23.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:54810f6e6afc4ffee7c2e0051b61722fbea9a4961b46192dcfae8ea12fa09059"}, + {file = "pyarrow-23.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:14de7d48052cf4b0ed174533eafa3cfe0711b8076ad70bede32cf59f744f0d7c"}, + {file = "pyarrow-23.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:427deac1f535830a744a4f04a6ac183a64fcac4341b3f618e693c41b7b98d2b0"}, + {file = "pyarrow-23.0.0.tar.gz", hash = "sha256:180e3150e7edfcd182d3d9afba72f7cf19839a497cc76555a8dce998a8f67615"}, +] + [[package]] name = "pycparser" -version = "2.23" +version = "3.0" description = "C parser in Python" optional = false -python-versions = ">=3.8" +python-versions = ">=3.10" groups = ["main", "dev"] files = [ - {file = "pycparser-2.23-py3-none-any.whl", hash = "sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934"}, - {file = "pycparser-2.23.tar.gz", hash = "sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2"}, + {file = "pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992"}, + {file = "pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29"}, ] -markers = {main = "os_name == \"nt\" and implementation_name != \"pypy\" and implementation_name != \"PyPy\"", dev = "implementation_name == \"pypy\""} +markers = {main = "implementation_name != \"pypy\" and implementation_name != \"PyPy\" and os_name == \"nt\"", dev = "implementation_name == \"pypy\""} [[package]] name = "pydantic" @@ -1787,6 +2036,7 @@ description = "World timezone definitions, modern and historical" optional = false python-versions = "*" groups = ["main"] +markers = "python_version == \"3.10\"" files = [ {file = "pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00"}, {file = "pytz-2025.2.tar.gz", hash = "sha256:360b9e3dbb49a209c21ad61809c7fb453643e048b38924c765813546746e81c3"}, @@ -1936,14 +2186,14 @@ requests = ">=1.0.0" [[package]] name = "rich" -version = "14.2.0" +version = "14.3.2" description = "Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal" optional = false python-versions = ">=3.8.0" groups = ["dev"] files = [ - {file = "rich-14.2.0-py3-none-any.whl", hash = "sha256:76bc51fe2e57d2b1be1f96c524b890b816e334ab4c1e45888799bfaab0021edd"}, - {file = "rich-14.2.0.tar.gz", hash = "sha256:73ff50c7c0c1c77c8243079283f4edb376f0f6442433aecb8ce7e6d0b92d1fe4"}, + {file = "rich-14.3.2-py3-none-any.whl", hash = "sha256:08e67c3e90884651da3239ea668222d19bea7b589149d8014a21c633420dbb69"}, + {file = "rich-14.3.2.tar.gz", hash = "sha256:e712f11c1a562a11843306f5ed999475f09ac31ffb64281f73ab29ffdda8b3b8"}, ] [package.dependencies] @@ -1955,24 +2205,48 @@ jupyter = ["ipywidgets (>=7.5.1,<9)"] [[package]] name = "selenium" -version = "4.38.0" +version = "4.40.0" description = "Official Python bindings for Selenium WebDriver" optional = false python-versions = ">=3.10" groups = ["main"] files = [ - {file = "selenium-4.38.0-py3-none-any.whl", hash = "sha256:ed47563f188130a6fd486b327ca7ba48c5b11fb900e07d6457befdde320e35fd"}, - {file = "selenium-4.38.0.tar.gz", hash = "sha256:c117af6727859d50f622d6d0785b945c5db3e28a45ec12ad85cee2e7cc84fc4c"}, + {file = "selenium-4.40.0-py3-none-any.whl", hash = "sha256:c8823fc02e2c771d9ad9a0cf899cee7de1a57a6697e3d0b91f67566129f2b729"}, + {file = "selenium-4.40.0.tar.gz", hash = "sha256:a88f5905d88ad0b84991c2386ea39e2bbde6d6c334be38df5842318ba98eaa8c"}, ] [package.dependencies] -certifi = ">=2025.10.5" +certifi = ">=2026.1.4" trio = ">=0.31.0,<1.0" +trio-typing = ">=0.10.0" trio-websocket = ">=0.12.2,<1.0" +types-certifi = ">=2021.10.8.3" +types-urllib3 = ">=1.26.25.14" typing_extensions = ">=4.15.0,<5.0" -urllib3 = {version = ">=2.5.0,<3.0", extras = ["socks"]} +urllib3 = {version = ">=2.6.3,<3.0", extras = ["socks"]} websocket-client = ">=1.8.0,<2.0" +[[package]] +name = "setuptools" +version = "80.10.2" +description = "Easily download, build, install, upgrade, and uninstall Python packages" +optional = false +python-versions = ">=3.9" +groups = ["dev"] +files = [ + {file = "setuptools-80.10.2-py3-none-any.whl", hash = "sha256:95b30ddfb717250edb492926c92b5221f7ef3fbcc2b07579bcd4a27da21d0173"}, + {file = "setuptools-80.10.2.tar.gz", hash = "sha256:8b0e9d10c784bf7d262c4e5ec5d4ec94127ce206e8738f29a437945fbc219b70"}, +] + +[package.extras] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\"", "ruff (>=0.8.0) ; sys_platform != \"cygwin\""] +core = ["importlib_metadata (>=6) ; python_version < \"3.10\"", "jaraco.functools (>=4)", "jaraco.text (>=3.7)", "more_itertools", "more_itertools (>=8.8)", "packaging (>=24.2)", "platformdirs (>=4.2.2)", "tomli (>=2.0.1) ; python_version < \"3.11\"", "wheel (>=0.43.0)"] +cover = ["pytest-cov"] +doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "pygments-github-lexers (==0.0.5)", "pyproject-hooks (!=1.1)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-favicon", "sphinx-inline-tabs", "sphinx-lint", "sphinx-notfound-page (>=1,<2)", "sphinx-reredirects", "sphinxcontrib-towncrier", "towncrier (<24.7)"] +enabler = ["pytest-enabler (>=2.2)"] +test = ["build[virtualenv] (>=1.0.3)", "filelock (>=3.4.0)", "ini2toml[lite] (>=0.14)", "jaraco.develop (>=7.21) ; python_version >= \"3.9\" and sys_platform != \"cygwin\"", "jaraco.envs (>=2.2)", "jaraco.path (>=3.7.2)", "jaraco.test (>=5.5)", "packaging (>=24.2)", "pip (>=19.1)", "pyproject-hooks (!=1.1)", "pytest (>=6,!=8.1.*)", "pytest-home (>=0.5)", "pytest-perf ; sys_platform != \"cygwin\"", "pytest-subprocess", "pytest-timeout", "pytest-xdist (>=3)", "tomli-w (>=1.0.0)", "virtualenv (>=13.0.0)", "wheel (>=0.44.0)"] +type = ["importlib_metadata (>=7.0.2) ; python_version < \"3.10\"", "jaraco.develop (>=7.21) ; sys_platform != \"cygwin\"", "mypy (==1.14.*)", "pytest-mypy"] + [[package]] name = "shellingham" version = "1.5.4" @@ -2023,14 +2297,14 @@ files = [ [[package]] name = "soupsieve" -version = "2.8" +version = "2.8.3" description = "A modern CSS selector implementation for Beautiful Soup." optional = false python-versions = ">=3.9" groups = ["main"] files = [ - {file = "soupsieve-2.8-py3-none-any.whl", hash = "sha256:0cc76456a30e20f5d7f2e14a98a4ae2ee4e5abdc7c5ea0aafe795f344bc7984c"}, - {file = "soupsieve-2.8.tar.gz", hash = "sha256:e2dd4a40a628cb5f28f6d4b0db8800b8f581b65bb380b97de22ba5ca8d72572f"}, + {file = "soupsieve-2.8.3-py3-none-any.whl", hash = "sha256:ed64f2ba4eebeab06cc4962affce381647455978ffc1e36bb79a545b91f45a95"}, + {file = "soupsieve-2.8.3.tar.gz", hash = "sha256:3267f1eeea4251fb42728b6dfb746edc9acaffc4a45b27e19450b676586e8349"}, ] [[package]] @@ -2068,16 +2342,31 @@ files = [ [package.dependencies] pytest = ">=7.0.0,<9.0.0" +[[package]] +name = "tabulate" +version = "0.9.0" +description = "Pretty-print tabular data" +optional = false +python-versions = ">=3.7" +groups = ["dev"] +files = [ + {file = "tabulate-0.9.0-py3-none-any.whl", hash = "sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f"}, + {file = "tabulate-0.9.0.tar.gz", hash = "sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c"}, +] + +[package.extras] +widechars = ["wcwidth"] + [[package]] name = "tldextract" -version = "5.3.0" +version = "5.3.1" description = "Accurately separates a URL's subdomain, domain, and public suffix, using the Public Suffix List (PSL). By default, this includes the public ICANN TLDs and their exceptions. You can optionally support the Public Suffix List's private domains as well." optional = false -python-versions = ">=3.9" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "tldextract-5.3.0-py3-none-any.whl", hash = "sha256:f70f31d10b55c83993f55e91ecb7c5d84532a8972f22ec578ecfbe5ea2292db2"}, - {file = "tldextract-5.3.0.tar.gz", hash = "sha256:b3d2b70a1594a0ecfa6967d57251527d58e00bb5a91a74387baa0d87a0678609"}, + {file = "tldextract-5.3.1-py3-none-any.whl", hash = "sha256:6bfe36d518de569c572062b788e16a659ccaceffc486d243af0484e8ecf432d9"}, + {file = "tldextract-5.3.1.tar.gz", hash = "sha256:a72756ca170b2510315076383ea2993478f7da6f897eef1f4a5400735d5057fb"}, ] [package.dependencies] @@ -2092,77 +2381,82 @@ testing = ["mypy", "pytest", "pytest-gitignore", "pytest-mock", "responses", "ru [[package]] name = "tomli" -version = "2.3.0" +version = "2.4.0" description = "A lil' TOML parser" optional = false python-versions = ">=3.8" groups = ["dev"] markers = "python_version == \"3.10\"" files = [ - {file = "tomli-2.3.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45"}, - {file = "tomli-2.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba"}, - {file = "tomli-2.3.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d1381caf13ab9f300e30dd8feadb3de072aeb86f1d34a8569453ff32a7dea4bf"}, - {file = "tomli-2.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a0e285d2649b78c0d9027570d4da3425bdb49830a6156121360b3f8511ea3441"}, - {file = "tomli-2.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0a154a9ae14bfcf5d8917a59b51ffd5a3ac1fd149b71b47a3a104ca4edcfa845"}, - {file = "tomli-2.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:74bf8464ff93e413514fefd2be591c3b0b23231a77f901db1eb30d6f712fc42c"}, - {file = "tomli-2.3.0-cp311-cp311-win32.whl", hash = "sha256:00b5f5d95bbfc7d12f91ad8c593a1659b6387b43f054104cda404be6bda62456"}, - {file = "tomli-2.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:4dc4ce8483a5d429ab602f111a93a6ab1ed425eae3122032db7e9acf449451be"}, - {file = "tomli-2.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d7d86942e56ded512a594786a5ba0a5e521d02529b3826e7761a05138341a2ac"}, - {file = "tomli-2.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:73ee0b47d4dad1c5e996e3cd33b8a76a50167ae5f96a2607cbe8cc773506ab22"}, - {file = "tomli-2.3.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:792262b94d5d0a466afb5bc63c7daa9d75520110971ee269152083270998316f"}, - {file = "tomli-2.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4f195fe57ecceac95a66a75ac24d9d5fbc98ef0962e09b2eddec5d39375aae52"}, - {file = "tomli-2.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e31d432427dcbf4d86958c184b9bfd1e96b5b71f8eb17e6d02531f434fd335b8"}, - {file = "tomli-2.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7b0882799624980785240ab732537fcfc372601015c00f7fc367c55308c186f6"}, - {file = "tomli-2.3.0-cp312-cp312-win32.whl", hash = "sha256:ff72b71b5d10d22ecb084d345fc26f42b5143c5533db5e2eaba7d2d335358876"}, - {file = "tomli-2.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:1cb4ed918939151a03f33d4242ccd0aa5f11b3547d0cf30f7c74a408a5b99878"}, - {file = "tomli-2.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:5192f562738228945d7b13d4930baffda67b69425a7f0da96d360b0a3888136b"}, - {file = "tomli-2.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:be71c93a63d738597996be9528f4abe628d1adf5e6eb11607bc8fe1a510b5dae"}, - {file = "tomli-2.3.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c4665508bcbac83a31ff8ab08f424b665200c0e1e645d2bd9ab3d3e557b6185b"}, - {file = "tomli-2.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4021923f97266babc6ccab9f5068642a0095faa0a51a246a6a02fccbb3514eaf"}, - {file = "tomli-2.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4ea38c40145a357d513bffad0ed869f13c1773716cf71ccaa83b0fa0cc4e42f"}, - {file = "tomli-2.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ad805ea85eda330dbad64c7ea7a4556259665bdf9d2672f5dccc740eb9d3ca05"}, - {file = "tomli-2.3.0-cp313-cp313-win32.whl", hash = "sha256:97d5eec30149fd3294270e889b4234023f2c69747e555a27bd708828353ab606"}, - {file = "tomli-2.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:0c95ca56fbe89e065c6ead5b593ee64b84a26fca063b5d71a1122bf26e533999"}, - {file = "tomli-2.3.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:cebc6fe843e0733ee827a282aca4999b596241195f43b4cc371d64fc6639da9e"}, - {file = "tomli-2.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4c2ef0244c75aba9355561272009d934953817c49f47d768070c3c94355c2aa3"}, - {file = "tomli-2.3.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c22a8bf253bacc0cf11f35ad9808b6cb75ada2631c2d97c971122583b129afbc"}, - {file = "tomli-2.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0eea8cc5c5e9f89c9b90c4896a8deefc74f518db5927d0e0e8d4a80953d774d0"}, - {file = "tomli-2.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b74a0e59ec5d15127acdabd75ea17726ac4c5178ae51b85bfe39c4f8a278e879"}, - {file = "tomli-2.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b5870b50c9db823c595983571d1296a6ff3e1b88f734a4c8f6fc6188397de005"}, - {file = "tomli-2.3.0-cp314-cp314-win32.whl", hash = "sha256:feb0dacc61170ed7ab602d3d972a58f14ee3ee60494292d384649a3dc38ef463"}, - {file = "tomli-2.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:b273fcbd7fc64dc3600c098e39136522650c49bca95df2d11cf3b626422392c8"}, - {file = "tomli-2.3.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:940d56ee0410fa17ee1f12b817b37a4d4e4dc4d27340863cc67236c74f582e77"}, - {file = "tomli-2.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f85209946d1fe94416debbb88d00eb92ce9cd5266775424ff81bc959e001acaf"}, - {file = "tomli-2.3.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a56212bdcce682e56b0aaf79e869ba5d15a6163f88d5451cbde388d48b13f530"}, - {file = "tomli-2.3.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c5f3ffd1e098dfc032d4d3af5c0ac64f6d286d98bc148698356847b80fa4de1b"}, - {file = "tomli-2.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5e01decd096b1530d97d5d85cb4dff4af2d8347bd35686654a004f8dea20fc67"}, - {file = "tomli-2.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:8a35dd0e643bb2610f156cca8db95d213a90015c11fee76c946aa62b7ae7e02f"}, - {file = "tomli-2.3.0-cp314-cp314t-win32.whl", hash = "sha256:a1f7f282fe248311650081faafa5f4732bdbfef5d45fe3f2e702fbc6f2d496e0"}, - {file = "tomli-2.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:70a251f8d4ba2d9ac2542eecf008b3c8a9fc5c3f9f02c56a9d7952612be2fdba"}, - {file = "tomli-2.3.0-py3-none-any.whl", hash = "sha256:e95b1af3c5b07d9e643909b5abbec77cd9f1217e6d0bca72b0234736b9fb1f1b"}, - {file = "tomli-2.3.0.tar.gz", hash = "sha256:64be704a875d2a59753d80ee8a533c3fe183e3f06807ff7dc2232938ccb01549"}, + {file = "tomli-2.4.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b5ef256a3fd497d4973c11bf142e9ed78b150d36f5773f1ca6088c230ffc5867"}, + {file = "tomli-2.4.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:5572e41282d5268eb09a697c89a7bee84fae66511f87533a6f88bd2f7b652da9"}, + {file = "tomli-2.4.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:551e321c6ba03b55676970b47cb1b73f14a0a4dce6a3e1a9458fd6d921d72e95"}, + {file = "tomli-2.4.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5e3f639a7a8f10069d0e15408c0b96a2a828cfdec6fca05296ebcdcc28ca7c76"}, + {file = "tomli-2.4.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1b168f2731796b045128c45982d3a4874057626da0e2ef1fdd722848b741361d"}, + {file = "tomli-2.4.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:133e93646ec4300d651839d382d63edff11d8978be23da4cc106f5a18b7d0576"}, + {file = "tomli-2.4.0-cp311-cp311-win32.whl", hash = "sha256:b6c78bdf37764092d369722d9946cb65b8767bfa4110f902a1b2542d8d173c8a"}, + {file = "tomli-2.4.0-cp311-cp311-win_amd64.whl", hash = "sha256:d3d1654e11d724760cdb37a3d7691f0be9db5fbdaef59c9f532aabf87006dbaa"}, + {file = "tomli-2.4.0-cp311-cp311-win_arm64.whl", hash = "sha256:cae9c19ed12d4e8f3ebf46d1a75090e4c0dc16271c5bce1c833ac168f08fb614"}, + {file = "tomli-2.4.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:920b1de295e72887bafa3ad9f7a792f811847d57ea6b1215154030cf131f16b1"}, + {file = "tomli-2.4.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7d6d9a4aee98fac3eab4952ad1d73aee87359452d1c086b5ceb43ed02ddb16b8"}, + {file = "tomli-2.4.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:36b9d05b51e65b254ea6c2585b59d2c4cb91c8a3d91d0ed0f17591a29aaea54a"}, + {file = "tomli-2.4.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1c8a885b370751837c029ef9bc014f27d80840e48bac415f3412e6593bbc18c1"}, + {file = "tomli-2.4.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8768715ffc41f0008abe25d808c20c3d990f42b6e2e58305d5da280ae7d1fa3b"}, + {file = "tomli-2.4.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7b438885858efd5be02a9a133caf5812b8776ee0c969fea02c45e8e3f296ba51"}, + {file = "tomli-2.4.0-cp312-cp312-win32.whl", hash = "sha256:0408e3de5ec77cc7f81960c362543cbbd91ef883e3138e81b729fc3eea5b9729"}, + {file = "tomli-2.4.0-cp312-cp312-win_amd64.whl", hash = "sha256:685306e2cc7da35be4ee914fd34ab801a6acacb061b6a7abca922aaf9ad368da"}, + {file = "tomli-2.4.0-cp312-cp312-win_arm64.whl", hash = "sha256:5aa48d7c2356055feef06a43611fc401a07337d5b006be13a30f6c58f869e3c3"}, + {file = "tomli-2.4.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:84d081fbc252d1b6a982e1870660e7330fb8f90f676f6e78b052ad4e64714bf0"}, + {file = "tomli-2.4.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:9a08144fa4cba33db5255f9b74f0b89888622109bd2776148f2597447f92a94e"}, + {file = "tomli-2.4.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c73add4bb52a206fd0c0723432db123c0c75c280cbd67174dd9d2db228ebb1b4"}, + {file = "tomli-2.4.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fb2945cbe303b1419e2706e711b7113da57b7db31ee378d08712d678a34e51e"}, + {file = "tomli-2.4.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:bbb1b10aa643d973366dc2cb1ad94f99c1726a02343d43cbc011edbfac579e7c"}, + {file = "tomli-2.4.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4cbcb367d44a1f0c2be408758b43e1ffb5308abe0ea222897d6bfc8e8281ef2f"}, + {file = "tomli-2.4.0-cp313-cp313-win32.whl", hash = "sha256:7d49c66a7d5e56ac959cb6fc583aff0651094ec071ba9ad43df785abc2320d86"}, + {file = "tomli-2.4.0-cp313-cp313-win_amd64.whl", hash = "sha256:3cf226acb51d8f1c394c1b310e0e0e61fecdd7adcb78d01e294ac297dd2e7f87"}, + {file = "tomli-2.4.0-cp313-cp313-win_arm64.whl", hash = "sha256:d20b797a5c1ad80c516e41bc1fb0443ddb5006e9aaa7bda2d71978346aeb9132"}, + {file = "tomli-2.4.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:26ab906a1eb794cd4e103691daa23d95c6919cc2fa9160000ac02370cc9dd3f6"}, + {file = "tomli-2.4.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:20cedb4ee43278bc4f2fee6cb50daec836959aadaf948db5172e776dd3d993fc"}, + {file = "tomli-2.4.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:39b0b5d1b6dd03684b3fb276407ebed7090bbec989fa55838c98560c01113b66"}, + {file = "tomli-2.4.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a26d7ff68dfdb9f87a016ecfd1e1c2bacbe3108f4e0f8bcd2228ef9a766c787d"}, + {file = "tomli-2.4.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:20ffd184fb1df76a66e34bd1b36b4a4641bd2b82954befa32fe8163e79f1a702"}, + {file = "tomli-2.4.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:75c2f8bbddf170e8effc98f5e9084a8751f8174ea6ccf4fca5398436e0320bc8"}, + {file = "tomli-2.4.0-cp314-cp314-win32.whl", hash = "sha256:31d556d079d72db7c584c0627ff3a24c5d3fb4f730221d3444f3efb1b2514776"}, + {file = "tomli-2.4.0-cp314-cp314-win_amd64.whl", hash = "sha256:43e685b9b2341681907759cf3a04e14d7104b3580f808cfde1dfdb60ada85475"}, + {file = "tomli-2.4.0-cp314-cp314-win_arm64.whl", hash = "sha256:3d895d56bd3f82ddd6faaff993c275efc2ff38e52322ea264122d72729dca2b2"}, + {file = "tomli-2.4.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:5b5807f3999fb66776dbce568cc9a828544244a8eb84b84b9bafc080c99597b9"}, + {file = "tomli-2.4.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c084ad935abe686bd9c898e62a02a19abfc9760b5a79bc29644463eaf2840cb0"}, + {file = "tomli-2.4.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f2e3955efea4d1cfbcb87bc321e00dc08d2bcb737fd1d5e398af111d86db5df"}, + {file = "tomli-2.4.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0e0fe8a0b8312acf3a88077a0802565cb09ee34107813bba1c7cd591fa6cfc8d"}, + {file = "tomli-2.4.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:413540dce94673591859c4c6f794dfeaa845e98bf35d72ed59636f869ef9f86f"}, + {file = "tomli-2.4.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0dc56fef0e2c1c470aeac5b6ca8cc7b640bb93e92d9803ddaf9ea03e198f5b0b"}, + {file = "tomli-2.4.0-cp314-cp314t-win32.whl", hash = "sha256:d878f2a6707cc9d53a1be1414bbb419e629c3d6e67f69230217bb663e76b5087"}, + {file = "tomli-2.4.0-cp314-cp314t-win_amd64.whl", hash = "sha256:2add28aacc7425117ff6364fe9e06a183bb0251b03f986df0e78e974047571fd"}, + {file = "tomli-2.4.0-cp314-cp314t-win_arm64.whl", hash = "sha256:2b1e3b80e1d5e52e40e9b924ec43d81570f0e7d09d11081b797bc4692765a3d4"}, + {file = "tomli-2.4.0-py3-none-any.whl", hash = "sha256:1f776e7d669ebceb01dee46484485f43a4048746235e683bcdffacdf1fb4785a"}, + {file = "tomli-2.4.0.tar.gz", hash = "sha256:aa89c3f6c277dd275d8e243ad24f3b5e701491a860d5121f2cdd399fbb31fc9c"}, ] [[package]] name = "tornado" -version = "6.5.2" +version = "6.5.4" description = "Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed." optional = false python-versions = ">=3.9" groups = ["dev"] files = [ - {file = "tornado-6.5.2-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:2436822940d37cde62771cff8774f4f00b3c8024fe482e16ca8387b8a2724db6"}, - {file = "tornado-6.5.2-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:583a52c7aa94ee046854ba81d9ebb6c81ec0fd30386d96f7640c96dad45a03ef"}, - {file = "tornado-6.5.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b0fe179f28d597deab2842b86ed4060deec7388f1fd9c1b4a41adf8af058907e"}, - {file = "tornado-6.5.2-cp39-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b186e85d1e3536d69583d2298423744740986018e393d0321df7340e71898882"}, - {file = "tornado-6.5.2-cp39-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e792706668c87709709c18b353da1f7662317b563ff69f00bab83595940c7108"}, - {file = "tornado-6.5.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:06ceb1300fd70cb20e43b1ad8aaee0266e69e7ced38fa910ad2e03285009ce7c"}, - {file = "tornado-6.5.2-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:74db443e0f5251be86cbf37929f84d8c20c27a355dd452a5cfa2aada0d001ec4"}, - {file = "tornado-6.5.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b5e735ab2889d7ed33b32a459cac490eda71a1ba6857b0118de476ab6c366c04"}, - {file = "tornado-6.5.2-cp39-abi3-win32.whl", hash = "sha256:c6f29e94d9b37a95013bb669616352ddb82e3bfe8326fccee50583caebc8a5f0"}, - {file = "tornado-6.5.2-cp39-abi3-win_amd64.whl", hash = "sha256:e56a5af51cc30dd2cae649429af65ca2f6571da29504a07995175df14c18f35f"}, - {file = "tornado-6.5.2-cp39-abi3-win_arm64.whl", hash = "sha256:d6c33dc3672e3a1f3618eb63b7ef4683a7688e7b9e6e8f0d9aa5726360a004af"}, - {file = "tornado-6.5.2.tar.gz", hash = "sha256:ab53c8f9a0fa351e2c0741284e06c7a45da86afb544133201c5cc8578eb076a0"}, + {file = "tornado-6.5.4-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:d6241c1a16b1c9e4cc28148b1cda97dd1c6cb4fb7068ac1bedc610768dff0ba9"}, + {file = "tornado-6.5.4-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:2d50f63dda1d2cac3ae1fa23d254e16b5e38153758470e9956cbc3d813d40843"}, + {file = "tornado-6.5.4-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d1cf66105dc6acb5af613c054955b8137e34a03698aa53272dbda4afe252be17"}, + {file = "tornado-6.5.4-cp39-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:50ff0a58b0dc97939d29da29cd624da010e7f804746621c78d14b80238669335"}, + {file = "tornado-6.5.4-cp39-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e5fb5e04efa54cf0baabdd10061eb4148e0be137166146fff835745f59ab9f7f"}, + {file = "tornado-6.5.4-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:9c86b1643b33a4cd415f8d0fe53045f913bf07b4a3ef646b735a6a86047dda84"}, + {file = "tornado-6.5.4-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:6eb82872335a53dd063a4f10917b3efd28270b56a33db69009606a0312660a6f"}, + {file = "tornado-6.5.4-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:6076d5dda368c9328ff41ab5d9dd3608e695e8225d1cd0fd1e006f05da3635a8"}, + {file = "tornado-6.5.4-cp39-abi3-win32.whl", hash = "sha256:1768110f2411d5cd281bac0a090f707223ce77fd110424361092859e089b38d1"}, + {file = "tornado-6.5.4-cp39-abi3-win_amd64.whl", hash = "sha256:fa07d31e0cd85c60713f2b995da613588aa03e1303d75705dca6af8babc18ddc"}, + {file = "tornado-6.5.4-cp39-abi3-win_arm64.whl", hash = "sha256:053e6e16701eb6cbe641f308f4c1a9541f91b6261991160391bfc342e8a551a1"}, + {file = "tornado-6.5.4.tar.gz", hash = "sha256:a22fa9047405d03260b483980635f0b041989d8bcc9a313f8fe18b411d84b1d7"}, ] [[package]] @@ -2202,6 +2496,29 @@ outcome = "*" sniffio = ">=1.3.0" sortedcontainers = "*" +[[package]] +name = "trio-typing" +version = "0.10.0" +description = "Static type checking support for Trio and related projects" +optional = false +python-versions = "*" +groups = ["main"] +files = [ + {file = "trio-typing-0.10.0.tar.gz", hash = "sha256:065ee684296d52a8ab0e2374666301aec36ee5747ac0e7a61f230250f8907ac3"}, + {file = "trio_typing-0.10.0-py3-none-any.whl", hash = "sha256:6d0e7ec9d837a2fe03591031a172533fbf4a1a95baf369edebfc51d5a49f0264"}, +] + +[package.dependencies] +async-generator = "*" +importlib-metadata = "*" +mypy-extensions = ">=0.4.2" +packaging = "*" +trio = ">=0.16.0" +typing-extensions = ">=3.7.4" + +[package.extras] +mypy = ["mypy (>=1.0)"] + [[package]] name = "trio-websocket" version = "0.12.2" @@ -2238,6 +2555,30 @@ rich = ">=10.11.0" shellingham = ">=1.3.0" typing-extensions = ">=3.7.4.3" +[[package]] +name = "types-certifi" +version = "2021.10.8.3" +description = "Typing stubs for certifi" +optional = false +python-versions = "*" +groups = ["main"] +files = [ + {file = "types-certifi-2021.10.8.3.tar.gz", hash = "sha256:72cf7798d165bc0b76e1c10dd1ea3097c7063c42c21d664523b928e88b554a4f"}, + {file = "types_certifi-2021.10.8.3-py3-none-any.whl", hash = "sha256:b2d1e325e69f71f7c78e5943d410e650b4707bb0ef32e4ddf3da37f54176e88a"}, +] + +[[package]] +name = "types-urllib3" +version = "1.26.25.14" +description = "Typing stubs for urllib3" +optional = false +python-versions = "*" +groups = ["main"] +files = [ + {file = "types-urllib3-1.26.25.14.tar.gz", hash = "sha256:229b7f577c951b8c1b92c1bc2b2fdb0b49847bd2af6d1cc2a2e3dd340f3bda8f"}, + {file = "types_urllib3-1.26.25.14-py3-none-any.whl", hash = "sha256:9683bbb7fb72e32bfe9d2be6e04875fbe1b3eeec3cbb4ea231435aa7fd6b4f0e"}, +] + [[package]] name = "typing-extensions" version = "4.15.0" @@ -2267,14 +2608,15 @@ typing-extensions = ">=4.12.0" [[package]] name = "tzdata" -version = "2025.2" +version = "2025.3" description = "Provider of IANA time zone data" optional = false python-versions = ">=2" groups = ["main"] +markers = "sys_platform == \"win32\" or sys_platform == \"emscripten\" or python_version == \"3.10\"" files = [ - {file = "tzdata-2025.2-py2.py3-none-any.whl", hash = "sha256:1a403fada01ff9221ca8044d701868fa132215d84beb92242d9acd2147f667a8"}, - {file = "tzdata-2025.2.tar.gz", hash = "sha256:b60a638fcc0daffadf82fe0f57e53d06bdec2f36c4df66280ae79bce6bd6f2b9"}, + {file = "tzdata-2025.3-py2.py3-none-any.whl", hash = "sha256:06a47e5700f3081aab02b2e513160914ff0694bce9947d6b76ebd6bf57cfc5d1"}, + {file = "tzdata-2025.3.tar.gz", hash = "sha256:de39c2ca5dc7b0344f2eba86f49d614019d29f060fc4ebc8a417896a620b56a7"}, ] [[package]] @@ -2316,14 +2658,14 @@ zstd = ["backports-zstd (>=1.0.0) ; python_version < \"3.14\""] [[package]] name = "wcwidth" -version = "0.2.14" +version = "0.5.3" description = "Measures the displayed width of unicode strings in a terminal" optional = false -python-versions = ">=3.6" +python-versions = ">=3.8" groups = ["dev"] files = [ - {file = "wcwidth-0.2.14-py2.py3-none-any.whl", hash = "sha256:a7bb560c8aee30f9957e5f9895805edd20602f2d7f720186dfd906e82b4982e1"}, - {file = "wcwidth-0.2.14.tar.gz", hash = "sha256:4d478375d31bc5395a3c55c40ccdf3354688364cd61c4f6adacaa9215d0b3605"}, + {file = "wcwidth-0.5.3-py3-none-any.whl", hash = "sha256:d584eff31cd4753e1e5ff6c12e1edfdb324c995713f75d26c29807bb84bf649e"}, + {file = "wcwidth-0.5.3.tar.gz", hash = "sha256:53123b7af053c74e9fe2e92ac810301f6139e64379031f7124574212fb3b4091"}, ] [[package]] @@ -2345,81 +2687,73 @@ test = ["pytest", "websockets"] [[package]] name = "websockets" -version = "15.0.1" +version = "16.0" description = "An implementation of the WebSocket Protocol (RFC 6455 & 7692)" optional = false -python-versions = ">=3.9" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "websockets-15.0.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d63efaa0cd96cf0c5fe4d581521d9fa87744540d4bc999ae6e08595a1014b45b"}, - {file = "websockets-15.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:ac60e3b188ec7574cb761b08d50fcedf9d77f1530352db4eef1707fe9dee7205"}, - {file = "websockets-15.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:5756779642579d902eed757b21b0164cd6fe338506a8083eb58af5c372e39d9a"}, - {file = "websockets-15.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0fdfe3e2a29e4db3659dbd5bbf04560cea53dd9610273917799f1cde46aa725e"}, - {file = "websockets-15.0.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4c2529b320eb9e35af0fa3016c187dffb84a3ecc572bcee7c3ce302bfeba52bf"}, - {file = "websockets-15.0.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ac1e5c9054fe23226fb11e05a6e630837f074174c4c2f0fe442996112a6de4fb"}, - {file = "websockets-15.0.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:5df592cd503496351d6dc14f7cdad49f268d8e618f80dce0cd5a36b93c3fc08d"}, - {file = "websockets-15.0.1-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:0a34631031a8f05657e8e90903e656959234f3a04552259458aac0b0f9ae6fd9"}, - {file = "websockets-15.0.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:3d00075aa65772e7ce9e990cab3ff1de702aa09be3940d1dc88d5abf1ab8a09c"}, - {file = "websockets-15.0.1-cp310-cp310-win32.whl", hash = "sha256:1234d4ef35db82f5446dca8e35a7da7964d02c127b095e172e54397fb6a6c256"}, - {file = "websockets-15.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:39c1fec2c11dc8d89bba6b2bf1556af381611a173ac2b511cf7231622058af41"}, - {file = "websockets-15.0.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:823c248b690b2fd9303ba00c4f66cd5e2d8c3ba4aa968b2779be9532a4dad431"}, - {file = "websockets-15.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:678999709e68425ae2593acf2e3ebcbcf2e69885a5ee78f9eb80e6e371f1bf57"}, - {file = "websockets-15.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d50fd1ee42388dcfb2b3676132c78116490976f1300da28eb629272d5d93e905"}, - {file = "websockets-15.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d99e5546bf73dbad5bf3547174cd6cb8ba7273062a23808ffea025ecb1cf8562"}, - {file = "websockets-15.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:66dd88c918e3287efc22409d426c8f729688d89a0c587c88971a0faa2c2f3792"}, - {file = "websockets-15.0.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8dd8327c795b3e3f219760fa603dcae1dcc148172290a8ab15158cf85a953413"}, - {file = "websockets-15.0.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:8fdc51055e6ff4adeb88d58a11042ec9a5eae317a0a53d12c062c8a8865909e8"}, - {file = "websockets-15.0.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:693f0192126df6c2327cce3baa7c06f2a117575e32ab2308f7f8216c29d9e2e3"}, - {file = "websockets-15.0.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:54479983bd5fb469c38f2f5c7e3a24f9a4e70594cd68cd1fa6b9340dadaff7cf"}, - {file = "websockets-15.0.1-cp311-cp311-win32.whl", hash = "sha256:16b6c1b3e57799b9d38427dda63edcbe4926352c47cf88588c0be4ace18dac85"}, - {file = "websockets-15.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:27ccee0071a0e75d22cb35849b1db43f2ecd3e161041ac1ee9d2352ddf72f065"}, - {file = "websockets-15.0.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3"}, - {file = "websockets-15.0.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665"}, - {file = "websockets-15.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2"}, - {file = "websockets-15.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215"}, - {file = "websockets-15.0.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5"}, - {file = "websockets-15.0.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65"}, - {file = "websockets-15.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe"}, - {file = "websockets-15.0.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4"}, - {file = "websockets-15.0.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597"}, - {file = "websockets-15.0.1-cp312-cp312-win32.whl", hash = "sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9"}, - {file = "websockets-15.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7"}, - {file = "websockets-15.0.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:ee443ef070bb3b6ed74514f5efaa37a252af57c90eb33b956d35c8e9c10a1931"}, - {file = "websockets-15.0.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:5a939de6b7b4e18ca683218320fc67ea886038265fd1ed30173f5ce3f8e85675"}, - {file = "websockets-15.0.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:746ee8dba912cd6fc889a8147168991d50ed70447bf18bcda7039f7d2e3d9151"}, - {file = "websockets-15.0.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:595b6c3969023ecf9041b2936ac3827e4623bfa3ccf007575f04c5a6aa318c22"}, - {file = "websockets-15.0.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3c714d2fc58b5ca3e285461a4cc0c9a66bd0e24c5da9911e30158286c9b5be7f"}, - {file = "websockets-15.0.1-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0f3c1e2ab208db911594ae5b4f79addeb3501604a165019dd221c0bdcabe4db8"}, - {file = "websockets-15.0.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:229cf1d3ca6c1804400b0a9790dc66528e08a6a1feec0d5040e8b9eb14422375"}, - {file = "websockets-15.0.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:756c56e867a90fb00177d530dca4b097dd753cde348448a1012ed6c5131f8b7d"}, - {file = "websockets-15.0.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:558d023b3df0bffe50a04e710bc87742de35060580a293c2a984299ed83bc4e4"}, - {file = "websockets-15.0.1-cp313-cp313-win32.whl", hash = "sha256:ba9e56e8ceeeedb2e080147ba85ffcd5cd0711b89576b83784d8605a7df455fa"}, - {file = "websockets-15.0.1-cp313-cp313-win_amd64.whl", hash = "sha256:e09473f095a819042ecb2ab9465aee615bd9c2028e4ef7d933600a8401c79561"}, - {file = "websockets-15.0.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:5f4c04ead5aed67c8a1a20491d54cdfba5884507a48dd798ecaf13c74c4489f5"}, - {file = "websockets-15.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:abdc0c6c8c648b4805c5eacd131910d2a7f6455dfd3becab248ef108e89ab16a"}, - {file = "websockets-15.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:a625e06551975f4b7ea7102bc43895b90742746797e2e14b70ed61c43a90f09b"}, - {file = "websockets-15.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d591f8de75824cbb7acad4e05d2d710484f15f29d4a915092675ad3456f11770"}, - {file = "websockets-15.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:47819cea040f31d670cc8d324bb6435c6f133b8c7a19ec3d61634e62f8d8f9eb"}, - {file = "websockets-15.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ac017dd64572e5c3bd01939121e4d16cf30e5d7e110a119399cf3133b63ad054"}, - {file = "websockets-15.0.1-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:4a9fac8e469d04ce6c25bb2610dc535235bd4aa14996b4e6dbebf5e007eba5ee"}, - {file = "websockets-15.0.1-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:363c6f671b761efcb30608d24925a382497c12c506b51661883c3e22337265ed"}, - {file = "websockets-15.0.1-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:2034693ad3097d5355bfdacfffcbd3ef5694f9718ab7f29c29689a9eae841880"}, - {file = "websockets-15.0.1-cp39-cp39-win32.whl", hash = "sha256:3b1ac0d3e594bf121308112697cf4b32be538fb1444468fb0a6ae4feebc83411"}, - {file = "websockets-15.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:b7643a03db5c95c799b89b31c036d5f27eeb4d259c798e878d6937d71832b1e4"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:0c9e74d766f2818bb95f84c25be4dea09841ac0f734d1966f415e4edfc4ef1c3"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:1009ee0c7739c08a0cd59de430d6de452a55e42d6b522de7aa15e6f67db0b8e1"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:76d1f20b1c7a2fa82367e04982e708723ba0e7b8d43aa643d3dcd404d74f1475"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f29d80eb9a9263b8d109135351caf568cc3f80b9928bccde535c235de55c22d9"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b359ed09954d7c18bbc1680f380c7301f92c60bf924171629c5db97febb12f04"}, - {file = "websockets-15.0.1-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:cad21560da69f4ce7658ca2cb83138fb4cf695a2ba3e475e0559e05991aa8122"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-macosx_10_15_x86_64.whl", hash = "sha256:7f493881579c90fc262d9cdbaa05a6b54b3811c2f300766748db79f098db9940"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:47b099e1f4fbc95b701b6e85768e1fcdaf1630f3cbe4765fa216596f12310e2e"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:67f2b6de947f8c757db2db9c71527933ad0019737ec374a8a6be9a956786aaf9"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d08eb4c2b7d6c41da6ca0600c077e93f5adcfd979cd777d747e9ee624556da4b"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4b826973a4a2ae47ba357e4e82fa44a463b8f168e1ca775ac64521442b19e87f"}, - {file = "websockets-15.0.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:21c1fa28a6a7e3cbdc171c694398b6df4744613ce9b36b1a498e816787e28123"}, - {file = "websockets-15.0.1-py3-none-any.whl", hash = "sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f"}, - {file = "websockets-15.0.1.tar.gz", hash = "sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee"}, + {file = "websockets-16.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:04cdd5d2d1dacbad0a7bf36ccbcd3ccd5a30ee188f2560b7a62a30d14107b31a"}, + {file = "websockets-16.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:8ff32bb86522a9e5e31439a58addbb0166f0204d64066fb955265c4e214160f0"}, + {file = "websockets-16.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:583b7c42688636f930688d712885cf1531326ee05effd982028212ccc13e5957"}, + {file = "websockets-16.0-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:7d837379b647c0c4c2355c2499723f82f1635fd2c26510e1f587d89bc2199e72"}, + {file = "websockets-16.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:df57afc692e517a85e65b72e165356ed1df12386ecb879ad5693be08fac65dde"}, + {file = "websockets-16.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:2b9f1e0d69bc60a4a87349d50c09a037a2607918746f07de04df9e43252c77a3"}, + {file = "websockets-16.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:335c23addf3d5e6a8633f9f8eda77efad001671e80b95c491dd0924587ece0b3"}, + {file = "websockets-16.0-cp310-cp310-win32.whl", hash = "sha256:37b31c1623c6605e4c00d466c9d633f9b812ea430c11c8a278774a1fde1acfa9"}, + {file = "websockets-16.0-cp310-cp310-win_amd64.whl", hash = "sha256:8e1dab317b6e77424356e11e99a432b7cb2f3ec8c5ab4dabbcee6add48f72b35"}, + {file = "websockets-16.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:31a52addea25187bde0797a97d6fc3d2f92b6f72a9370792d65a6e84615ac8a8"}, + {file = "websockets-16.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:417b28978cdccab24f46400586d128366313e8a96312e4b9362a4af504f3bbad"}, + {file = "websockets-16.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:af80d74d4edfa3cb9ed973a0a5ba2b2a549371f8a741e0800cb07becdd20f23d"}, + {file = "websockets-16.0-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:08d7af67b64d29823fed316505a89b86705f2b7981c07848fb5e3ea3020c1abe"}, + {file = "websockets-16.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7be95cfb0a4dae143eaed2bcba8ac23f4892d8971311f1b06f3c6b78952ee70b"}, + {file = "websockets-16.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d6297ce39ce5c2e6feb13c1a996a2ded3b6832155fcfc920265c76f24c7cceb5"}, + {file = "websockets-16.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1c1b30e4f497b0b354057f3467f56244c603a79c0d1dafce1d16c283c25f6e64"}, + {file = "websockets-16.0-cp311-cp311-win32.whl", hash = "sha256:5f451484aeb5cafee1ccf789b1b66f535409d038c56966d6101740c1614b86c6"}, + {file = "websockets-16.0-cp311-cp311-win_amd64.whl", hash = "sha256:8d7f0659570eefb578dacde98e24fb60af35350193e4f56e11190787bee77dac"}, + {file = "websockets-16.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:71c989cbf3254fbd5e84d3bff31e4da39c43f884e64f2551d14bb3c186230f00"}, + {file = "websockets-16.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:8b6e209ffee39ff1b6d0fa7bfef6de950c60dfb91b8fcead17da4ee539121a79"}, + {file = "websockets-16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:86890e837d61574c92a97496d590968b23c2ef0aeb8a9bc9421d174cd378ae39"}, + {file = "websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:9b5aca38b67492ef518a8ab76851862488a478602229112c4b0d58d63a7a4d5c"}, + {file = "websockets-16.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e0334872c0a37b606418ac52f6ab9cfd17317ac26365f7f65e203e2d0d0d359f"}, + {file = "websockets-16.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a0b31e0b424cc6b5a04b8838bbaec1688834b2383256688cf47eb97412531da1"}, + {file = "websockets-16.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:485c49116d0af10ac698623c513c1cc01c9446c058a4e61e3bf6c19dff7335a2"}, + {file = "websockets-16.0-cp312-cp312-win32.whl", hash = "sha256:eaded469f5e5b7294e2bdca0ab06becb6756ea86894a47806456089298813c89"}, + {file = "websockets-16.0-cp312-cp312-win_amd64.whl", hash = "sha256:5569417dc80977fc8c2d43a86f78e0a5a22fee17565d78621b6bb264a115d4ea"}, + {file = "websockets-16.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:878b336ac47938b474c8f982ac2f7266a540adc3fa4ad74ae96fea9823a02cc9"}, + {file = "websockets-16.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:52a0fec0e6c8d9a784c2c78276a48a2bdf099e4ccc2a4cad53b27718dbfd0230"}, + {file = "websockets-16.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e6578ed5b6981005df1860a56e3617f14a6c307e6a71b4fff8c48fdc50f3ed2c"}, + {file = "websockets-16.0-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:95724e638f0f9c350bb1c2b0a7ad0e83d9cc0c9259f3ea94e40d7b02a2179ae5"}, + {file = "websockets-16.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c0204dc62a89dc9d50d682412c10b3542d748260d743500a85c13cd1ee4bde82"}, + {file = "websockets-16.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:52ac480f44d32970d66763115edea932f1c5b1312de36df06d6b219f6741eed8"}, + {file = "websockets-16.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6e5a82b677f8f6f59e8dfc34ec06ca6b5b48bc4fcda346acd093694cc2c24d8f"}, + {file = "websockets-16.0-cp313-cp313-win32.whl", hash = "sha256:abf050a199613f64c886ea10f38b47770a65154dc37181bfaff70c160f45315a"}, + {file = "websockets-16.0-cp313-cp313-win_amd64.whl", hash = "sha256:3425ac5cf448801335d6fdc7ae1eb22072055417a96cc6b31b3861f455fbc156"}, + {file = "websockets-16.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:8cc451a50f2aee53042ac52d2d053d08bf89bcb31ae799cb4487587661c038a0"}, + {file = "websockets-16.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:daa3b6ff70a9241cf6c7fc9e949d41232d9d7d26fd3522b1ad2b4d62487e9904"}, + {file = "websockets-16.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:fd3cb4adb94a2a6e2b7c0d8d05cb94e6f1c81a0cf9dc2694fb65c7e8d94c42e4"}, + {file = "websockets-16.0-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:781caf5e8eee67f663126490c2f96f40906594cb86b408a703630f95550a8c3e"}, + {file = "websockets-16.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:caab51a72c51973ca21fa8a18bd8165e1a0183f1ac7066a182ff27107b71e1a4"}, + {file = "websockets-16.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:19c4dc84098e523fd63711e563077d39e90ec6702aff4b5d9e344a60cb3c0cb1"}, + {file = "websockets-16.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:a5e18a238a2b2249c9a9235466b90e96ae4795672598a58772dd806edc7ac6d3"}, + {file = "websockets-16.0-cp314-cp314-win32.whl", hash = "sha256:a069d734c4a043182729edd3e9f247c3b2a4035415a9172fd0f1b71658a320a8"}, + {file = "websockets-16.0-cp314-cp314-win_amd64.whl", hash = "sha256:c0ee0e63f23914732c6d7e0cce24915c48f3f1512ec1d079ed01fc629dab269d"}, + {file = "websockets-16.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:a35539cacc3febb22b8f4d4a99cc79b104226a756aa7400adc722e83b0d03244"}, + {file = "websockets-16.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:b784ca5de850f4ce93ec85d3269d24d4c82f22b7212023c974c401d4980ebc5e"}, + {file = "websockets-16.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:569d01a4e7fba956c5ae4fc988f0d4e187900f5497ce46339c996dbf24f17641"}, + {file = "websockets-16.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:50f23cdd8343b984957e4077839841146f67a3d31ab0d00e6b824e74c5b2f6e8"}, + {file = "websockets-16.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:152284a83a00c59b759697b7f9e9cddf4e3c7861dd0d964b472b70f78f89e80e"}, + {file = "websockets-16.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:bc59589ab64b0022385f429b94697348a6a234e8ce22544e3681b2e9331b5944"}, + {file = "websockets-16.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:32da954ffa2814258030e5a57bc73a3635463238e797c7375dc8091327434206"}, + {file = "websockets-16.0-cp314-cp314t-win32.whl", hash = "sha256:5a4b4cc550cb665dd8a47f868c8d04c8230f857363ad3c9caf7a0c3bf8c61ca6"}, + {file = "websockets-16.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b14dc141ed6d2dde437cddb216004bcac6a1df0935d79656387bd41632ba0bbd"}, + {file = "websockets-16.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:349f83cd6c9a415428ee1005cadb5c2c56f4389bc06a9af16103c3bc3dcc8b7d"}, + {file = "websockets-16.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:4a1aba3340a8dca8db6eb5a7986157f52eb9e436b74813764241981ca4888f03"}, + {file = "websockets-16.0-pp311-pypy311_pp73-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:f4a32d1bd841d4bcbffdcb3d2ce50c09c3909fbead375ab28d0181af89fd04da"}, + {file = "websockets-16.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0298d07ee155e2e9fda5be8a9042200dd2e3bb0b8a38482156576f863a9d457c"}, + {file = "websockets-16.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:a653aea902e0324b52f1613332ddf50b00c06fdaf7e92624fbf8c77c78fa5767"}, + {file = "websockets-16.0-py3-none-any.whl", hash = "sha256:1637db62fad1dc833276dded54215f2c7fa46912301a24bd94d45d46a011ceec"}, + {file = "websockets-16.0.tar.gz", hash = "sha256:5f6261a5e56e8d5c42a4497b364ea24d94d9563e8fbd44e78ac40879c60179b5"}, ] [[package]] @@ -2437,7 +2771,27 @@ files = [ [package.dependencies] h11 = ">=0.16.0,<1" +[[package]] +name = "zipp" +version = "3.23.0" +description = "Backport of pathlib-compatible object wrapper for zip files" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "zipp-3.23.0-py3-none-any.whl", hash = "sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e"}, + {file = "zipp-3.23.0.tar.gz", hash = "sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166"}, +] + +[package.extras] +check = ["pytest-checkdocs (>=2.4)", "pytest-ruff (>=0.2.1) ; sys_platform != \"cygwin\""] +cover = ["pytest-cov"] +doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (>=3.5)", "sphinx-lint"] +enabler = ["pytest-enabler (>=2.2)"] +test = ["big-O", "jaraco.functools", "jaraco.itertools", "jaraco.test", "more_itertools", "pytest (>=6,!=8.1.*)", "pytest-ignore-flaky"] +type = ["pytest-mypy"] + [metadata] lock-version = "2.1" python-versions = ">=3.10" -content-hash = "c571829b60451314f3df0749f1f8f8b553bdfe22d4e8a183c096335cfae000ae" +content-hash = "fda79d7e690a2cf1ded85adf5ce0cc7210d2f023d72dbd55e32177e655aa17ef" diff --git a/poetry.toml b/poetry.toml new file mode 100644 index 0000000..ab1033b --- /dev/null +++ b/poetry.toml @@ -0,0 +1,2 @@ +[virtualenvs] +in-project = true diff --git a/pyproject.toml b/pyproject.toml index 739b08d..a18e112 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "WebSearcher" -version = "0.6.6" +version = "0.6.7" description = "Tools for conducting, collecting, and parsing web search" authors = [{name = "Ronald E. Robertson", email = ""}] keywords = ["web", "search", "parser"] @@ -17,8 +17,8 @@ dependencies = [ "pandas>=2.2.3", "undetected-chromedriver>=3.5.5", "selenium>=4.9.0", - "protobuf (>=6.31.1,<7.0.0)", - "orjson (>=3.10.16,<4.0.0)", + "protobuf (>=6.33.5,<7.0.0)", + "orjson (>=3.11.5,<4.0.0)", ] [project.urls] @@ -39,6 +39,10 @@ pytest = "^8.3.4" syrupy = "^4.8.1" ipykernel = "^6.29.5" typer = "^0.15.2" +pyarrow = "^23.0.0" +polars = "^1.37.1" +setuptools = "^80.9.0" +tabulate = "^0.9.0" [build-system] requires = ["poetry-core>=2.0.0,<3.0.0"] diff --git a/scripts/condense_fixtures.py b/scripts/condense_fixtures.py new file mode 100644 index 0000000..16afd0f --- /dev/null +++ b/scripts/condense_fixtures.py @@ -0,0 +1,60 @@ +"""Condense prerelease demo data into a single bz2-compressed test fixture""" + +import bz2 + +import orjson +import typer + +app = typer.Typer() + + +@app.command() +def main( + version: str = typer.Argument(..., help="Version to condense (e.g. 0.6.7)"), + data_dir: str = typer.Option("data", help="Directory containing demo data"), + output_dir: str = typer.Option("tests/fixtures", help="Output directory for fixture"), +): + """Combine prerelease demo SERPs into a bz2-compressed JSONL fixture. + + Globs data/demo-ws-v{version}*/serps.json, deduplicates by serp_id, + and writes tests/fixtures/serps-v{version}.json.bz2. + """ + from pathlib import Path + + # Collect records from all prerelease directories + pattern = f"demo-ws-v{version}*" + data_path = Path(data_dir) + all_records: dict[str, dict] = {} + total = 0 + + for d in sorted(data_path.glob(pattern)): + fp = d / "serps.json" + if not fp.exists(): + continue + with open(fp) as f: + for line in f: + r = orjson.loads(line) + all_records[r["serp_id"]] = r + total += 1 + print(f" {d.name}: {sum(1 for _ in open(fp))} records") + + if not all_records: + print(f"No data found matching {data_path / pattern}") + raise typer.Exit(1) + + # Write compressed fixture + out_path = Path(output_dir) + out_path.mkdir(parents=True, exist_ok=True) + out_file = out_path / f"serps-v{version}.json.bz2" + + with bz2.open(out_file, "wb") as f: + for r in all_records.values(): + f.write(orjson.dumps(r) + b"\n") + + size_mb = out_file.stat().st_size / 1024 / 1024 + print(f"\n total: {total} records, {len(all_records)} unique") + print(f" wrote: {out_file} ({size_mb:.2f} MB)") + + +if __name__ == "__main__": + app() diff --git a/scripts/demo_screenshot.py b/scripts/demo_screenshot.py new file mode 100644 index 0000000..c588d04 --- /dev/null +++ b/scripts/demo_screenshot.py @@ -0,0 +1,147 @@ +"""Screenshot SERP HTML for visual inspection + +Renders raw HTML from serps.json in a headless browser and saves a full-page +screenshot. Optionally highlights extracted components with colored borders +showing their classified types. Highlights are injected directly into the +BeautifulSoup elements the extractor actually finds, so borders match exactly. +""" + +import json +import os +import tempfile + +import typer +import WebSearcher as ws + +DEFAULT_DATA_DIR = os.path.join("data", f"demo-ws-v{ws.__version__}") + +app = typer.Typer() + +TYPE_COLORS = { + "knowledge": "#4285f4", + "general": "#34a853", + "discussions_and_forums": "#fbbc05", + "perspectives": "#ea4335", + "top_stories": "#ff6d01", + "people_also_ask": "#46bdc6", + "searches_related": "#7b1fa2", + "unknown": "#d32f2f", + "ad": "#f44336", +} +DEFAULT_COLOR = "#9e9e9e" + + +def load_serp_html(serps_path: str, index: int = 0) -> str: + """Load HTML from a serps.json file at the given line index""" + with open(serps_path) as f: + for i, line in enumerate(f): + if i == index: + return json.loads(line)["html"] + raise IndexError(f"No SERP at index {index} in {serps_path}") + + +def highlight_components(html: str) -> tuple[str, dict]: + """Extract, classify, and inject highlights into the actual HTML. + + Uses the same extractor and classifier pipeline as parse_serp, then + modifies the BeautifulSoup elements in-place with colored borders + and type labels before serializing back to HTML. + + Returns: + (modified_html, type_counts) + """ + from bs4 import Tag + + soup = ws.make_soup(html) + ext = ws.Extractor(soup) + ext.extract_components() + + type_counts = {} + for cmpt in ext.components: + cmpt.classify_component() + ctype = cmpt.type + type_counts[ctype] = type_counts.get(ctype, 0) + 1 + + color = TYPE_COLORS.get(ctype, DEFAULT_COLOR) + elem = cmpt.elem + + # Add border style to the element + existing_style = elem.get("style", "") + border_style = ( + f"border: 3px solid {color} !important; " + f"border-radius: 4px !important; " + f"margin-top: 24px !important; " + f"position: relative !important; " + ) + elem["style"] = f"{existing_style}; {border_style}" + + # Create a label tag + label = soup.new_tag("div") + label.string = f"{cmpt.cmpt_rank}: {ctype}" + label["style"] = ( + f"position: absolute; top: -18px; left: 4px; z-index: 9999; " + f"font: bold 11px monospace; color: {color}; " + f"background: white; padding: 0 4px; " + f"border: 1px solid {color}; border-radius: 2px; " + ) + elem.insert(0, label) + + return str(soup), type_counts + + +@app.command() +def main( + data_dir: str = typer.Option(DEFAULT_DATA_DIR, help="Directory with serps.json"), + index: int = typer.Option(0, help="SERP index (line number) in serps.json"), + output: str = typer.Option("", help="Output path (default: data_dir/screenshot.png)"), + highlight: bool = typer.Option(True, help="Highlight components with type labels"), + width: int = typer.Option(1400, help="Browser viewport width"), +) -> None: + """Screenshot a SERP from serps.json for visual inspection""" + + serps_path = os.path.join(data_dir, "serps.json") + if not os.path.exists(serps_path): + print(f"Error: {serps_path} not found") + raise typer.Exit(1) + + output_path = output or os.path.join(data_dir, "screenshot.png") + + print(f"Loading SERP {index} from {serps_path}") + html = load_serp_html(serps_path, index) + + if highlight: + html, type_counts = highlight_components(html) + print(f"Components: {dict(sorted(type_counts.items()))}") + + # Write HTML to temp file (data: URIs truncate large pages) + tmp = tempfile.NamedTemporaryFile(mode="w", suffix=".html", delete=False) + try: + tmp.write(html) + tmp.close() + + from selenium import webdriver + from selenium.webdriver.chrome.options import Options + + opts = Options() + opts.add_argument("--headless") + opts.add_argument("--no-sandbox") + opts.add_argument(f"--window-size={width},900") + + driver = webdriver.Chrome(options=opts) + driver.get(f"file://{tmp.name}") + + # Resize to full page height + height = driver.execute_script("return document.body.scrollHeight") + driver.set_window_size(width, min(height + 200, 16000)) + + driver.save_screenshot(output_path) + driver.quit() + + finally: + os.unlink(tmp.name) + + print(f"Screenshot: {output_path}") + + +if __name__ == "__main__": + app() diff --git a/scripts/demo_search.py b/scripts/demo_search.py index ebfdd1c..1004ac3 100644 --- a/scripts/demo_search.py +++ b/scripts/demo_search.py @@ -22,7 +22,7 @@ def main( data_dir: str = typer.Option(DEFAULT_DATA_DIR, help="Prefix for output files"), headless: bool = typer.Option(False, help="Run browser in headless mode"), use_subprocess: bool = typer.Option(False, help="Run browser in a separate subprocess"), - version_main: int = typer.Option(141, help="Main version of Chrome to use"), + version_main: int = typer.Option(144, help="Main version of Chrome to use"), ai_expand: bool = typer.Option(True, help="Expand AI overviews if present"), driver_executable_path: str = typer.Option("", help="Path to ChromeDriver executable"), ) -> None: diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json new file mode 100644 index 0000000..be886d9 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json @@ -0,0 +1,467 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2980000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=4IWFaar5B6mWwbkPmI2oiQE&ved=2ahUKEwi1uKmwmsSSAxW2QjABHazhBQcQgK4QegYIAQgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html#:~:text=We%20have%20three%20types%20of%20colour%20receptors%2C,visual%20system%20constructs%20the%20colours%20we%20see." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "What is the true color of our sky?", + "How to explain to a kid why the sky is blue?", + "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "We see the sky as blue, but in reality it is completely black. The blue color is simply an optical illusion caused by the scattering of sunlight in our atmosphere.", + "type": "perspectives", + "url": "https://www.facebook.com/groups/ScienceKiDuniya/posts/3440058569495988/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue?", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DT7ufKNj70z/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", + "type": "perspectives", + "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", + "type": "perspectives", + "url": "https://www.instagram.com/p/DR2aqixgJUa/" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", + "title": "Why Is the Sky Blue? - NESDIS - NOAA", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "864K+ views · 5 years ago", + "cmpt_rank": 8, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 9, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": "https://www.britannica.com › ... › Matter & Energy", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "cite": "https://www.uu.edu › dept › physics › scienceguys", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", + "title": "Why is the sky blue on Earth, but black in space or ...", + "type": "general", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[305b53af69be].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[305b53af69be].json new file mode 100644 index 0000000..dfa1ced --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[305b53af69be].json @@ -0,0 +1,632 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 748000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "Choose what you’re giving feedback on", + "img_url": null, + "text": "Choose what you’re giving feedback on|Donald Trump|45th and 47th U.S. President|45th and 47th U.S. President|About this result|Share|Share|Click to copy link|Click to copy link|Share link|Share link|Link copied|Link copied|Claim this knowledge panel|Send feedback|Overview|Overview|Overview|Books|Books|Books", + "urls": [ + { + "misc": {}, + "text": "Claim this knowledge panel", + "title": "", + "url": "https://posts.google.com/claim/?mid=/m/0cqt90" + }, + { + "misc": {}, + "text": "Overview", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump&si=AL3DRZHCeZEsX3-CsvvLZIg5Ht46TyzeLCv6JU-VYAbQYYXTU9ujPIprOzM-hzQKNCSBdKLfTJqVB7b0OPGOc1lursgMTYzpdXF96kKgT-ydlhSX0hWf7Lo%3D&sa=X&ved=2ahUKEwio9aONqcSSAxUNlmoFHbiZI1cQyNoBKAB6BAgZEAA&ictx=1" + }, + { + "misc": {}, + "text": "Books", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump+books&si=AL3DRZEg35eg0o6rvkas7wDZ2ZuaCp6q5KVFnkc8XBmxZKVnhhGLbNELffVz9J_d5LQPDN5AZb7LMQc3Bqpx6hKQFShg4uFY2Gi-LePpCp9zB8wo44N8pT8syhV2q64o-D2LfqKlsXnU&sa=X&ved=2ahUKEwio9aONqcSSAxUNlmoFHbiZI1cQyNoBKAB6BAgaEAA&ictx=1" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": " Donald Trump 45th and 47th U.S. President About this result Share Share Facebook WhatsApp X Email Click to copy link Share link Link copied Claim this knowledge panel Send feedback Overview Books", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump&udm=2&source=univ&sa=X&ved=2ahUKEwio9aONqcSSAxUNlmoFHbiZI1cQnN8JegQIHBAD" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.instagram.com/reel/DUWrLYEjNWf/" + }, + { + "misc": {}, + "text": "Age|79 years|Jun 14, 1946", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump+age&stick=H4sIAAAAAAAAAOPgE-LUz9U3SC4ssTTQEs1OttIvSM0vyEkFUkXF-XlWiempi1gFUvLzEnNSFEqKSnMLFIBCAMqYnZc3AAAA&sa=X&ved=2ahUKEwio9aONqcSSAxUNlmoFHbiZI1cQ18AJegQIJRAB" + }, + { + "misc": {}, + "text": "Party|Republican Party", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=Republican+Party&si=AL3DRZHmwLjWhgnaPB3UTu10R6S5qNLXiQiKMeezfKyB1FMsRrUdcsLX7-nPSpyKZYIqfOJDBGOdz2E57-99eIANe1Pl8wwO7G3XqVt3qiEO4uERO4KYYXvGXEXRT6jOHlGcixAxcLt6zGSb8XfSKPYd5JeDPjiWx1lTb4E4mlK9t2l0ky4-1M_qoJaYHMURBGCmYARFqYnA&sa=X&ved=2ahUKEwio9aONqcSSAxUNlmoFHbiZI1cQ18AJegQIJBAB" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.instagram.com/p/DUYFnxqEZ16/" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_results", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "Trump promises Schumer funding for NY tunnel project — if Penn Station and Dulles Airport are renamed after him", + "type": "top_stories", + "url": "https://www.cnn.com/2026/02/05/politics/schumer-trump-ny-funding-rename" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "Trump wanted Dulles Airport, Penn Station named after him — in exchange for releasing federal funds", + "type": "top_stories", + "url": "https://www.nbcnews.com/politics/donald-trump/trump-asked-dulles-penn-station-named-exchange-gateway-money-released-rcna257708" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "Democrats erupt in fury over Trump demanding Schumer help rename Dulles, Penn Station", + "type": "top_stories", + "url": "https://www.axios.com/2026/02/06/trump-schumer-dulles-penn-gateway-democrats-nyc" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 3, + "sub_type": null, + "text": null, + "title": "‘Grind the country to a halt’: Democrat urges national strike if Trump meddles in midterms", + "type": "top_stories", + "url": "https://www.theguardian.com/us-news/2026/feb/06/donald-trump-voting-midterms-democrat-national-strike" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 4, + "sub_type": null, + "text": null, + "title": "Furious Jimmy Kimmel Loses It at Donald Trump: ‘How Dare You?’", + "type": "top_stories", + "url": "https://www.thedailybeast.com/obsessed/furious-jimmy-kimmel-loses-it-at-donald-trump-how-dare-you/" + }, + { + "cite": "https://en.wikipedia.org › wiki › Donald_Trump", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "14 hours ago — Donald John Trump (born June 14, 1946) is an American politician, media personality, and businessman who is the 47th president of the United States. Read more", + "title": "Donald Trump", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Donald_Trump" + }, + { + "cite": "https://www.whitehouse.gov › Administration", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "President Donald J. Trump is returning to the White House to build upon his previous successes and use his mandate to reject the extremist policies. Read more", + "title": "President Donald J. Trump", + "type": "general", + "url": "https://www.whitehouse.gov/administration/donald-j-trump/" + }, + { + "cite": "https://trumpwhitehouse.archives.gov › people › donald...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Donald J. Trump is the 45th President of the United States . He believes the United States has incredible potential and will go on to exceed even its remarkable ... Read more", + "title": "Donald J. Trump – The White House", + "type": "general", + "url": "https://trumpwhitehouse.archives.gov/people/donald-j-trump/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": "vertical", + "text": null, + "title": "US President Donald Trump touts 'softer touch' on immigration ... YouTube · BBC News 14 hours ago", + "type": "videos", + "url": "https://www.youtube.com/watch?v=rv7c0jAm_ys" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 1, + "sub_type": "vertical", + "text": null, + "title": "Extended interview: Tom Llamas exclusive with President Trump YouTube · NBC News 1 day ago", + "type": "videos", + "url": "https://www.youtube.com/watch?v=L5qpkmzaRXo" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 2, + "sub_type": "vertical", + "text": null, + "title": "President Trump Attends the National Prayer Breakfast – The ... The White House · The White House 9 hours ago", + "type": "videos", + "url": "https://www.whitehouse.gov/videos/president-trump-attends-the-national-prayer-breakfast/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Trump’s All Over the Epstein Files, Attacks Jimmy Kimmel After the Grammys & Don Lemon Gets Arrested", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=oC8qsi9Uu7c" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "President Trump declined to address the complains of Epstein survivors when pressed by CNN's @kaitlancollins over their concerns about the extent of redactions in the files released by the Justice Department.\n\nRead more at the link in @cnn's bio.", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DUUZrEOCeON/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "WATCH: Trump says he will revoke church tax exempt status if leaders 'say something bad about' him", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=7hmzdUBt0M4" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "👀Donald Trump endorses Viktor Orbán who has spent years branding critics & the opposition as US pawns, accusing the Obama and Biden White House of meddling in Hungary’s affairs. Orbán will surely be outraged by such a direct interference in sovereign Hungary’s electoral process.", + "type": "perspectives", + "url": "https://x.com/panyiszabolcs/status/2019485967980097839?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Trump launched his new prescription drug website TrumpRx today. Now you’ll be able to buy Xanax from the reason you need Xanax. #FallonTonight", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DUZtm-6joXd/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Trump Agrees to Super Bowl Interview, House Ends Shutdown with New Spending Bill | The Tonight Show", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=CX6QBUlTo2A" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "At the National Prayer Breakfast on Thursday, President Donald Trump said he doesn't know “how a person of faith can vote for a Democrat.” “I know we have some [Democrats] here today. I don’t know why they’re here. Because they certainly don’t give us thei", + "type": "perspectives", + "url": "https://www.facebook.com/ABC10News/posts/at-the-national-prayer-breakfast-on-thursday-president-donald-trump-said-he-does/1316893117136276/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "FOX 35 Orlando. . President Donald Trump said TrumpRX makes healthcare much more affordable -- \"at a level you can't even believe.\" Here's what prices are on TrumpRX.gov.", + "type": "perspectives", + "url": "https://www.facebook.com/FOX35Orlando/videos/president-donald-trump-said-trumprx-makes-healthcare-much-more-affordable-at-a-l/902078955966377/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Jimmy Kimmel is once again at the center of President Donald Trump’s criticism. Read more in our bio link. | 📷: Gilbert Flores/Variety via Getty", + "type": "perspectives", + "url": "https://www.instagram.com/p/DUVgM0SkpcB/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "US President Donald Trump declared religion is “coming back” in a speech at the National Prayer Breakfast.", + "type": "perspectives", + "url": "https://www.facebook.com/SkyNewsAustralia/posts/us-president-donald-trump-declared-religion-is-coming-back-in-a-speech-at-the-na/1342673757890219/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "What Trump says he learned from Minneapolis chaos", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=Og51cLThBYQ" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "When Donald Trump backs off his maddest ideas, normality returns. But the spasms offer glimpses of a topsy-turvy world in which dollar assets are no longer safe", + "type": "perspectives", + "url": "https://x.com/TheEconomist/status/2019448488870748257?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" + }, + { + "cite": "https://truthsocial.com › @realDonaldTrump", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 0, + "sub_type": null, + "text": "To use this website, please enable JavaScript.", + "title": "Donald J. Trump (@realDonaldTrump)", + "type": "general", + "url": "https://truthsocial.com/@realDonaldTrump" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "President Trump Attends the National Prayer Breakfast", + "type": "recent_posts", + "url": "https://www.facebook.com/POTUS/posts/president-trump-attends-the-national-prayer-breakfast/1271317968218851/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "WARNING: AMERICA FIRST AGENDA DETECTED\n\nPresident Trump wants to…\n🇺🇸 Unleash American energy\n🇺🇸 Arrest & deport criminal illegals\n🇺🇸 Slash drug prices\n🇺🇸 Enforce the law\n🇺🇸 Put America first\n\nIf this continues… we may experience TOO MUCH WINNING. \n\nHOW DARE HE.", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUW-J_Fimz9/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": null, + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUWrLYEjNWf/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 3, + "sub_type": null, + "text": null, + "title": "All U.S. cities are safer now with the Trump administration's law and order policies. 🇺🇸", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUWThmGk7i5/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 4, + "sub_type": null, + "text": null, + "title": "MY PICK TO WIN SUPER BOWL LX IS…", + "type": "recent_posts", + "url": "https://www.tiktok.com/@realdonaldtrump/video/7603054998091484447" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 5, + "sub_type": null, + "text": null, + "title": "President Donald J. Trump completes an excellent telephone conversation with President Xi of China on trade, military, and more:", + "type": "recent_posts", + "url": "https://www.facebook.com/WhiteHouse/posts/president-donald-j-trump-completes-an-excellent-telephone-conversation-with-pres/122165172788723345/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 6, + "sub_type": null, + "text": null, + "title": "✍️INTO LAW: President Donald J. Trump signs a fiscally responsible package that SLASHES wasteful federal spending while supporting critical programs that keep our nation SAFE, SECURE, & PROSPEROUS. \n\nBIG WIN for the American People 🇺🇸", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUUNCBej2C1/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 7, + "sub_type": null, + "text": null, + "title": "PRICE CUTS ARE HERE. \n\nThanks to President Donald J. Trump's Most-Favored-Nation drug pricing, drugmakers are slashing prices on widely used medications - putting American patients FIRST. 🇺🇸💊", + "type": "recent_posts", + "url": "https://www.facebook.com/WhiteHouse/posts/price-cuts-are-here-thanks-to-president-donald-j-trumps-most-favored-nation-drug/122165091416723345/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 8, + "sub_type": null, + "text": null, + "title": "THE GOLDEN AGE IS HERE✨\n\nSiemens Energy is investing a whopping $1,000,000,000 in America, creating jobs and expanding across the country! 🇺🇸", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUTc0QlFNs5/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 9, + "sub_type": null, + "text": null, + "title": "The American people back President Trump's crackdown on illegal immigration. \n\nENFORCE THE LAW. PROTECT AMERICA 🇺🇸", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUTL5YSFHu6/" + }, + { + "cite": "https://www.whitehousehistory.org › bios › donald-j-trump", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 0, + "sub_type": null, + "text": "Donald John Trump was born in Queens, New York, on June 14, 1946 . Trump was educated at the New York Military Academy and the Wharton School of Finance and ... Read more", + "title": "Donald J. Trump", + "type": "general", + "url": "https://www.whitehousehistory.org/bios/donald-j-trump" + }, + { + "cite": "https://www.cnn.com › president-donald-trump-47", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 0, + "sub_type": null, + "text": "16 hours ago — President Donald Trump arrives for the world premiere of \"MELANIA\" at the Kennedy Center. Stefani Reynolds/Bloomberg/Getty Images. Read more", + "title": "Donald J. Trump news | CNN Politics", + "type": "general", + "url": "https://www.cnn.com/politics/president-donald-trump-47" + }, + { + "cite": "43.1M+ followers", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 0, + "sub_type": null, + "text": "43M followers · 47 following · 7853 posts · @real donald trump: “45th & 47th President of the United States”", + "title": "President Donald J. Trump (@realdonaldtrump)", + "type": "general", + "url": "https://www.instagram.com/realdonaldtrump/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 39, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json new file mode 100644 index 0000000..5f91a77 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json @@ -0,0 +1,449 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2980000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=7ieFabDdM-C4qtsP5diJ-AE&ved=2ahUKEwiSt-LkwMOSAxXXm2oFHXpFH9kQgK4QegYIAQgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "What is the true color of our sky?", + "How to explain to a kid why the sky is blue?", + "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue?", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DT7ufKNj70z/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", + "type": "perspectives", + "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the sky is blue", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "On a clear day, what is the colour of the sky?", + "type": "perspectives", + "url": "https://www.facebook.com/apna99036/posts/on-a-clear-day-what-is-the-colour-of-the-sky/1504710364995479/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue? - NESDIS - NOAA", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "864K+ views · 5 years ago", + "cmpt_rank": 8, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 9, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": "https://www.britannica.com › ... › Matter & Energy", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "cite": "https://www.uu.edu › dept › physics › scienceguys", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", + "title": "Why is the sky blue on Earth, but black in space or ...", + "type": "general", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[6aa70651b0cd].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[6aa70651b0cd].json new file mode 100644 index 0000000..15b7cbc --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[6aa70651b0cd].json @@ -0,0 +1,574 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2980000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=ZReFaf6SHo2uqtsP2fD5kAc&mstk=AUtExfD1Uiel88dwC18JdCTfYdXjwoPty236E8Y5qjP7fqrWL-l9QFxc-_4pZv4nzYZy5N6dxpmDFem9bzIIINb-w2zN-HicZPdy1gx5-eGstkHJ4yhwO8WxkTJXFBAK5PNPtg8&csui=3&ved=2ahUKEwjY746CscOSAxW7m2oFHQ88KZ0QgK4QegYIAQgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.instagram.com/reel/DIZJjPvOtIN/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.wsav.com/weather-news/why-the-sky-looks-bluer-in-fall-and-winter/#:~:text=In%20summer%2C%20the%20sun%20sits%20higher%20in,intensifying%20the%20blue%20appearance%20of%20the%20sky." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.wsav.com/weather-news/why-the-sky-looks-bluer-in-fall-and-winter/" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "What is the true color of our sky?", + "How to explain to a kid why the sky is blue?", + "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": "vertical", + "text": null, + "title": "Why Is the Sky Blue? YouTube · NASA Space Place Mar 2, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 1, + "sub_type": "vertical", + "text": null, + "title": "Space Place in a Snap: Why Is the Sky Blue? NASA+ · NASA+ Streaming Service Oct 31, 2023", + "type": "videos", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 2, + "sub_type": "vertical", + "text": null, + "title": "Why is the sky blue? It's not. You might know that as sunlight ... Instagram · Cleo Abram Apr 13, 2025", + "type": "videos", + "url": "https://www.instagram.com/reel/DIZJjPvOtIN/?hl=en" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 6, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue?", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DT7ufKNj70z/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", + "type": "perspectives", + "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the sky is blue", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "On a clear day, what is the colour of the sky?", + "type": "perspectives", + "url": "https://www.facebook.com/apna99036/posts/on-a-clear-day-what-is-the-colour-of-the-sky/1504710364995479/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue?", + "type": "images", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? | Live Science", + "type": "images", + "url": "https://www.livescience.com/planet-earth/why-is-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NASA Space Place – NASA Science for Kids", + "type": "images", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 3, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NESDIS | National Environmental ...", + "type": "images", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 4, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue?", + "type": "images", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 5, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? It's an age old question that actually ...", + "type": "images", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 6, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NASA Space Place – NASA Science for Kids", + "type": "images", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 7, + "sub_type": "medium", + "text": null, + "title": "The Physics of Light - Why is the Sky Blue, Really?", + "type": "images", + "url": "https://www.youtube.com/watch?v=zq-rDYvxAZ4" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 8, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? It's an age old question that actually ...", + "type": "images", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue? - NESDIS - NOAA", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "https://www.uu.edu › dept › physics › scienceguys", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 0, + "sub_type": null, + "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", + "title": "Why is the sky blue on Earth, but black in space or ...", + "type": "general", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[7049404a2dd6].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[7049404a2dd6].json new file mode 100644 index 0000000..b902125 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[7049404a2dd6].json @@ -0,0 +1,428 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2560000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&google_abuse=GOOGLE_ABUSE_EXEMPTION%3DID%3D5dafb1175885f93b%3ATM%3D1768890719%3AC%3D%3E%3AIP%3D108.247.126.23-%3AS%3DO1MBTYyklSBJpDcySqRlQw%3B+path%3D%2F%3B+domain%3Dgoogle.com%3B+expires%3DTue%2C+20-Jan-2026+09%3A31%3A59+GMT&sei=YCFvaebyELOHptQP-7PO2Q0&ved=2ahUKEwjYlca4v5mSAxXBmYkEHXMpKecQgK4QegYIAAgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=The%20sky%20appears%20blue%20because%20of%20the,we%20can%20see%20look%20blue%20or%20violet**" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "How to explain to a kid why the sky is blue?", + "Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...", + "Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>How to explain to a kid why the sky is blue?<|>Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...<|>Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", + "title": "Why Is the Sky Blue? | NESDIS", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the Sky Looks Blue\n\n#atmosphere #sky #bluesky #whyskyisblue #space", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DTsZrgoD0xa/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The #sky achieves its color from sunlight scattering off Earth's atmosphere , a process called #Rayleigh scattering , where tiny air molecules would scatter shorter blue #wavelengths more than longer red ones, making the sky blue during the day . \n\nWe do know so .", + "type": "perspectives", + "url": "https://x.com/PARODY_CAIT/status/2005596656985678030?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", + "type": "perspectives", + "url": "https://www.instagram.com/p/DR2aqixgJUa/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Look at the sky. It appears blue during the daytime and reddish during sunrise and sunset. Why? Let me introduce you today to a British man who explained these phenomena very clearly. Further, his work for classical black-body radiation, later played an imp", + "type": "perspectives", + "url": "https://www.facebook.com/100064519939797/posts/look-at-the-sky-it-appears-blue-during-the-daytime-and-reddish-during-sunrise-an/1257662263061093/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The sky looks blue because of Rayleigh scattering. Sunlight is actually white, not yellow. Clouds weigh millions of kilograms. The sky on Mars looks reddish-pink. The highest clouds are called noctilucent clouds. Rainbows are full circles, not arcs. You can nev", + "type": "perspectives", + "url": "https://www.facebook.com/teacherceppee/posts/the-sky-looks-blue-because-of-rayleigh-scatteringsunlight-is-actually-white-not-/1429848428511661/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the sky is blue", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" + }, + { + "cite": "859.4K+ views · 5 years ago", + "cmpt_rank": 8, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 9, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": "30.9K+ likes · 9 months ago", + "cmpt_rank": 10, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "video", + "text": "The reason we see it as blue is because our human eyes have receptors that are more significantly responsive to Blue wavelengths.", + "title": "Why is the sky blue? It's not. You might know that as sunlight ...", + "type": "general", + "url": "https://www.instagram.com/reel/DIZJjPvOtIN/?hl=en" + }, + { + "cite": "https://www.britannica.com › ... › Earth Sciences", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[7333536d2911].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[7333536d2911].json new file mode 100644 index 0000000..b880f8b --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[7333536d2911].json @@ -0,0 +1,398 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 128000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Chlorophyll Degradation", + "title": "", + "url": "https://www.google.com/search?q=Chlorophyll+Degradation&sei=EzGGadziB-2i0PEP7eiguAU&ved=2ahUKEwiOiNrSvcWSAxVbePUHHYrbKukQgK4QegYIAQgAEAo" + }, + { + "misc": {}, + "text": "Ethylene Gas", + "title": "", + "url": "https://www.google.com/search?q=Ethylene+Gas&sei=EzGGadziB-2i0PEP7eiguAU&ved=2ahUKEwiOiNrSvcWSAxVbePUHHYrbKukQgK4QegYIAQgAEAw" + }, + { + "misc": {}, + "text": "Visual Recognition", + "title": "", + "url": "https://www.google.com/search?q=Visual+Recognition&sei=EzGGadziB-2i0PEP7eiguAU&ved=2ahUKEwiOiNrSvcWSAxVbePUHHYrbKukQgK4QegYIAQgCEAE" + }, + { + "misc": {}, + "text": "Ripening Process", + "title": "", + "url": "https://www.google.com/search?q=Ripening+Process&sei=EzGGadziB-2i0PEP7eiguAU&ved=2ahUKEwiOiNrSvcWSAxVbePUHHYrbKukQgK4QegYIAQgCEAM" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.youtube.com/watch?v=0WCErY3OYng" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.scienceabc.com/nature/bananas-change-colour-upon-ripening.html" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://pekoproduce.com/blogs/produce-nutrition/green-to-yellow-to-spotty-how-do-bananas-ripen" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.quora.com/Why-are-bananas-yellow" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.ck12.org/flexi/physical-science/Light-in-Physics/what-makes-bananas-yellow/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://pace.oceansciences.org/color_determination.cgi#:~:text=A%20banana%20appears%20yellow%20to%20the%20human,our%20brain%20recognizes%20as%20a%20%22YELLOW%20BANANA!%22%22" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://pace.oceansciences.org/color_determination.cgi" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Why Do Bananas Change To Yellow When Ripening?", + "type": "images", + "url": "https://www.scienceabc.com/nature/bananas-change-colour-upon-ripening.html" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "How to Ripen and Store Bananas", + "type": "images", + "url": "https://www.seriouseats.com/how-to-store-and-ripen-bananas-8651168" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Yellow Bananas Information and Facts", + "type": "images", + "url": "https://specialtyproduce.com/produce/Yellow_Bananas_919.php" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 3, + "sub_type": "medium", + "text": null, + "title": "Why Do Bananas and Apple Turn Brown", + "type": "images", + "url": "https://www.youtube.com/watch?v=3aoSkvYxDuM" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 4, + "sub_type": "medium", + "text": null, + "title": "Why Do Bananas Turn Brown? | Britannica", + "type": "images", + "url": "https://www.britannica.com/story/why-do-bananas-turn-brown" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 5, + "sub_type": "medium", + "text": null, + "title": "Why are organic bananas not turning yellow?", + "type": "images", + "url": "https://www.facebook.com/groups/517390799182443/posts/1330387357882779/" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 6, + "sub_type": "medium", + "text": null, + "title": "Green to Yellow to Spotty: How Do Bananas Ripen? – Peko", + "type": "images", + "url": "https://pekoproduce.com/blogs/produce-nutrition/green-to-yellow-to-spotty-how-do-bananas-ripen?srsltid=AfmBOoquPl01O8i9kkGqkbJZgoxyNajIxzi3Jk_iELf4EqGCK1Az3utE" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 7, + "sub_type": "medium", + "text": null, + "title": "Yellow, Green or Spotted: Knowing When Bananas Are Ripe | by ...", + "type": "images", + "url": "https://medium.com/@willloiseau/yellow-green-or-spotted-knowing-when-bananas-are-ripe-baeef8614650" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 8, + "sub_type": "medium", + "text": null, + "title": "Your Complete Ripe Banana Guide - The FruitGuys", + "type": "images", + "url": "https://fruitguys.com/blog/going-bananas/" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": [ + "Why are bananas yellow? : r/answers - Reddit Reddit https://www.reddit.com › answers › comments › why_a... Reddit https://www.reddit.com › answers › comments › why_a...", + "Cavendish banana - Wikipedia Wikipedia https://en.wikipedia.org › wiki › Cavendish_banana Wikipedia https://en.wikipedia.org › wiki › Cavendish_banana", + "Going Bananas: Your Complete Ripe Banana Guide - The FruitGuys The FruitGuys https://fruitguys.com › blog › going-bananas The FruitGuys https://fruitguys.com › blog › going-bananas", + "What color is the healthiest banana?" + ], + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Why are bananas yellow? : r/answers - Reddit Reddit https://www.reddit.com › answers › comments › why_a... Reddit https://www.reddit.com › answers › comments › why_a...<|>Cavendish banana - Wikipedia Wikipedia https://en.wikipedia.org › wiki › Cavendish_banana Wikipedia https://en.wikipedia.org › wiki › Cavendish_banana<|>Going Bananas: Your Complete Ripe Banana Guide - The FruitGuys The FruitGuys https://fruitguys.com › blog › going-bananas The FruitGuys https://fruitguys.com › blog › going-bananas<|>What color is the healthiest banana?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "10+ comments · 1 year ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "The yellow pigment is called Xantophyll. When bananas ripen, the chlorophyll (green) breaks down and the yellow color is more visible. Read more", + "title": "Why are bananas yellow? : r/answers", + "type": "general", + "url": "https://www.reddit.com/r/answers/comments/1b6hkf3/why_are_bananas_yellow/" + }, + { + "cite": "https://pekoproduce.com › blogs › produce-nutrition", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "Jul 22, 2022 — TURNING YELLOW : As chlorophyll (green pigment) breaks down and carotenoids ( yellow pigment) are created, bananas will slowly transition from ... Read more", + "title": "Green to Yellow to Spotty: How Do Bananas Ripen?", + "type": "general", + "url": "https://pekoproduce.com/blogs/produce-nutrition/green-to-yellow-to-spotty-how-do-bananas-ripen?srsltid=AfmBOopjbk0QpbXhVTOOEFM36-3MKk5_qIE6Hejh0b7NfGAhFPWE880I" + }, + { + "cite": "https://www.scienceabc.com › Nature", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "Sep 22, 2015 — Ethylene is a crucial ripening hormone that makes bananas change color , as it aids the fruit in its ripening. The chlorophyll in the peel breaks ... Read more", + "title": "Why Do Bananas Change To Yellow When Ripening?", + "type": "general", + "url": "https://www.scienceabc.com/nature/bananas-change-colour-upon-ripening.html" + }, + { + "cite": "10+ answers · 5 years ago", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "The yellow color in a banana indicates that the peel is beginning to deteriorate and releasing sugars into the rest of the fruit. This is when ... Read more", + "title": "Why are supermarket bananas so yellow?", + "type": "general", + "url": "https://www.quora.com/Why-are-supermarket-bananas-so-yellow" + }, + { + "cite": "https://www.ck12.org › ... › Physical Science › Light", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": "Bananas are yellow due to the presence of a pigment called xanthophylls . When bananas are green, they contain chlorophyll, which gives them their green color. Read more", + "title": "Flexi answers - What makes bananas yellow?", + "type": "general", + "url": "https://www.ck12.org/flexi/physical-science/Light-in-Physics/what-makes-bananas-yellow/" + }, + { + "cite": "10+ comments · 3 months ago", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": null, + "text": "Bananas are yellow because of the xanthophylls pigment , which is revealed as the chlorophyll breaks down during ripening.", + "title": "What compound makes bananas yellow?", + "type": "general", + "url": "https://www.facebook.com/groups/4382588805101476/posts/25961064853493893/" + }, + { + "cite": "300+ views · 2 years ago", + "cmpt_rank": 9, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "video", + "text": "Yellow banana while it might look concerning thatcolor change is actually a sign the fruit is ripeningnot rotting.", + "title": "Why Do Bananas Change Color? #science #sciencefun #facts ...", + "type": "general", + "url": "https://www.youtube.com/shorts/6gaf-BseVGw" + }, + { + "cite": "https://www.lsg-group.com › Stories", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "Low temperatures lead to stress in bananas , which also causes them to react with an increased production of ethylene, mature faster and turn brown. The “Global ... Read more", + "title": "Food Science: Why Do Bananas Turn Brown?", + "type": "general", + "url": "https://www.lsg-group.com/news/global-food-myths-5-do-bananas-turn-brown-because-of-its-iron-content/" + }, + { + "cite": "https://pmc.ncbi.nlm.nih.gov › articles › PMC10118268", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": null, + "text": "by P Liu · 2023 · Cited by 2 — The color of banana peels is associated with chlorophyll content, which degrades during the ripening process (Drury et al. 1999). The authors ... Read more", + "title": "Uncovering the mechanism of banana “green ripening”", + "type": "general", + "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC10118268/" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[7d76d3a83ebc].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[7d76d3a83ebc].json new file mode 100644 index 0000000..3374c04 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[7d76d3a83ebc].json @@ -0,0 +1,604 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 740000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "Choose what you’re giving feedback on", + "img_url": null, + "text": "Choose what you’re giving feedback on|Donald Trump|45th and 47th U.S. President|45th and 47th U.S. President|About this result|Share|Share|Click to copy link|Click to copy link|Share link|Share link|Link copied|Link copied|Claim this knowledge panel|Send feedback|Overview|Overview|Overview|Books|Books|Books", + "urls": [ + { + "misc": {}, + "text": "Claim this knowledge panel", + "title": "", + "url": "https://posts.google.com/claim/?mid=/m/0cqt90" + }, + { + "misc": {}, + "text": "Overview", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump&si=AL3DRZHCeZEsX3-CsvvLZIg5Ht46TyzeLCv6JU-VYAbQYYXTU9ujPIprOzM-hzQKNCSBdKLfTJqVB7b0OPGOc1lursgMTYzpdXF96kKgT-ydlhSX0hWf7Lo%3D&sa=X&ved=2ahUKEwjLqraipMSSAxVrSjABHWpuH8oQyNoBKAB6BAgTEAA&ictx=1" + }, + { + "misc": {}, + "text": "Books", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump+books&si=AL3DRZEg35eg0o6rvkas7wDZ2ZuaCp6q5KVFnkc8XBmxZKVnhhGLbNELffVz9J_d5LQPDN5AZb7LMQc3Bqpx6hKQFShg4uFY2Gi-LePpCp9zB8wo44N8pT8syhV2q64o-D2LfqKlsXnU&sa=X&ved=2ahUKEwjLqraipMSSAxVrSjABHWpuH8oQyNoBKAB6BAgXEAA&ictx=1" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": " Donald Trump 45th and 47th U.S. President About this result Share Share Facebook WhatsApp X Email Click to copy link Share link Link copied Claim this knowledge panel Send feedback Overview Books", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump&udm=2&source=univ&sa=X&ved=2ahUKEwjLqraipMSSAxVrSjABHWpuH8oQnN8JegQIFhAD" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://x.com/realDonaldTrump" + }, + { + "misc": {}, + "text": "Age|79 years|Jun 14, 1946", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=donald+trump+age&stick=H4sIAAAAAAAAAOPgE-LUz9U3SC4ssTTQEs1OttIvSM0vyEkFUkXF-XlWiempi1gFUvLzEnNSFEqKSnMLFIBCAMqYnZc3AAAA&sa=X&ved=2ahUKEwjLqraipMSSAxVrSjABHWpuH8oQ18AJegQILhAB" + }, + { + "misc": {}, + "text": "Party|Republican Party", + "title": "", + "url": "/search?sca_esv=bd16726c6a722966&q=Republican+Party&si=AL3DRZHmwLjWhgnaPB3UTu10R6S5qNLXiQiKMeezfKyB1FMsRrUdcsLX7-nPSpyKZYIqfOJDBGOdz2E57-99eIANe1Pl8wwO7G3XqVt3qiEO4uERO4KYYXvGXEXRT6jOHlGcixAxcLt6zGSb8XfSKPYd5JeDPjiWx1lTb4E4mlK9t2l0ky4-1M_qoJaYHMURBGCmYARFqYnA&sa=X&ved=2ahUKEwjLqraipMSSAxVrSjABHWpuH8oQ18AJegQILxAB" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/DonaldTrump/posts/1470862751066207/" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_results", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "Trump promises Schumer funding for NY tunnel project — if Penn Station and Dulles Airport are renamed after him", + "type": "top_stories", + "url": "https://www.cnn.com/2026/02/05/politics/schumer-trump-ny-funding-rename" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "Trump wanted Dulles Airport, Penn Station named after him — in exchange for releasing federal funds", + "type": "top_stories", + "url": "https://www.nbcnews.com/politics/donald-trump/trump-asked-dulles-penn-station-named-exchange-gateway-money-released-rcna257708" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "Democrats erupt in fury over Trump demanding Schumer help rename Dulles, Penn Station", + "type": "top_stories", + "url": "https://www.axios.com/2026/02/06/trump-schumer-dulles-penn-gateway-democrats-nyc" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 3, + "sub_type": null, + "text": null, + "title": "‘Grind the country to a halt’: Democrat urges national strike if Trump meddles in midterms", + "type": "top_stories", + "url": "https://www.theguardian.com/us-news/2026/feb/06/donald-trump-voting-midterms-democrat-national-strike" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 4, + "sub_type": null, + "text": null, + "title": "Furious Jimmy Kimmel Loses It at Donald Trump: ‘How Dare You?’", + "type": "top_stories", + "url": "https://www.thedailybeast.com/obsessed/furious-jimmy-kimmel-loses-it-at-donald-trump-how-dare-you/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "No confusion.\nNo overpaying. \nJust the lowest prices. 💊🇺🇸\n\nTrumpRx.Gov", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUZgw0UES1g/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "IT'S HERE: Finally, a direct to consumer website designed to find the lowest drug prices for YOU.\n\nTrumpRx.gov", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUZU0YLEntC/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "TONIGHT AT 7 PM EST.", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUZGePGCZIo/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 3, + "sub_type": null, + "text": null, + "title": "We are going to rededicate America as ONE NATION UNDER GOD. 🇺🇸", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUZDSEIDkca/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 4, + "sub_type": null, + "text": null, + "title": "THIS IS OUR WHY\n\nIllegal alien Beishekeev swerved into oncoming traffic, tragically killing four Americans. He entered the U.S. illegally via the Biden-era CBP One app & obtained a commercial driver's license in Pennsylvania.\n\nThis PREVENTABLE TRAGEDY highlights the need to continue POTUS' mass deportation efforts & END sanctuary city policies. \n\nWe are praying for the families of those lost.", + "type": "recent_posts", + "url": "https://www.instagram.com/p/DUY2kxqj6R8/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 5, + "sub_type": null, + "text": null, + "title": "\"To be a great nation, you have to have RELIGION. You have to have FAITH. You have to have GOD.\" - President Donald J. Trump at the 74th National Prayer Breakfast.", + "type": "recent_posts", + "url": "https://www.facebook.com/WhiteHouse/posts/to-be-a-great-nation-you-have-to-have-religion-you-have-to-have-faith-you-have-t/122165272874723345/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 6, + "sub_type": null, + "text": null, + "title": "We are endowed with our sacred rights to LIFE, LIBERTY, and not by government, but by GOD Almighty Himself. 🙏🕊️", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUYWyg9As7L/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 7, + "sub_type": null, + "text": null, + "title": "President Trump Attends the National Prayer Breakfast", + "type": "recent_posts", + "url": "https://www.facebook.com/POTUS/posts/president-trump-attends-the-national-prayer-breakfast/1271317968218851/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 8, + "sub_type": null, + "text": null, + "title": "WARNING: AMERICA FIRST AGENDA DETECTED\n\nPresident Trump wants to…\n🇺🇸 Unleash American energy\n🇺🇸 Arrest & deport criminal illegals\n🇺🇸 Slash drug prices\n🇺🇸 Enforce the law\n🇺🇸 Put America first\n\nIf this continues… we may experience TOO MUCH WINNING. \n\nHOW DARE HE.", + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUW-J_Fimz9/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 9, + "sub_type": null, + "text": null, + "title": null, + "type": "recent_posts", + "url": "https://www.instagram.com/reel/DUWrLYEjNWf/" + }, + { + "cite": "https://en.wikipedia.org › wiki › Donald_Trump", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "2 hours ago — Donald John Trump (born June 14, 1946) is an American politician, media personality, and businessman who is the 47th president of the United States. Read more", + "title": "Donald Trump", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Donald_Trump" + }, + { + "cite": "https://www.whitehouse.gov › Administration", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "President Donald J. Trump is returning to the White House to build upon his previous successes and use his mandate to reject the extremist policies. Read more", + "title": "President Donald J. Trump", + "type": "general", + "url": "https://www.whitehouse.gov/administration/donald-j-trump/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Trump’s All Over the Epstein Files, Attacks Jimmy Kimmel After the Grammys & Don Lemon Gets Arrested", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=oC8qsi9Uu7c" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "President Trump declined to address the complains of Epstein survivors when pressed by CNN's @kaitlancollins over their concerns about the extent of redactions in the files released by the Justice Department.\n\nRead more at the link in @cnn's bio.", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DUUZrEOCeON/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "At the National Prayer Breakfast on Thursday, President Donald Trump said he doesn't know “how a person of faith can vote for a Democrat.” “I know we have some [Democrats] here today. I don’t know why they’re here. Because they certainly don’t give us thei", + "type": "perspectives", + "url": "https://www.facebook.com/ABC10News/posts/at-the-national-prayer-breakfast-on-thursday-president-donald-trump-said-he-does/1316893117136276/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "WATCH: Trump says he will revoke church tax exempt status if leaders 'say something bad about' him", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=7hmzdUBt0M4" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "I’m still thinking about the exchange between Donald Trump and White House correspondent Kaitlan Collins earlier this week. Trump berated her for not smiling and criticized her performance (and that of her employer, CNN). That was all horrible and unaccep", + "type": "perspectives", + "url": "https://www.linkedin.com/posts/bethccollier_im-still-thinking-about-the-exchange-between-activity-7425289621921566720-X8Ns" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "During a press briefing this week, President Donald Trump told Kaitlan Collins, CNN's chief White House correspondent, that she should smile more when he didn't like her questions. He then proceeded to throw a fit of sorts, berating her for simply doing her job, calling her \"the worst reporter.\"\n\nTelling a woman to smile is offensive at any time—we all know that—but it feels especially abusive in this context. See the exchange and read the full story at the link in bio.", + "type": "perspectives", + "url": "https://www.instagram.com/p/DUWbKNGEiN5/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Yesterday, CNN reporter Kaitlan Collins asked President Donald Trump a series of questions about the newly-released Epstein Files. Trump became visibly frustrated and switched the topic to Collins’ smile and reporting chops.\n\n(🎥: CNN / 🧵: @aaronparnas, @revkarla, @attorneyryan, @jasonegenberg, @matt_gromlich_for_congress, @kosherhotdogz, @kelseycombe, @gingerdiest, @portmoody2019)", + "type": "perspectives", + "url": "https://www.instagram.com/p/DUWFdwyjtXJ/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "(Reuters) - U.S. President Donald Trump on Thursday said he retained the right to \"militarily secure\" a joint US-UK airbase in the Chagos Archipelago following \"productive\" discussions with British Prime Minister Keir Starmer. \n\n\"I understand that the deal Prime Minister Starmer has made, according to many, the best he could make,\" Trump said in a post on Truth Social, referencing a 2025 deal Starmer made to cede sovereignty of the Chagos Archipelago which includes the island with the joint US-UK air base.", + "type": "perspectives", + "url": "https://x.com/phildstewart/status/2019478590530715784?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Trump launched his new prescription drug website TrumpRx today. Now you’ll be able to buy Xanax from the reason you need Xanax. #FallonTonight", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DUZtm-6joXd/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "FOX 35 Orlando. . President Donald Trump said TrumpRX makes healthcare much more affordable -- \"at a level you can't even believe.\" Here's what prices are on TrumpRX.gov.", + "type": "perspectives", + "url": "https://www.facebook.com/FOX35Orlando/videos/president-donald-trump-said-trumprx-makes-healthcare-much-more-affordable-at-a-l/902078955966377/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "What Trump says he learned from Minneapolis chaos", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=Og51cLThBYQ" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "US President Donald Trump declared religion is “coming back” in a speech at the National Prayer Breakfast.", + "type": "perspectives", + "url": "https://www.facebook.com/SkyNewsAustralia/posts/us-president-donald-trump-declared-religion-is-coming-back-in-a-speech-at-the-na/1342673757890219/" + }, + { + "cite": "109.9M+ followers", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 0, + "sub_type": null, + "text": "Donald J. Trump (@realDonaldTrump) - Posts - 45th & 47th President of the United States of America | X (formerly Twitter)", + "title": "Donald J. Trump (@realDonaldTrump) / Posts / X", + "type": "general", + "url": "https://x.com/realDonaldTrump" + }, + { + "cite": "https://trumpwhitehouse.archives.gov › people › donald...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 0, + "sub_type": null, + "text": "Donald J. Trump is the 45th President of the United States . He believes the United States has incredible potential and will go on to exceed even its remarkable ... Read more", + "title": "Donald J. Trump – The White House", + "type": "general", + "url": "https://trumpwhitehouse.archives.gov/people/donald-j-trump/" + }, + { + "cite": "40.6M+ followers", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": null, + "text": "Donald J. Trump . 40593418 likes · 1042198 talking about this. This is the official Facebook page for Donald J. Trump.", + "title": "Donald J. Trump", + "type": "general", + "url": "https://www.facebook.com/DonaldTrump/" + }, + { + "cite": "https://www.nbcnews.com › politics › donald-trump › t...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 0, + "sub_type": null, + "text": "5 hours ago — Trump wanted Dulles Airport, Penn Station named after him — in exchange for releasing federal funds. Trump has named several landmarks and ... Read more", + "title": "Trump wanted Dulles Airport, Penn Station named after him", + "type": "general", + "url": "https://www.nbcnews.com/politics/donald-trump/trump-asked-dulles-penn-station-named-exchange-gateway-money-released-rcna257708" + }, + { + "cite": "https://www.whitehousehistory.org › bios › donald-j-trump", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 0, + "sub_type": null, + "text": "Donald John Trump was born in Queens, New York, on June 14, 1946 . Trump was educated at the New York Military Academy and the Wharton School of Finance and ...", + "title": "Donald J. Trump", + "type": "general", + "url": "https://www.whitehousehistory.org/bios/donald-j-trump" + }, + { + "cite": "43.1M+ followers", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 0, + "sub_type": null, + "text": "43M followers · 47 following · 7853 posts · @real donald trump: “45th & 47th President of the United States”", + "title": "President Donald J. Trump (@realdonaldtrump)", + "type": "general", + "url": "https://www.instagram.com/realdonaldtrump/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 37, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json new file mode 100644 index 0000000..6348ea0 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json @@ -0,0 +1,425 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2560000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=uyFvaeSENcCKwbkPqPGKsAk&ved=2ahUKEwjSkYDkv5mSAxWXSDABHUvgByYQgK4QegYIAAgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=The%20sky%20appears%20blue%20because%20of%20the,we%20can%20see%20look%20blue%20or%20violet**" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "How to explain to a kid why the sky is blue?", + "Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...", + "Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>How to explain to a kid why the sky is blue?<|>Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...<|>Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", + "title": "Why Is the Sky Blue? | NESDIS", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the Sky Looks Blue\n\n#atmosphere #sky #bluesky #whyskyisblue #space", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DTsZrgoD0xa/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The #sky achieves its color from sunlight scattering off Earth's atmosphere , a process called #Rayleigh scattering , where tiny air molecules would scatter shorter blue #wavelengths more than longer red ones, making the sky blue during the day . \n\nWe do know so .", + "type": "perspectives", + "url": "https://x.com/PARODY_CAIT/status/2005596656985678030?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", + "type": "perspectives", + "url": "https://www.instagram.com/p/DR2aqixgJUa/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Look at the sky. It appears blue during the daytime and reddish during sunrise and sunset. Why? Let me introduce you today to a British man who explained these phenomena very clearly. Further, his work for classical black-body radiation, later played an imp", + "type": "perspectives", + "url": "https://www.facebook.com/100064519939797/posts/look-at-the-sky-it-appears-blue-during-the-daytime-and-reddish-during-sunrise-an/1257662263061093/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The sky looks blue because of Rayleigh scattering. Sunlight is actually white, not yellow. Clouds weigh millions of kilograms. The sky on Mars looks reddish-pink. The highest clouds are called noctilucent clouds. Rainbows are full circles, not arcs. You can nev", + "type": "perspectives", + "url": "https://www.facebook.com/teacherceppee/posts/the-sky-looks-blue-because-of-rayleigh-scatteringsunlight-is-actually-white-not-/1429848428511661/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the sky is blue", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": "859.4K+ views · 5 years ago", + "cmpt_rank": 8, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": "https://www.weather.gov › fgz › SkyBlue", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": null, + "text": "The sky looks blue, not violet, because our eyes are more sensitive to blue light (and the sun also emits more energy as blue light than as violet). Read more", + "title": "Why Is The Sky Blue?", + "type": "general", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 10, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": "https://www.uu.edu › dept › physics › scienceguys", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", + "title": "Why is the sky blue on Earth, but black in space or ...", + "type": "general", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[aa594f199c3d].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[aa594f199c3d].json new file mode 100644 index 0000000..a007936 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[aa594f199c3d].json @@ -0,0 +1,574 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2980000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=Yg6FaazIEMy7mtkP5KyvyQQ&ved=2ahUKEwjn0_i1qMOSAxVVlGoFHXFIF7YQgK4QegYIAQgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.instagram.com/reel/DIZJjPvOtIN/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.wsav.com/weather-news/why-the-sky-looks-bluer-in-fall-and-winter/#:~:text=In%20summer%2C%20the%20sun%20sits%20higher%20in,intensifying%20the%20blue%20appearance%20of%20the%20sky." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.wsav.com/weather-news/why-the-sky-looks-bluer-in-fall-and-winter/" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "What is the true color of our sky?", + "How to explain to a kid why the sky is blue?", + "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": "vertical", + "text": null, + "title": "Why Is the Sky Blue? YouTube · NASA Space Place Mar 2, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 1, + "sub_type": "vertical", + "text": null, + "title": "Space Place in a Snap: Why Is the Sky Blue? NASA+ · NASA+ Streaming Service Oct 31, 2023", + "type": "videos", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 2, + "sub_type": "vertical", + "text": null, + "title": "Why is the sky blue? It's not. You might know that as sunlight ... Instagram · Cleo Abram Apr 13, 2025", + "type": "videos", + "url": "https://www.instagram.com/reel/DIZJjPvOtIN/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue?", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DT7ufKNj70z/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", + "type": "perspectives", + "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why the sky is blue", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "On a clear day, what is the colour of the sky?", + "type": "perspectives", + "url": "https://www.facebook.com/apna99036/posts/on-a-clear-day-what-is-the-colour-of-the-sky/1504710364995479/" + }, + { + "cite": null, + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 7, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue?", + "type": "images", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? | Live Science", + "type": "images", + "url": "https://www.livescience.com/planet-earth/why-is-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NASA Space Place – NASA Science for Kids", + "type": "images", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 3, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NESDIS | National Environmental ...", + "type": "images", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 4, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue?", + "type": "images", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 5, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? It's an age old question that actually ...", + "type": "images", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 6, + "sub_type": "medium", + "text": null, + "title": "Why Is the Sky Blue? | NASA Space Place – NASA Science for Kids", + "type": "images", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 7, + "sub_type": "medium", + "text": null, + "title": "The Physics of Light - Why is the Sky Blue, Really?", + "type": "images", + "url": "https://www.youtube.com/watch?v=zq-rDYvxAZ4" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 8, + "sub_type": "medium", + "text": null, + "title": "Why is the sky blue? It's an age old question that actually ...", + "type": "images", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue? - NESDIS - NOAA", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "https://www.britannica.com › ... › Matter & Energy", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json new file mode 100644 index 0000000..9a5b13b --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json @@ -0,0 +1,467 @@ +{ + "features": { + "infinity_scroll": false, + "language": "en", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "result_estimate_count": 2980000000.0, + "result_estimate_time": null + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": null, + "img_url": null, + "urls": [ + { + "misc": {}, + "text": "Rayleigh scattering", + "title": "", + "url": "https://www.google.com/search?q=Rayleigh+scattering&sei=f1yFadntLOaOwbkP6tLk-QY&ved=2ahUKEwjCo7718sOSAxXZi7AFHSGSC98QgK4QegYIAQgAEAY" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.weather.gov/fgz/SkyBlue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html#:~:text=We%20have%20three%20types%20of%20colour%20receptors%2C,visual%20system%20constructs%20the%20colours%20we%20see." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/" + }, + { + "misc": {}, + "text": "", + "title": "", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "misc": {}, + "text": "Learn more", + "title": "", + "url": "https://support.google.com/websearch?p=ai_overviews&hl=en" + }, + { + "misc": {}, + "text": "Privacy Policy", + "title": "", + "url": "https://policies.google.com/privacy?hl=en" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "ai_overview", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": [ + "Why is the sky blue short answer?", + "What is the true color of our sky?", + "How to explain to a kid why the sky is blue?", + "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" + ], + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://spaceplace.nasa.gov › blue-sky", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://spaceplace.nasa.gov/blue-sky/" + }, + { + "cite": "160+ comments · 2 years ago", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", + "title": "Why is the sky blue? Do I understand it correctly", + "type": "general", + "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" + }, + { + "cite": "https://kids.nationalgeographic.com › books › article", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://kids.nationalgeographic.com/books/article/sky" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": "what_people_are_saying", + "text": null, + "title": "We see the sky as blue, but in reality it is completely black. The blue color is simply an optical illusion caused by the scattering of sunlight in our atmosphere.", + "type": "perspectives", + "url": "https://www.facebook.com/groups/ScienceKiDuniya/posts/3440058569495988/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 1, + "sub_type": "what_people_are_saying", + "text": null, + "title": "The REAL Reason the Sky Looks Blue", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 2, + "sub_type": "what_people_are_saying", + "text": null, + "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", + "type": "perspectives", + "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 3, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue?", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 4, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Sky Blue? Credit to Discover.", + "type": "perspectives", + "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 5, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DT7ufKNj70z/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 6, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", + "type": "perspectives", + "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 7, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", + "type": "perspectives", + "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 8, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", + "type": "perspectives", + "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 9, + "sub_type": "what_people_are_saying", + "text": null, + "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", + "type": "perspectives", + "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 10, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", + "type": "perspectives", + "url": "https://www.youtube.com/shorts/o1pTCw8cg10" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 11, + "sub_type": "what_people_are_saying", + "text": null, + "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", + "type": "perspectives", + "url": "https://www.instagram.com/p/DR2aqixgJUa/" + }, + { + "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", + "title": "Why Is the Sky Blue? - NESDIS - NOAA", + "type": "general", + "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" + }, + { + "cite": "https://math.ucr.edu › home › baez › physics › General", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", + "title": "Why is the sky blue?", + "type": "general", + "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" + }, + { + "cite": "864K+ views · 5 years ago", + "cmpt_rank": 8, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" + }, + { + "cite": "50+ reactions · 1 year ago", + "cmpt_rank": 9, + "details": { + "duration": null, + "source": null + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "video", + "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", + "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", + "type": "general", + "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" + }, + { + "cite": "https://www.britannica.com › ... › Matter & Energy", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", + "title": "Why Is the Sky Blue?", + "type": "general", + "url": "https://www.britannica.com/story/why-is-the-sky-blue" + }, + { + "cite": "https://www.uu.edu › dept › physics › scienceguys", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", + "title": "Why is the sky blue on Earth, but black in space or ...", + "type": "general", + "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": [], + "error": null, + "section": "footer", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/fixtures/serps-v0.6.7.json.bz2 b/tests/fixtures/serps-v0.6.7.json.bz2 new file mode 100644 index 0000000..1a9dd95 Binary files /dev/null and b/tests/fixtures/serps-v0.6.7.json.bz2 differ diff --git a/tests/test_parse_serp.py b/tests/test_parse_serp.py index 2895cee..0b8af61 100644 --- a/tests/test_parse_serp.py +++ b/tests/test_parse_serp.py @@ -1,21 +1,148 @@ -import glob +"""Test SERP parsing pipeline end-to-end""" + +import bz2 +from pathlib import Path + +import orjson + import pytest import WebSearcher as ws - from syrupy.extensions.json import JSONSnapshotExtension + +# --------------------------------------------------------------------------- +# Data loading +# --------------------------------------------------------------------------- + +FIXTURES_DIR = Path(__file__).parent / "fixtures" +SERPS_PATH = FIXTURES_DIR / "serps-v0.6.7.json.bz2" + + +def load_serps(path: Path) -> list[dict]: + """Load SERP records from a bz2-compressed JSON-lines file""" + with bz2.open(path, "rt") as f: + return [orjson.loads(line) for line in f] + + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + @pytest.fixture def snapshot_json(snapshot): - """Store or retrieve json for init or testing""" return snapshot.use_extension(JSONSnapshotExtension) + def pytest_generate_tests(metafunc): - """Create file_name list that test_parsing inputs""" - file_list = glob.glob("./data/demo-ws-v0.3.10/html/*") - metafunc.parametrize("file_name", file_list) - -def test_parsing(snapshot_json, file_name): - """Parse each file_name and compare to existing snapshot""" - soup = ws.load_soup(file_name) - results = ws.parse_serp(soup) - assert results == snapshot_json + """Parametrize tests by serp_id from demo data""" + if "serp_record" not in metafunc.fixturenames: + return + if not SERPS_PATH.exists(): + metafunc.parametrize("serp_record", []) + return + records = load_serps(SERPS_PATH) + ids = [r["serp_id"][:12] for r in records] + metafunc.parametrize("serp_record", records, ids=ids) + + +# --------------------------------------------------------------------------- +# Snapshot tests +# --------------------------------------------------------------------------- + +@pytest.mark.skipif(not SERPS_PATH.exists(), reason="Demo data not available") +def test_parse_serp(snapshot_json, serp_record): + """Parse SERP and compare to snapshot""" + parsed = ws.parse_serp(serp_record["html"], extract_features=True) + assert parsed == snapshot_json + + +# --------------------------------------------------------------------------- +# Structural tests +# --------------------------------------------------------------------------- + +EXPECTED_KEYS = { + "section", "cmpt_rank", "sub_rank", "type", "sub_type", + "title", "url", "text", "cite", "details", "error", "serp_rank", +} + + +@pytest.fixture(scope="module") +def all_parsed_serps(): + """Parse all SERPs and return list of parsed outputs""" + if not SERPS_PATH.exists(): + pytest.skip("Demo data not available") + return [ws.parse_serp(record["html"], extract_features=True) + for record in load_serps(SERPS_PATH)] + + +@pytest.fixture(scope="module") +def all_results(all_parsed_serps): + """Flat list of all results across SERPs""" + results = [] + for serp in all_parsed_serps: + results.extend(serp["results"]) + return results + + +def test_results_have_expected_keys(all_results): + """Every result dict has exactly the expected keys""" + for r in all_results: + assert set(r.keys()) == EXPECTED_KEYS, f"cmpt {r.get('cmpt_rank')}: {set(r.keys()) ^ EXPECTED_KEYS}" + + +def test_no_unclassified_results(all_results): + """No result should have type 'unclassified' (the BaseResult default)""" + unclassified = [r for r in all_results if r["type"] == "unclassified"] + assert len(unclassified) == 0 + + +def test_no_unknown_types(all_results): + """No unknown types after classifier fixes""" + unknowns = [r for r in all_results if r["type"] == "unknown"] + assert len(unknowns) == 0, f"Found {len(unknowns)} unknown results" + + +def test_no_parse_errors(all_results): + """No parsing errors in results""" + errors = [r for r in all_results if r["error"] is not None] + assert len(errors) == 0, f"Found {len(errors)} errors: {[r['error'] for r in errors]}" + + +def test_general_results_have_title_or_url(all_results): + """General results without errors should have at least title or url""" + for r in all_results: + if r["type"] == "general" and r["error"] is None: + assert r["title"] is not None or r["url"] is not None, ( + f"cmpt {r['cmpt_rank']} sub {r['sub_rank']}: general result with no title or url" + ) + + +def test_perspectives_have_url(all_results): + """Perspectives results should have a url""" + for r in all_results: + if r["type"] == "perspectives": + assert r["url"] is not None, f"perspectives sub {r['sub_rank']}: no url" + + +def test_serp_rank_is_sequential(all_parsed_serps): + """serp_rank values should be sequential from 0 within each SERP""" + for serp in all_parsed_serps: + ranks = [r["serp_rank"] for r in serp["results"]] + assert ranks == list(range(len(ranks))) + + +def test_field_types(all_results): + """Validate field types for all results""" + valid_sections = {"main", "header", "footer", "rhs"} + for r in all_results: + assert isinstance(r["section"], str) and r["section"] in valid_sections + assert isinstance(r["cmpt_rank"], int) and r["cmpt_rank"] >= 0 + assert isinstance(r["serp_rank"], int) and r["serp_rank"] >= 0 + assert isinstance(r["sub_rank"], int) and r["sub_rank"] >= 0 + assert isinstance(r["type"], str) + assert r["sub_type"] is None or isinstance(r["sub_type"], str) + assert r["title"] is None or isinstance(r["title"], str) + assert r["url"] is None or isinstance(r["url"], str) + assert r["text"] is None or isinstance(r["text"], str) + assert r["cite"] is None or isinstance(r["cite"], str) + assert r["error"] is None or isinstance(r["error"], str)