Migrate satcom forecast to NWS API #5
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces a new Weather.gov API client, data processing, formatting, caching, and error handling modules. It replaces the previous HTML scraping method with a more robust and reliable API-based approach. The changes include: - **New API Modules**: `api_client`, `api_config`, `api_models`, `api_data_processor`, `api_formatter`, `api_cache`, `api_error_handler`. - **Enhanced Fetcher**: `forecast_fetcher_api.py` provides a hybrid mode supporting both API and HTML fallback. - **Configuration**: Environment variables and feature flags for flexible control. - **Caching**: Implemented for improved performance and reduced API calls. - **Error Handling**: Comprehensive framework with retry logic and circuit breaker. - **Testing**: New unit, integration, and compatibility tests added. - **Documentation**: Updated READMEs and examples. The `fetch_forecast` function is updated to use the new API-based implementation automatically, ensuring backward compatibility. Co-authored-by: clayton <clayton@clayauld.com>
|
Cursor Agent can help with this pull request. Just |
Co-authored-by: clayton <clayton@clayauld.com>
This commit enhances the testing framework by replacing the previous test execution method with `pytest` for improved test management and reporting. It also updates the API integration to include new features such as: - Transition from manual test scripts to `pytest` for running tests. - Improved error handling in the API client, including better response validation. - Added new test cases for API functionality, ensuring comprehensive coverage of edge cases. - Introduced a new logo asset for branding purposes. Additionally, the `pytest.ini` configuration has been updated to reflect the new testing structure and added markers for better test categorization. Co-authored-by: clayton <clayton@clayauld.com>
📊 Pull Request Analysis Report🧪 Test Results
📋 Coverage Report
📁 File Coverage Breakdown
🎯 Recommendations📊 Analysis generated by GitHub Actions • View detailed test report |
This comment was marked as resolved.
This comment was marked as resolved.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…ub.com:clayauld/satcom-forecast into cursor/migrate-satcom-forecast-to-nws-api-dee5
…text and add tests for multiple percentage scenarios.
…lists of strings for improved consistency.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…"Today" and refactor period filtering to weather utilities.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…r handling, and refine text splitting logic.
…ub.com:clayauld/satcom-forecast into cursor/migrate-satcom-forecast-to-nws-api-dee5
…tor: update asyncio loop time retrieval and GitHub workflow comment construction.
…dpoint` return value.
… permissions to the test workflow.
…oss components and tests.
…t.toml for mypy settings; refactor type hints and method signatures across various components for consistency and clarity.
…factor logging messages for improved readability across various components.
This comment was marked as resolved.
This comment was marked as resolved.
…e forecast div validation in fetch_forecast function
…hance pytest settings in pyproject.toml, and add Bandit security checks to CI workflow; consolidate testing and linting strategy documentation.
Owner
Author
|
/gemini review |
…ests; replace with updated YAML configurations for improved clarity and user guidance.
Contributor
There was a problem hiding this comment.
Code Review
This is an excellent and comprehensive pull request that successfully migrates the forecast fetching mechanism from fragile web scraping to the robust NWS API. The new architecture is well-designed, incorporating best practices like caching, robust error handling with retries and circuit breakers, and clear separation of concerns with new modules for the API client, data processing, and configuration. The inclusion of a hybrid mode for a smooth transition is a thoughtful touch. The code quality is high, and the addition of extensive documentation and tests is commendable. I have a couple of suggestions for refactoring to improve maintainability and robustness, but overall, this is a fantastic contribution.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migrate the SatCom forecast system from web scraping to the NWS API to improve reliability, data quality, and maintainability.
The previous web scraping implementation was fragile and prone to breakage due to changes in the target website's HTML structure. This migration to the official Weather.gov API provides a stable, structured data source, along with robust error handling, caching, and a hybrid mode for a smooth transition, ensuring long-term stability and better data accuracy.
Note
Migrate SatCom Forecast to Weather.gov API with hybrid fallback, caching, robust error handling, updated parsing/formatting, and comprehensive tests/CI tooling.
api_client.py,api_models.py,api_data_processor.py,api_formatter.py,api_cache.py,api_error_handler.py,api_config.py, andforecast_fetcher_api.py(hybrid API/HTML, caching, rate limiting, retries, circuit breaker).coordinator.pyto use newfetch_forecast(hybrid) and improve logging and config handling.forecast_parser.py(expanded valid period names incl. holidays, probability/context parsing, wind/temp extraction, summary/compact/full formatting consistency).weather_utils.pyfor event detection, chance inference, wind/temp extraction, and day filtering.imap_handler.py(robust decoding, folder handling, days override) and type-safety improvements innotifier.py.__init__.pyentry migration (v2→v3, v3→v4) and debug service.tests/plus scripts (scripts/run_api_tests.py, discovery/connectivity).pyproject.toml/requirements.txt.Written by Cursor Bugbot for commit 5b81d1f. This will update automatically on new commits. Configure here.