RFC: ML energy consumption predictor#31
Closed
pookey wants to merge 2 commits intojohanzander:mainfrom
Closed
Conversation
Standalone ML module (ml/) using XGBoost to predict 24h energy consumption at 15-minute resolution. Integrates with the battery optimizer via a new consumption_strategy setting that supports four modes: sensor (default), fixed, influxdb_profile (7-day weekly average), and ml_prediction. Key components: - ml/ module: CLI tool with train, predict, evaluate, and report commands - MLReportPage: Frontend dashboard tab showing model metrics and forecasts - /api/ml-report endpoint for model status and prediction data - Configurable via config.yaml ml section (location, features, training params) - Daily retrain cron job at 23:00, predictions cached with date-based invalidation - InfluxDB batch power sensor fetching for training data - Historical data persistence across container restarts Infrastructure changes: - Switched Docker base from Alpine to Debian (required for xgboost wheels) - Added scikit-learn, xgboost, astral to requirements - Power sensor gap-filling in SensorCollector for historical backfill - Nordpool price sensor methods in HA controller Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two bugs prevented ML predictions from working: 1. battery_system_manager.py: _retrain_ml_model() and _generate_ml_predictions() called load_config() with nonexistent keyword args (ml_section, influxdb_url, etc.), causing silent failures. Fixed to use load_config(app_options=self._addon_options). 2. ml/config.py: Used HA_TOKEN env var for HA API auth, but the HA supervisor sets HASSIO_TOKEN. Weather forecast calls failed with 401 errors. Now checks HASSIO_TOKEN first, falling back to HA_TOKEN for local development. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds an optional ML module for predicting 24h energy consumption at 15-minute resolution using XGBoost. It integrates with the battery optimizer via a new
consumption_strategyconfig setting.Important caveats — please read before reviewing:
influxdb_profilestrategy). The model needs significant training data and tuning to outperform naive baselines.mlconfig section is fully optional — if omitted, the system behaves exactly as before with zero overhead.I'm happy to discuss whether this should live in core, be a separate add-on, or be structured differently. Opening this PR mainly to share the work and get your thoughts on the approach.
What's included
ml/module (6 files, ~2500 lines): Standalone XGBoost predictor with CLI (train,predict,evaluate,report)sensor(default, unchanged),fixed,influxdb_profile(7-day weekly average from InfluxDB),ml_prediction/api/ml-reportendpoint: Model status, predictions, and comparison datamlsection in config.yaml (location, feature sensors, training params)Changes from previous PR #23
All 9 review items from @johanzander's feedback on #23 have been addressed:
config.yamlml section with placeholder valuesml.location.timezone), not hardcodedconsumption_strategyloaded infrom_ha_config()with direct dict access (no silent fallback).get()with fallback defaults for required settings — fails loudly if missingconsumptionStrategyfield has no default inAPIBatterySettingsrequirements.txt(needed at runtime for strategy dispatch)What's NOT included (intentionally excluded)
Test plan
ml reportcommand completes and generates HTML dashboardconsumption_strategy: "sensor"(no ML overhead)consumption_strategy: "ml_prediction"and uses ML forecastsinfluxdb_profilestrategy produces reasonable weekly average profiles