API Authentication not working #2
Description
I have set up the app but im having an issue with the authentication. I have nyseg and I put my username and password in the config file. it says the initial authentication worked but when it goes to make the api call to grab the data, the auth fails. without digging into the code it looks like to me the auth being sent with the request is either wrong or missing. Potentially the API endpoint changed what authentication it accepts, therefor breaking the request.
This is what I get after set up and trying to run a fetch the first time
=== Fetch started at 2026-01-25 13:40:45 EST ===
No cookies found, performing initial login...
Refreshing authentication...
✓ Authentication refreshed successfully
✓ Login successful, credentials saved
Fetching data from nyseg (last 90 days)...
Refreshing authentication...
✓ Authentication refreshed successfully
Making API request to: https://engage-api-gw-dod79bsd.ue.gateway.dev/usage/usage/download?commodity=electric&date=2025-10-27&end_date=2026-01-25&format=csv&from_ces=True
⚠ Scraping failed: authentication failed (status 401): {"code":"AUTHENTICATION_REQUIRED","message":"X-Authentication header is required"}
⚠ Attempting to refresh credentials and retry...
Refreshing authentication...
✓ Authentication refreshed successfully
✓ Credentials refreshed and saved
Retrying fetch with fresh credentials...
Making API request to: https://engage-api-gw-dod79bsd.ue.gateway.dev/usage/usage/download?commodity=electric&date=2025-10-27&end_date=2026-01-25&format=csv&from_ces=True
Error: scraping failed after auth refresh: authentication failed (status 401): {"code":"AUTHENTICATION_REQUIRED","message":"X-Authentication header is required"}
Usage:
gridscraper fetch [service] [flags]
Flags:
-h, --help help for fetch
--visible Show browser window (for debugging)
Global Flags:
--config string config file (default is ./config.yaml)
--db string database file (default is ./data.db)
Error: scraping failed after auth refresh: authentication failed (status 401): {"code":"AUTHENTICATION_REQUIRED","message":"X-Authentication header is required"}