Login Occuring but Still Unauthenticated Per Status Code

[ntilley905]

I've been noticing that the sensor has been unknown for a while, probably the same issue as #8. I made sure that there isn't a security check by logging in via a browser and then enabled debug logs and restarted. It appears that the component is logging in, but then the actual data is returning a 401 Unauthenticated error code. Logs are below.

2020-06-29 08:56:25 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for xfinity which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant. 2020-06-29 08:56:42 DEBUG (SyncWorker_12) [custom_components.xfinity.sensor] Finding reqId for login... 2020-06-29 08:56:43 DEBUG (SyncWorker_12) [custom_components.xfinity.sensor] Found reqId = 5664b480-3509-48e1-884e-25ab3a225389 2020-06-29 08:56:43 DEBUG (SyncWorker_12) [custom_components.xfinity.sensor] Posting to login... 2020-06-29 08:56:44 DEBUG (SyncWorker_12) [custom_components.xfinity.sensor] Logged in successfully, status_code: 200 2020-06-29 08:56:44 DEBUG (SyncWorker_12) [custom_components.xfinity.sensor] Fetching internet usage AJAX... 2020-06-29 08:56:44 ERROR (SyncWorker_12) [custom_components.xfinity.sensor] Failed to fetch data, status_code:401, resp: {'error': 'unauthenticated'}

[ntilley905]

Not sure why the logs above formatted strangely, the line breaks are in the right place. Regardless, some additional info. I tried getting the API data through a few other sources and it appears that it's returning a 401 Unauthorized. Not sure why Requests throws that as Unauthenticated, but it's just a run of the mill 401. I'm wondering if this API endpoint is no longer available? I'm having trouble finding any data as to what is made available via API access, it doesn't look like Xfinity makes any of that publicly available, at least not easily.

[ntilley905]

Disregard the above, when I logged in via Firefox and pulled the API data through there, it returned successfully. When I dug in a little more to the response being provided through requests, it is returning an error of "errorMessage" : ["authn.unsupported_browser.error"] and further clarifying that the browser is not supported. Not really sure how to get around that one...

This is also causing the debug logs to be incorrect. The response provided after the login action does have a response code of 200, but it is not actually logging you in.

[ntilley905]

Some more info (sorry for not consolidating into one comment): the unsupported browser is easily fixed by adding headers. Once I got past that issue, however, it is saying that I am using an invalid user/password. I've confirmed that I'm not, it just seems like something is getting tripped up in the post process. It's still returning a 200 on the login post action, but within that response there is an error that says the info is incorrect (thanks for using the correct status codes, Comcast). To be continued...

[robert-alfaro]

Thanks for debugging! I'll be looking into this soon

[kendrome]

@robert-alfaro Any luck? Would be really helpful now that caps are back.

[cszucko]

I stumbled upon this thread the other day while looking for a fix for a similar tool of mine. I haven't come up with a solution yet but, to share what I've seen, I believe the source of the trouble is that the login page is now using some obfuscated JavaScript to generate a series of fields (see below) that need to be present in the POST request.

X-hzfdeCEGvt-f
X-hzfdeCEGvt-b
X-hzfdeCEGvt-c
X-hzfdeCEGvt-d
X-hzfdeCEGvt-z
X-hzfdeCEGvt-a

Without those, the response comes back HTTP 200 but it just dumps you back to the login page and shows a "The Xfinity ID or password you entered was incorrect. Please try again." error.

[migitcheetah]

I am also seeing this issue. Login in from a browser then hitting the api from the same browser seems to work. Maybe a cookie or session issue? Its definitely per browser, as hitting the api from different machines gives me the same error as HA until I log in from that machine.

[jasii]

Is it even possible to get around those generated fields necessary in the POST request? If you're able to log in using a browser then hit the API, maybe it will be necessary to use a headless browser to log in first? That will significantly increase the size of this script and increased processing power though.

[gomsnothereman]

Any updates on this? spent a few hours trying to get this to work on home assistant before i checked the issues tab :). Plugin looks great/handy with so many working from home now and comsucks being comsucks and having a ridiculous low limit for 2020...

[apippin]

It appears that the xfinity login page has started rejecting authentication attempts by automation engines like selenium or python based session requests. Even though I am using a valid username and password, and can login from the same computer and browser into my xfinity account, xfinity returns an invalid username and password error when attempting to do the same thing via this script.

The work around I found was with another homeassistant project that uses Selenium to login and get this data. I posted an issue with a solution to fix the Selenium based approach here: jantman/xfinity-usage#30. I use that approach to get the JSON needed for homeassistant.

[robert-alfaro]

No luck on my attempts to get this working. I've very occupied with work and have little to no time to give this the attention is deserves. I used to use the headless approach in the past but it stopped working. Great that's still a go.

[Vendo232]

It appears that the xfinity login page has started rejecting authentication attempts by automation engines like selenium or python based session requests. Even though I am using a valid username and password, and can login from the same computer and browser into my xfinity account, xfinity returns an invalid username and password error when attempting to do the same thing via this script.

The work around I found was with another homeassistant project that uses Selenium to login and get this data. I posted an issue with a solution to fix the Selenium based approach here: jantman/xfinity-usage#30. I use that approach to get the JSON needed for homeassistant.

would you be able to share the working json for HA? maybe the whole XFINITY custome folder? thnak you very much

[apippin]

This is still working great for me. I have compressed my comcast custom config dir and pointed to the url at the end of this post that you can download. It contains a comcast.yamls file with all of the different places I changed in the hass yaml files to support this flow. You can uncompress this into your hass/config/ dir. You can look through the files and figure out what to change to make it work for you. I’ve posted it on my website since I couldn’t attach it here. You can download it from here: comcast.tar.gz

[Vendo232]

I`m getting this error when I try to run compast.py, any idea what could be wrong?

…

________________________________ From: Alan Pippin <notifications@github.com> Sent: Friday, October 2, 2020 13:43 To: robert-alfaro/xfinity-usage <xfinity-usage@noreply.github.com> Cc: Vendo232 <vendo232@hotmail.com>; Comment <comment@noreply.github.com> Subject: Re: [robert-alfaro/xfinity-usage] Login Occuring but Still Unauthenticated Per Status Code (#9) This is still working great for me. I have compressed my comcast custom config dir and pointed to the url at the end of this post that you can download. It contains a comcast.yamls file with all of the different places I changed in the hass yaml files to support this flow. You can uncompress this into your hass/config/ dir. You can look through the files and figure out what to change to make it work for you. I’ve posted it on my website since I couldn’t attach it here. You can download it from here: comcast.tar.gz<https://cloud.pippins.net/s/ej85TopaqGfajTF/download?path=%2Fhass&files=comcast.tar.gz> — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#9 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKQOFNRWYILPYGU3KH3PXY3SIYUOBANCNFSM4OLF7LIA>.

[migitcheetah]

This is still working great for me. I have compressed my comcast custom config dir and pointed to the url at the end of this post that you can download. It contains a comcast.yamls file with all of the different places I changed in the hass yaml files to support this flow. You can uncompress this into your hass/config/ dir. You can look through the files and figure out what to change to make it work for you. I’ve posted it on my website since I couldn’t attach it here. You can download it from here: comcast.tar.gz

I have not attempted you fix yet, but I was looking through the files on my phone, and saw references to comcast.py in the readme, but the comcast.py supplied labeled old.broken.
Also there are references to comcast_usage.json and xfinity-usage.data that I am not seeing in the package.

I was just wondering if there was something missing from the zip file?

edit:
I do see xfinity-usage file, but not .data, so that might be it, just want to make sure.