Possible miscategorizing of aws-hosted sites

[S4lt5]

💡 Summary

When running findcdn against a site like www.ahcp.gov, I get the following output:

❯ findcdn list www.achp.gov

{
   "date": "10/27/2022, 14:22:16",
   "cdn_count": "1",
   "domains": {
       "www.achp.gov": {
           "IP": "'3.32.142.183', '3.32.4.248'",
           "cdns": "'.amazonaws.com'",
           "cdns_by_names": "'Amazon AWS'"
       }
   }
}

I also put some debug prints in to see the following values:

HEADERS:  ['Apache']
Whois:  ['AMAZON EXPANSION, IE', 'AMAZON-EC2-USGOVCLD', 'AMAZON EXPANSION, IE', 'AMAZON-EC2-USGOVCLD']

When looking at various static files on the site, none appear to have any indication of being served via CDN (no x-cache-*, no via, no .cloudfront url, etc)

Also looking at the whois data, there appears to be at least some hint that this is an ec2 instance serving static files, which probably does not fall into our "has a CDN" category.

What do you think?

Motivation and context

Accuracy of reported output is important to me, and I'm not sure if there is a change to how such a site should be classified.

Implementation notes

Unsure

Acceptance criteria

Unsure

[Pascal-0x90]

This is a good idea to fix. The classification of Cloudfront domains should be more accurate than to just assume all AWS hosted sites use Cloudfront. Stashing this in a TODO for now.

[Pascal-0x90]

Suggestion for using IP blocks instead of just domain name listed here: #43 should help. Also, I think what is causing this is this line

findcdn/src/findcdn/cdnEngine/detectCDN/cdn_config.py

Line 28 in 7bbf50e

".amazonaws.com": "Amazon AWS",

Adds anything that has amazonaws.com in the name to see it as an "Amazon AWS" CDN which realistically is not fully correct as you mention above. I think a lot of the other domains in this list suffer from this.

For example, something ending in .discord.com does not mean it was specifically hosted on cdn.discord.com.