Scan results for security.txt appear to only check in the root directory and not under /.well-known

[bmay-doi]

Describe the bug

The scan for blm.gov previously marked us as successfully having a security.txt file, but the latest scan lists it as missing, with no changes to the security.txt file. Other sites that have a security.txt file only under /.well-known also are flagged as failing, while sites that have the security.txt file under the root directory (or both) are marked as passing. According to RFC 9116, the /.well-known directory is the preferred location for the security.txt file.

To Reproduce

Steps to reproduce the behavior:

1. Go to the Security section of the scan results for blm.gov.
2. See that security.txt is listed as missing.
3. Go to https://www.blm.gov/.well-known/security.txt and see that a security.txt file is present.
4. Go to https://www.blm.gov/security.txt and see that no security.txt file is present.
5. Go to the Security section of the scan results for fws.gov.
6. See that security.txt is listed as present.
7. Go to https://www.fws.gov/.well-known/security.txt and see that no security.txt file is present.
8. Go to https://www.fws.gov/security.txt and see that a security.txt file is present.

Expected behavior

The security.txt file should be detected as present when found in either the root level or the /.well-known directory.

Screenshots

Desktop (please complete the following information):

• OS: Windows 11 Enterprise (23H2)
• Browser: Chrome
• Version: 139.0.7258.155