Automated Testing / CI

[JesusMcCloud]

Hi!

I've realized that the (old) SW attestation root is hardwired in the code as explicitly blacklisted.
How would I then test inside a CI without having to manually construct an attestation certificate chain to a custom root? After all, I don't want to be inadvertently testing whether I correctly constructed an attestation chain, but just use an emulator to make sure the chain is representative of a real chain. (See also our GitHub action that spins up an emulator to attest against a service running on localhost).

On a related note:
This approach for CI only works up to Android 12, because more recent Images seem to rotate(?) the root certificate. I can only guess that it might have something to do with also emulating RKP (Android 13+ emulators can also "run out of keys" just like a real RKP-capable device).
This begs the question: how is CI supposed to be done?

• Use Android 12 images and patch out the hardwired blacklist for the SW-certificate?
• Use newer emulator images, create a chain, extract the root, configure that as trust anchor before each test?

Full disclosure: I have yet to wrap my head around how the rotation/rollover of emulator root certificates works, because in the GitHub CI pipeline that use the same Gradle-managed emulator for testing uses different roots than on my local machine and (other) post-Android-12 emulators on other build hosts use yet other certificates. Are those root certs dynamically created on first boot of the emulator on each host?

All that being said, blacklisting the SW attestation root does not affect us so far, but should any future refactor pull the "SW attestation root = bad" logic into the PKIX cert chain validator, it would break not only our CI but also those downstream who use Warden Supreme in production and rely on emulators for testing on T stages, etc…

[sethmoo]

Ah, your concern makes more sense now! It was previously relayed to me and I didn't quite get it.

We should probably have an "ignore roots" specifically for test paths. As you've noticed, devices with remote key provisioning get test certs with rotating roots. This is because we don't have a test root (an intentional design choice).

Any API that weakens the output is going to be fraught with peril, as it could be misused then cause a security vuln. We'll think about how to enable your use-case.

(I've mirrored this issue on our internal tracker. It may be some time before we update here.)

[JesusMcCloud]

Thanks for the update!

I don't think "ignore roots" is the right call. Ignoring roots vs rejecting everything but an SW attestation root will not make it possible to test the equivalent of what you'd have on a P stage with real devices on a T stage with an emulator. It will behave differently.

It's not just that the outcome will be different (i.e. some chains being accepted, even if it is an untrusted root), but also cases where the general outcome of "this device has been tampered with" is expected will behave differently. While this is irrelevant for the verdict of rejecting devices, it is very much relevant when it comes to customer support:
Even though you don't want to transparently communicate all error cases to client, you almost certainly want to show an opaque error code that differs depending on the cause of an attestation failure. Because users (esp. if you have a user base in the millions who use your service daily) will complain and contact customer support if their device gets rejected. So they need that code when they contact customer support.

And if your automated emulator-based tests on the T stage behave differently than real devices on P, you're back to manual testing with real devices also on T.
Think of it: If you start handling trust differently, how will you guarantee your automated tests to behave the same?

Concrete example that causes problems with different PKIX cert path validation outcomes. For customer support requests, you need to reliably distinguish between "this is a device intended for the Chinese market with an untrusted root and no Google Play certification" and "Unlocked bootloader detected". One will be a PKIX cert path validation error. The other will be triggered when examining the leaf cert extension value. Now if the cert path validator just passes everything through, pretty much all bets are off wrt. enumerating error cases that are relevant in very service-specific settings for customer support.

Again, this does not affect us, because we have our own logic for validating the contents of the certificate extension, but we do incorporate the cert path validator from this repo. So if this check remains in the codepath that we never use, it's fine for us. And while I have confidence in Warden Supreme and the policy-based approach we're following there, I am also realistic and I am hence equally confident that people will flock to this library once it is on maven central. By now you know that I do have opinions on quite some aspects of attestation verification. This here, though, I feel is just objectively problematic and has the potential of degrading security. People will shoot themselves in the foot. If you take their gun away, they'll use bow an arrow, or worse – a bazooka:
Nobody should rely on attestation to establish trust in mobile clients without a thorough understanding at least on a conceptual level. But integrators lacking expertise and judgement are a real problem!
And if, in scenarios where integrators are lacking even a conceptual understanding, you start offering an "ignore roots" switch instead of a way to explicitly manually configure an SW root for the T stage, my money is on root validation being turned off in production "because it will make the CI green and users are happy". In that sense, you might be making it harder for some people to shoot themselves in the foot, but those who do will blow their whole leg off.

All that being said, some way to pin a custom root in upcoming emulator images would be awesome, because this way we could just provision (or pin or whatever) a custom SW root in the emulator (without hacks, that is!), use that root for testing and effectively get the best of both worlds.