Allow aiodynamo to fetch credentials for snapstart lambdas.

[aclemons]

For snapstart, the lambda boots once to do any static init.
Subsequently, this snapshotted version is used for handling subsequent
requests.

Normally for lambdas, the AWS credentials are in the environment. Since
snapstart will reuse a snapshot of the running application, it can't
pass in variables in this way since they need to be refreshed.

Instead, the AWS_CONTAINER_CREDENTIALS_FULL_URI env var is set and the
ContainerMetadataCredentials in aiodynamo should be used to get
credentials if using Credentials.auto() to initialise.

Unfortunately, it looks like AWS formats the expiration timestamps
slightly differently for this API on lambda, including the microseconds
now. The parse_amazon_timestamp function fails to parse the value,
resulting in ContainerMetadataCredentials being rejected as a
ChainCredentials candidate and we end up with no credentials at all.

I can't really find a definitive documentation of what the format should
be so I can point to it, but obviously we know the code as it is works
on ECS/EC2 etc so we must continue to be able to parse those. I've
simply added a fallback to use microseconds if the string has a
full-stop in it.

It seems botocore is using the dateutil package to handle their
parsing:

https://github.com/boto/botocore/blob/f49ead849aa5a4ea428d9f378de14db6f4c6d645/botocore/utils.py#L950

[dimaqq]

The change make sense, but feels like it's only the first step in supporting snapstarts?

Or did I misread the PR description?

[dimaqq]

IIRC, Python's datetime parsing thing can parse fractional seconds, but only with three or six fractional digits exactly:

• 59.123 ok
• 59.120 ok
• 59.12 Error

Please collect more test vectors and/or add add more test inputs to cover this corner case.

[aclemons]

The parser will handle any length from 1 to 6, it just right pads the values. i.e. if AWS were to put 59.12 in the Expiration field of the credential response it will be interpreted as 59.120000

The docs are here:

https://docs.python.org/3/library/datetime.html#format-codes

Directive	Meaning	Example	Notes
%f	Microsecond as a decimal number, zero-padded to 6 digits.	000000, 000001, …, 999999	(5)

5. When used with the strptime() method, the %f directive accepts from one to six digits and zero pads on the right. %f is an extension to the set of format characters in the C standard (but implemented separately in datetime objects, and therefore always available).

[dimaqq]

cc @ojii

[aclemons]

The change make sense, but feels like it's only the first step in supporting snapstarts?

Or did I misread the PR description?

I think you misread.

Normally for lambdas, the AWS credentials are in the environment. Since
snapstart will reuse a snapshot of the running application, it can't
pass in variables in this way since they need to be refreshed.

Instead, the AWS_CONTAINER_CREDENTIALS_FULL_URI env var is set and the
ContainerMetadataCredentials in aiodynamo should be used to get
credentials if using Credentials.auto() to initialise.

In a snapstart lambda, the AWS_CONTAINER_CREDENTIALS_FULL_URI env var is set and aiodynamo tries to use ContainerMetadataCredentials but cannot parse the expiration field in the response. Once the timestamp parsing is fixed, it can correctly use the provider and everything else works as it does on a normal lambda.

We've been using aiodynamo for a production snapstart lambda for about 6 or 7 months now with the parsing function monkey patched to handle the microseconds.

[dimaqq]

The change looks good for me.

Tested manually on py3.8 ~ 3.14.

What works:

• without fractional part (old code path)
• with fractional part, from .1 to .123456

What doesn't work:

• ...01:02:03. (with a dot but without the fractional part): this is fair
• ...01:02:03.1234567 and longer: I guess that's OK?

[dimaqq]

cc @ojii