require strict base64 decoding

[reaperhulk]

Fernet tokens use URL safe base64 encoding. Platforms like Python (and Ruby, etc) have chosen to implement decoding of this in a very liberal way. Specifically, characters outside the base64 alphabet are silently dropped and parsing is terminated when valid padding is detected. This is at odds with the recommendations in RFC 4648 (section 3.3), but is not explicitly disallowed. In practice this means it is possible to take a valid Fernet token and craft one that appears different (e.g. with invalid characters or garbage after the padding char(s)) that will still validate. This is a problem as clients may not realize that, counterintuitively, the token is semi-mutable prior to decoding without affecting the HMAC.

The proposed fix is to add new invalid test vectors that exercise these behaviors and require conformant implementations to do more rigorous checking on the base64 prior to decoding. In Python this might look like:

INVALID_B64_RE = re.compile("[^-A-Za-z0-9_=]|=[^=]|===$")
if INVALID_B64_RE.search(token):
    raise InvalidToken

[dolph]

If an implementation could pass the proposed tests, then that would certainly fix the issue that was filed in OpenStack. I'm curious to see an example of the implementation changes required to match this proposal, though.

[reaperhulk]

Here is an implementation: https://github.com/reaperhulk/cryptography/tree/strictb64 (reaperhulk/cryptography@62f45f5 is the commit)

I'm not happy with it (regex scanning the token is an awful solution), but I'm unsure of a better way (at least until I can convince python to add a strict flag to b64decode).