One non-reversible FPE

[ssabnis]

Description of Issue

I want to use the FPE for one-way format preserving tokens only. How do I use the FF3 to achieve that. Any help appreciated.

Steps to Reproduce

If applicable, mention which version(s) of Go you are able to reproduce this on, along with go env output.

What did you expect to see?

[anitgandhi]

Generally, you can't really use an encryption algorithm in a one-way fashion; that problem is solved by cryptographic hashes.

However, I did come across this post that may point you in the right direction, to construct a hash using FPE as a primitive: https://crypto.stackexchange.com/questions/24284/is-there-a-format-preserving-cryptographically-secure-hash

In this context, the encryption algorithm E would be FF3. For the regular hash, you could use SHA256.
Meaning, you could hash your plaintext, and use that hash as the key for encrypting your original plaintext, over FF3.

Please note, I am not a cryptographic algorithm expert, I'm not making any claims, positive or negative, about constructing a format preserving cryptographic hash in this way, nor about its security properties/strength.

[ssabnis]

Thanks @anitgandhi , yes cryptographically secure is the perfect word. Basically I want to implement stateful secure tokenizaton scheme where I must obtain idempotent format preserving tokens obtained for the same data element with the format preserved. You mentioned about the SAH256 , it is an interesting approach. I will check it out- Thanks again.

[anitgandhi]

If you're looking for a stateful implementation, consider generating a random format-preserved value instead (and guaranteeing uniqueness at your DB layer), and maintaining a mapping between the plaintext value and the random value.

For example, if you have some plaintext of "123456789", you can generate a random format-preserving value using any typical cryptographically secure pseudo random number generator (CSPRNG) and an alphabet (just numbers and/or lowercase, uppercase, etc - this depends on your needs).

Of course, at rest, you don't want to store a direct mapping between the plaintext and the secure value. Instead, you can run the plaintext through regular SHA256 and store that as the lookup key.

For a first-time tokenization of a given plaintext value, it would go through the above path; any subsequent times that same value is given, you simply hash the incoming value, giving you the deterministic but secure lookup key in your DB --> the previous random value.

If you don't need to support detokenization, this is sufficient. If you need detokenization, you will need to do more :)

[anitgandhi]

I'm closing this, since it's a question, and not a bug report/feature request. However, feel free to respond with other thoughts 👍

[ssabnis]

@anitgandhi That is true, I am already doing the SHA-256, I need another another layer of lookup, and I need to be able to detokenize the information for further requirements. Any info/reference will help. Thanks again. Please close the issue.

[anitgandhi]

To de-tokenize, you can store an additional field which is simply the encrypted (regular AES-GCM should do fine) value of the plaintext. When you get a request to de-tokenize, you lookup using the random value, get back the ciphertext, and decrypt it back into the original plaintext.

In essence, a relational DB implementation would have 3 columns: SHA256(plaintext) | random generated UNIQUE value | AESGCM(plaintext)

Alternative, if you're using a KV store, you'd have 2 entries for each item:

1. SHA256(plaintext) --> random generated UNIQUE value
2. the same random value --> AESGCM(plaintext)

Anyway, this is just one possible implementation since you are looking at stateful approach. It is simpler in that it doesn't require FPE at all; it uses just standard, well-known and tested cryptographic algorithms.