Add HPKE support

[alex]

We've gotten some feedback that it'd be valuable for us to add HPKE support.

TODO:

• Review RFC, decide what subset (if any) we think makes sense to implemen
• Design an API
  • Is this sufficiently misuse resistant that it could be considered a recipe (i.e., not hazmat)?
• Gather test vectors/KATs
• Implement (including tests + docs)

[alex]

Inspiration on scope + API:

• https://pkg.go.dev/github.com/cloudflare/circl/hpke
• https://github.com/ctz/hpke-py/
• https://pkg.go.dev/crypto/internal/hpke
• https://docs.rs/hpke/latest/hpke/

[SalusaSecondus]

My own two cents:

1. 1 shot base mode is definitely safe enough to expose as a recipe. (This is especially true if it can return a single "ciphertext" consisting of both the encapsulated key and the ciphertext.)
2. Base mode (in general) is probably safe enough to expose as a recipe.
3. As useful as I personally find it, the Auth modes may not be worth building in :-(
4. PSK modes, while safe, might be better in hazmat simply because reasoning about them can be complicated.
5. Exported secrets is a critical feature (but very possibly hazmat)

[alex]

Thank you! If we think its right that different modes should live in different layers, that has some (fiddly!) implications for our API design.

[SalusaSecondus]

Something I've built into my (non-public, unfortunately) implementations of HPKE is a unified 1-shot interface which I think is the ideal one to expose as non-hazmat. While the specification does describe it, the below combines the return values into one which I think is better for the average user. (If people care about compatibility with other implementations, they can simply split off the constant-length prefix from their ciphertexts here.)

(Doing Java for API definition because I'm much better at it than Python. Sorry.)

enum HpkeKem{};
enum HpkeKdf{};
enum HpkeAead{};

// Returns encapsulated key + ciphertext
byte[] oneShotHpkeSeal(
    HpkeKem kem,
    HpkeKdf kdf,
    HpkeAead aead,
    byte[] recipientPublicKey,
    byte[] info,
    byte[] aad,
    byte[] plaintext);

byte[] oneShotHpkeOpen(
    HpkeKem kem,
    HpkeKdf kdf,
    HpkeAead aead,
    byte[] recipientPrivateKey,
    byte[] info,
    byte[] aad,
    byte[] ciphertext);

So, with the above in normal and all detailed implementations in hazmat, I think we've a safe interface.

[FiloSottile]

We are shipping a public API in Go 1.26, so I have thoughts :)

https://pkg.go.dev/crypto/hpke@go1.26rc1

We shipped only base mode, not PSK-authenticated (which requires high-entropy secrets and feels like it will be misused with passwords) nor KEM-authenticated which is being removed in -bis and is not compatible with PQ.

We actually shipped the PQ KEMs and SHAKE KDF, even if they are drafts. We got informal commitments the wire encoding won't change (except for security issues ofc) and it feels wrong to ship without PQ at this point.

You might find this summary useful since the PQ KEMs are defined over four different documents: filippo.io/hpke-pq

I played a lot with the API, you can see iterations and rationales at golang/go#75300.

In short, public functions return AED/KDF/KEM interface types with ID methods. The KEMs are interfaces with GenerateKey/NewPublicKey/NewPrivateKey/DeriveKeyPair methods. NewSender takes PublicKey + KDF + AEAD, and returns a type that can do Seal and Export. NewRecipient takes encapsulated key + PrivateKey + KDF + AEAD, and returns a type that can do Open and Export.

We have one-shot Open and Seal which take info BUT NOT aad (since they are redundant and the RFC recommends picking info for one-shot). They return/take a single enc || ciphertext slice.

The named public functions encourage picking one KEM, one KDF, one AEAD, instead of supporting them all.

The RFC has test vectors, and https://github.com/hpkewg/hpke-pq/blob/main/test-vectors.json are the vectors for the PQ KEMs and SHAKE KDF suites.

Maybe the difference between hazmat and recipe is that the recipe picks the AEAD, KDF, and PQ KEM for the user.

[alex]

@FiloSottile thanks! what's the argument for including export vs. skipping it entirely?

[FiloSottile]

No strong argument, it’s just a standard way to expose the KEMs, which could be useful in higher level protocols. It felt nearly free because I can’t really think of how it could be misused, and it was just a method on a concrete type.

[alex]

I'm currently leaning towards:

• Hazmat API supporting any of the ciphers/KEMs/KDFs that we already support
• base only, no PSK, no auth-only, no export
• API that looks roughly like Suite(Cipher, KEM, KDF) -- haven't decided what types we should do for each of those, something named though
  • suite.sender(public_key, info) + sender.seal(data, aad) (returns concatted like Go)
  • and the equivalent on the receiver side
  • I don't love the sender/receiver names (or the seal/open for that matter) so I'd love to hear pro/cons
• For us to put something in not-hazmat, I think I'd want:
  • We need PQ support
  • Some sort of agreement with other implementors on "this is The Right Default" and that would ensure cross-language compatibility

[FiloSottile]

suite.sender(public_key, info) + sender.seal(data, aad) (returns concatted like Go)

That looks like a multi-step API but concatted mostly makes sense for one-shot.

[bifurcation]

@alex - That seems like a sensible starting point to me. Couple of notes.

• Base-only - If you're aiming for a simple starting point, I would also restrict to the single-shot operations.
• For hazmat, I might preserve the separation between the enc and ct outputs, just because I prefer not to be opinionated about formats in hazmat-grade APIs. For non-hazmat, I agree that concat is fine.
• If you're going to restrict to single-shot, something like suite.seal(public_key, info, data, aad) would be sufficient.If you want to keep the door open to multi-shot later, your API seems fine.
• w.r.t. Sender/Receiver/Seal/Open - We used seal/open in the RFC because there seemed to be some use of that for AEAD operations, e.g., in Go, vs. encrypt/decrypt for unauthenticated encryption. If you don't like Sender/Receiver (which I guess reflect the origins in communications protocols), you could make nouns out of whatever verbs you use, e.g., Sealer / Opener.
• Re PQ - Coming soon. Basically stable now, should be final final I would guess 1H2026.
• Re "Right Default" - What level of agreement are you looking for? Ciphersuites? Data formats?

[alex]

re: default, what I'd be looking for is something to the effect of "a few implementers agree that this is a solid default that they'll have available and that we'll always be able to interop". Shouldn't be a high bar!

[bifurcation]

So it seems like maybe you're thinking more in terms of ciphersuites than the data format for (enc, ct), since in principle different forms of the latter can be freely converted.

I'm not an implementer in any salient way here, but my read of the community right now is:

• For non-PQ, there's pretty good agreement on P-256 and/or X25519 with HKDF-SHA-256 and AES-GCM as a baseline, with some folks looking for higher security levels / ChaCha
• For PQ, things are less consolidated, but I think the center of mass seems to be around ML-KEM, SHAKE, and AES-GCM, with a bit of disagreement around whether to use SHAKE or HKDF-SHA-256.
• For PQ/T hybrid, basically just pair things up, with the CFRG hybridization scheme.

There is of course not much implementation of the PQ or PQ/T hybrid versions right now, since they're not final.

[FiloSottile]

I think MLKEM768-X25519 (née X-Wing) + HKDF-SHA-256 + AES-GCM-256 is a fine default. It's the only named PQ cipher suite in Apple CryptoKit and I have no objection to it besides wasting some AES rounds and mixing SHA-2 and SHA-3 (if you squint hard enough to see the SHA-3 inside ML-KEM), neither of which actually matter to users.