AEAD Nonce Reuse and Its Security Consequences
AEAD nonce reuse means encrypting two messages with the same key and the same nonce. In most AEAD schemes, that breaks the security guarantees the scheme provides: confidentiality can fail, and authenticity can fail catastrophically.
A nonce is a value intended to be used only once with a particular encryption key. It usually does not need to be secret, so protocols commonly send it alongside the ciphertext. The requirement is about uniqueness, not concealment.
The problem AEAD solves
Before AEAD, an application often had to combine two separate mechanisms: encryption to hide a message and a MAC, or message authentication code, to detect changes. Getting the order, inputs, and key handling right was difficult. It was easy to encrypt one set of bytes but authenticate another, or to forget important metadata such as a request method or protocol version.
AEAD, authenticated encryption with associated data, combines these jobs. It encrypts the plaintext, authenticates the resulting message, and can also authenticate data that remains visible. That visible data is called associated data: for example, a packet header or database record identifier that must be checked but need not be encrypted.
An AEAD operation conceptually takes:
key, nonce, plaintext, associated_data
and produces a ciphertext and an authentication tag. On decryption, the recipient supplies the same key, nonce, and associated data. It gets the plaintext only if the tag verifies.
The tag is important: it lets the recipient reject ciphertext that has been modified. But AEAD does not automatically make nonce management safe. The caller, or the protocol around the caller, must provide a nonce that has not already been used with that key.
Why confidentiality fails first
Many AEAD encryption components behave like a stream cipher. They generate a pseudorandom keystream from the key and nonce, then combine that keystream with the plaintext. In simplified notation:
ciphertext = plaintext XOR keystream(key, nonce)
If two messages use the same key and nonce, they use the same keystream:
ciphertext_1 XOR ciphertext_2 = plaintext_1 XOR plaintext_2
The keystream itself cancels out. This does not always reveal both plaintexts immediately, but it removes the protection that encryption was supposed to provide. An attacker can compare the messages, infer structure, and use a known or guessed portion of one plaintext to recover the corresponding portion of the other. Natural-language text, protocol headers, JSON, and predictable file formats give attackers many useful guesses.
For example, if an attacker knows that one encrypted request begins with a predictable JSON prefix, the attacker can derive the keystream for that prefix and use it against the other ciphertext wherever the same positions overlap. Even without known plaintext, the XOR relationship leaks how the two plaintexts differ.
This is the same fundamental failure that makes reusing a one-time pad unsafe. AEAD's nonce is what tells the encryption component which fresh keystream to generate. Reusing it asks for the same one again.
Why authenticity can fail too
Nonce reuse also undermines the authentication tag, although the exact failure depends on the algorithm.
In AES-GCM, the tag is built partly from a polynomial authentication calculation using a secret value derived from the AES key. Reusing a nonce causes two encryptions to reuse the same underlying counter stream and authentication setup. The relationship between the two tags and the known ciphertexts can reveal information about that secret authentication value. With enough information—often including a chosen or predictable plaintext—an attacker may be able to construct valid tags for modified or entirely new ciphertexts.
That turns an apparent decryption failure into something much worse: the recipient may accept attacker-controlled data as authentic. A forged message might change an authorization decision, alter a payment amount, or inject commands into a protocol.
ChaCha20-Poly1305 has a related but different failure. Each nonce is supposed to produce a one-time Poly1305 authentication key. Reusing the nonce reuses that key, allowing relationships between tags to expose the authentication calculation and potentially enable forgeries. The details differ from GCM, but the engineering conclusion is the same: nonce reuse is not merely a small reduction in security margin.
With several widely used AEAD schemes, one nonce collision can be a security incident affecting both secrecy and message acceptance.
How nonce reuse happens in real systems
The mistake is often in protocol state rather than in the cryptographic library:
- A random nonce generator eventually produces a duplicate, especially if the nonce is too short or many messages are encrypted.
- A process restart resets a counter to zero while keeping the same key.
- Two machines independently use the same counter range under one shared key.
- A message is retried with a newly generated key assumption that was never actually true.
- A developer treats a nonce as a constant configuration value because it is public.
Random nonces can be safe when the space is large enough and collision probabilities are carefully bounded. A 96-bit random nonce, commonly used with AES-GCM, has a much better collision profile than a short random value, but “large” is not the same as “impossible.” At high message volumes or across many devices, a counter or coordinated allocation scheme can provide a stronger uniqueness guarantee.
A counter must itself be managed safely: its state must survive restarts, and concurrent or distributed writers must never allocate the same value under the same key. Another common design is to derive separate keys for separate devices or sessions, so each counter only needs to be unique within its own key domain.
Why encryption cannot repair it afterward
The encryption operation sees a key, nonce, plaintext, and associated data. It cannot know whether another process used that same nonce yesterday, or whether another host is using it now. A nonce is intentionally public, and the cryptographic output contains no universal record of every prior use.
A library can reject an explicitly repeated nonce if the application keeps a history, but maintaining that history is state management with storage, crash recovery, and coordination requirements. It is not something the cipher can infer after the fact. Some APIs make nonce generation convenient; convenience is not a protocol guarantee.
Once two ciphertexts have been produced with the same key and nonce, rotating the key protects future messages but does not restore the confidentiality of the old pair or invalidate a forged message that was already accepted. The protocol may need key revocation, incident response, and recovery of affected data.
The practical rule is therefore simple: design nonce allocation as part of the protocol, document its scope as “unique per key,” and test restarts, retries, replication, and key lifetimes. AEAD provides the security construction; the surrounding system must provide the uniqueness assumption that construction requires.