arthrop0d

Brief articles on varied topics.

DNS Negative Caching and Record Recovery

web developmentnetworking

DNS negative caching is the practice of temporarily storing a DNS answer that says a name or record does not exist. Instead of repeatedly asking an authoritative DNS server for the same failed lookup, a recursive resolver reuses that negative answer until its negative TTL expires.

This matters when you create or change a record. A resolver that previously learned “this does not exist” may continue returning that answer even after the authoritative zone has been updated.

The problem negative caching solves

Without negative caching, every miss would travel through the DNS hierarchy again. A typo, a missing service name, or an application that repeatedly asks for an optional record could generate unnecessary traffic for recursive resolvers and authoritative servers.

Negative caching makes those misses behave like ordinary cached answers. The difference is that the cached result is an absence rather than an address, mail server, or other record.

You may have encountered this as an incident where:

Two kinds of negative DNS answer

The most important negative result is NXDOMAIN, meaning that the queried DNS name does not exist. For example, querying api.example.com might return NXDOMAIN if that name is absent from the example.com zone.

A different result is often called NODATA. It means the name exists, but it has no record of the requested type. For example, example.com may have an address record but no IPv6 AAAA record. The name exists; the requested kind of data does not.

These distinctions matter. An application that asks for an AAAA record can receive NODATA and may correctly fall back to IPv4. NXDOMAIN generally indicates that the name itself is absent.

A DNS response can also fail for reasons that are not authoritative statements of absence. SERVFAIL means the resolver could not complete the lookup, perhaps because of an upstream failure or DNSSEC validation problem. REFUSED means a server declined to answer. Timeouts are not DNS answers at all. Resolvers may retry, briefly cache, or avoid caching these conditions according to their implementation; they are not the predictable negative cache described here.

How a resolver learns and stores a negative answer

A client usually asks a recursive resolver, the DNS service that performs lookups on the client’s behalf and keeps a cache. The resolver follows referrals until it reaches an authoritative server, which is a server responsible for the data in a DNS zone.

For a negative response, the authoritative server normally includes an SOA record in the response’s authority section. SOA, or Start of Authority, is the zone record containing administrative values such as the primary server, serial number, and timing fields.

The resolver uses the SOA data to determine how long the negative result may be cached. Under the rules described by RFC 2308, the negative TTL is the smaller of:

For example, imagine a response containing an SOA with an effective negative TTL of 300 seconds. A recursive resolver can retain the NXDOMAIN or NODATA result for up to five minutes. During that period, repeated clients using that resolver receive the cached negative answer without causing a new authoritative lookup.

The MINIMUM name can be confusing. In older DNS usage it was also associated with a default TTL for records, but for negative caching the relevant meaning is the maximum time to cache an authenticated absence. The SOA record’s own TTL also places a limit on it.

Resolvers reduce the remaining TTL as time passes. If a second client asks two minutes after the first lookup, it may receive the negative answer with roughly three minutes remaining. The exact presentation and internal behavior vary, but the cache entry has an expiration time rather than restarting its full lifetime for every client.

Why creating a record does not immediately fix every lookup

Suppose new-service.example.com does not exist. A resolver asks the authoritative server, receives NXDOMAIN, and caches it for one hour. Ten minutes later, you create an address record for that name.

The authoritative server now answers positively, but the resolver does not know that yet. It sees a valid negative cache entry and returns NXDOMAIN until the entry expires. Only then does it ask again and discover the new record.

This is why “DNS propagation” is often misleading. There is no single global cache that must update. Many recursive resolvers, operating for different networks and users, may have made the failed lookup at different times. Some have no negative entry and see the new record immediately; others are still counting down from their own negative response.

A practical upper bound is the configured negative TTL, measured from the most recent time a resolver cached the negative answer. It is not necessarily measured from the moment you changed the zone. If clients or intermediate resolvers perform a fresh negative lookup before the record is created, their timers can differ.

What happens when an existing record changes

Negative caching mainly affects transitions from “absent” to “present.” Changes to an existing record are governed by the record’s ordinary positive TTL. If an A record has a TTL of 3,600 seconds, a resolver may keep the old address for up to an hour after you change it.

There can be both caches at once. A resolver might have a positive cache entry for one record type and a negative entry for another—for example, an A record exists while an AAAA lookup is cached as NODATA. Cache entries are keyed by the queried name and record type, among other details.

Operational consequences

Before creating a name that may already have been queried, set an appropriately short negative TTL in the zone’s SOA record and allow the old value to age out. Do this ahead of a planned migration; lowering the value does not instantly erase negative entries that resolvers already cached using the previous value.

When debugging, inspect the authoritative answer and several recursive resolvers separately. A command such as dig can show the response status, the returned SOA, and the remaining TTL, but the result depends on which server you ask. Check whether the failure is NXDOMAIN, NODATA, SERVFAIL, or a timeout, and identify whether the stale result is negative data or an old positive record.

Negative caching is therefore not a failure to notice a DNS change. It is a deliberate cache of a valid answer: this name, or this type of record at this name, was absent when the authoritative server was asked. The negative TTL defines how long that answer is allowed to remain useful after the zone has changed.

← All articles