arthrop0d

Brief articles on varied topics.

Hybrid Logical Clocks

distributed systemsalgorithms

Hybrid logical clocks (HLCs) are timestamps that combine a machine’s physical clock with a logical counter. They let distributed systems order events in a way that respects causality, even when machines’ clocks are not perfectly synchronized.

The problem: time is not one thing in a distributed system

On one machine, event order is usually clear: the program executes one instruction or operation before another. Across machines, there is no shared sequence of execution. Two servers can process requests at nearly the same time, and network delay can make the order in which they observe those requests misleading.

Each machine does have a physical clock: its estimate of real-world time, usually expressed as milliseconds or microseconds since an epoch. But physical clocks drift and are adjusted. One server may be several milliseconds ahead of another. A message sent earlier can therefore receive a later-looking timestamp than a message sent afterward.

A logical clock avoids this problem by recording ordering rather than pretending to know the exact time. It advances when events occur and when messages are received. However, a purely logical clock says little about when an event happened in real time. That makes it less useful for finding recent data, comparing events with wall-clock deadlines, or inspecting logs.

An HLC combines both properties:

The result is an ordering that is causally correct while remaining approximately meaningful as a timestamp.

The two components

An HLC timestamp is commonly represented as a pair (l, c):

Pairs are compared lexicographically: (10, 2) is earlier than (10, 5), and (10, 5) is earlier than (11, 0).

The first component is not necessarily the exact reading of the machine’s clock. Once a server receives a message from a machine whose timestamp is ahead, its l component may jump forward. This is deliberate: it prevents a later event from receiving a timestamp that looks earlier than the event that caused it.

How an HLC advances

Suppose a process has current HLC (l, c) and reads its physical clock as p.

For a local event or a message being sent, it first compares p with l:

In simplified pseudocode:


def local_event(l, c, physical_time):
    if physical_time > l:
        return physical_time, 0
    return l, c + 1

The counter is what distinguishes events that happen during the same physical-time tick. It also handles a physical clock that has not advanced yet.

Receiving a message requires combining three values: the local physical reading p, the local HLC (l, c), and the message’s HLC (lm, cm). The new logical time is the greatest of the three physical-time components:

l_new = max(physical_time, l, lm)

The counter is then chosen according to which value produced l_new:

The receive event gets this new timestamp. Future events on that process start from it.

Why this preserves causality

Causality means that one event influenced another. A message send causally precedes the corresponding receive, and if one operation happens before another in the same process, it also causally precedes it. This relationship is often called the happens-before order.

When a process receives a message, it incorporates the message’s HLC before assigning timestamps to subsequent events. Therefore, an event caused by that message cannot receive a timestamp smaller than the message’s timestamp. More generally, if event A happens before event B, the HLC for A is less than the HLC for B under lexicographic comparison.

For example:

Event HLC
Server A writes a record (1,000, 0)
Server B receives the write notification (1,000, 1)
Server B updates an index (1,000, 2)

Even if Server B’s physical clock still reads 999, its HLC will not move backward below the received event.

Two events that are concurrent—neither caused the other—can still receive ordered HLC values. That ordering is not evidence that one really happened first; it is just a deterministic or implementation-dependent sequence. Systems that need a total order often compare an additional node identifier after the HLC, such as (l, c, node_id). The identifier breaks ties but does not add causal information.

What HLCs are useful for

HLCs commonly appear in distributed databases, replicated logs, version metadata, and conflict-resolution mechanisms. They can help determine whether one version is causally newer, produce stable ordering keys, and make timestamps in logs easier to relate to real time.

They are often the explanation behind fields or errors mentioning a “logical timestamp,” “hybrid time,” or a timestamp that is unexpectedly ahead of the local clock. That future-looking value usually means the process observed another process whose clock or HLC was ahead—not that the system has discovered the exact future.

An HLC does not make physical clocks accurate, provide a perfectly synchronized global clock, or reveal the true order of concurrent events. Its guarantee is narrower and useful: it preserves known causal order while keeping timestamps close to physical time. Implementations must also account for clock adjustments and counter growth, typically by using a monotonic clock source or refusing to let the physical component move backward.

← All articles