arthrop0d

Brief articles on varied topics.

Linux Page-Cache Readahead

linuxoperating systemsperformance

Linux page-cache readahead is a prediction mechanism: when a process reads a file sequentially, the kernel fetches nearby file pages before the process asks for them. Later reads can then be served from memory instead of waiting for storage.

The problem readahead solves

A file read normally involves two layers of work. The process asks for bytes, and the kernel must find those bytes in the page cache, the part of memory where Linux keeps recently used file contents. If the requested page is not cached, Linux must obtain it from the filesystem and storage device before read() can return.

Without readahead, a program reading a large file in small pieces could repeatedly pay storage latency:

  1. Ask for the next piece.
  2. Wait for the storage device.
  3. Copy the data to the process.
  4. Repeat.

That is particularly wasteful for sequential access. Once the process has asked for bytes at one file offset, the next bytes are often predictable. Linux can use the idle time while the process consumes the current data to fetch the next pages.

Readahead does not replace the page cache. It populates the page cache speculatively.

What happens during a sequential read

Imagine a process reading a file from offset zero onward. The first request misses in the page cache. Linux brings in the page containing the requested bytes, but it may also submit I/O for several following pages. The requested page is synchronous work: the process needs it before the read can finish. The following pages are asynchronous work: they are a prediction about what the process will need next.

When the process asks for the next range, those pages may already be in memory. The read becomes a cache hit, avoiding another storage wait. Linux can continue issuing I/O ahead of the process, keeping a moving buffer of likely future pages.

A page is the fixed-size unit Linux uses for virtual memory and much file-cache management; its size is commonly 4 KiB, though the exact size depends on the architecture and configuration. A read request can cover many pages, and readahead operates around those page boundaries rather than predicting individual application-level records.

The prediction is based primarily on observed file offsets. If successive reads advance through nearby, increasing offsets, Linux treats that as evidence of a sequential stream. It does not understand that the bytes are database rows, log records, or video frames.

Readahead predicts where the next file access will be, not what the file means.

Linux maintains readahead state associated with an open file access stream. That state includes a current window of pages to fetch and information about the previous access. As sequential behavior continues, the kernel can grow the window, up to limits imposed by the filesystem, device, memory pressure, and other policy. The goal is to keep enough data in flight to hide storage latency without fetching an unreasonable amount of unused data.

The process does not need to call a special function for ordinary buffered reads. The normal read() path can trigger readahead automatically. This is why a program can see good sequential throughput even when its code only makes straightforward file reads.

How Linux notices a bad prediction

Speculation has a cost. A prefetched page consumes memory and storage bandwidth. If the process never uses it, that work displaced something else from the cache and competed with useful I/O.

Linux therefore checks whether the process actually consumes the pages it fetched. Several patterns can weaken or reset the sequential assumption:

For a new position, Linux can start a new prediction rather than treating the jump as the next step in the old stream. For irregular access, it reduces or avoids speculative fetching. The exact decisions vary across kernel versions, filesystems, and storage types, but the principle is stable: successful sequential use justifies more lookahead; unused lookahead makes the policy more conservative.

This adaptation is not instantaneous omniscience. A workload that alternates between sequential phases and random jumps may still cause some unnecessary reads while the kernel learns what is happening. Conversely, a genuinely sequential workload can have an initially small window before the kernel has enough evidence to expand it.

Readahead and random access

Random access is not automatically bad, but it provides little evidence that nearby pages will be needed next. If a program reads page 10, then page 900, then page 42, fetching pages around each request is mostly wasted. In that case, the useful operation is usually to load the requested page and avoid a broad forward window.

Applications can also provide hints. For example, posix_fadvise() supports POSIX_FADV_SEQUENTIAL and POSIX_FADV_RANDOM, allowing a program to describe an expected access pattern. Memory mappings have corresponding advice through madvise(), including MADV_SEQUENTIAL and MADV_RANDOM. These are hints, not guarantees: they influence policy, while the kernel still manages memory and I/O.

An application can encounter the opposite issue when it bypasses the page cache. Direct I/O, commonly requested with O_DIRECT, is designed to transfer data without normal page-cache buffering and therefore does not use ordinary page-cache readahead in the same way. Programs using it generally take responsibility for their own I/O strategy.

Why this appears in incidents

Readahead often explains why sequential scans have high throughput while random lookups have high latency, even on the same disk. It can also explain unexpectedly high read bandwidth, cache pressure, or storage traffic: the kernel may be reading pages that the application has not requested yet.

The important operational distinction is useful readahead versus wasted readahead. A large sequential scan can benefit substantially from pages fetched early. A workload with sparse reads, many competing streams, or frequent seeks may pay for predictions that never become useful. Linux continuously adjusts the prediction, but it cannot eliminate the fundamental trade-off: looking farther ahead can hide more latency when the guess is right and waste more resources when it is wrong.

← All articles