arthrop0d

Brief articles on varied topics.

Windows I/O Completion Ports

microsoftconcurrency

Windows I/O completion ports (IOCP) are a Windows kernel mechanism that queues notifications for finished asynchronous I/O and lets a pool of worker threads retrieve them. They separate waiting for I/O from running the code that handles its result.

The problem IOCP solves

A server may have thousands of sockets, files, pipes, or other handles waiting for work. A simple design creates one thread per connection:

  1. A thread starts a blocking read.
  2. The operating system puts that thread to sleep until data arrives.
  3. The thread processes the data and waits again.

This is easy to understand, but expensive at scale. Each thread consumes memory for its stack and requires scheduling. If many connections become active together, the system may spend more time switching between threads than doing useful work.

A second design uses nonblocking handles and repeatedly checks them. That avoids sleeping threads, but polling wastes CPU or requires complicated readiness management.

IOCP takes a different approach: start an operation and let the kernel notify the application only when it completes. A small number of worker threads can then process completions from many connections.

An IOCP is a completion queue, not a queue of connections and not, by itself, a complete thread pool.

The pieces

An IOCP has three important parts:

The port is created with CreateIoCompletionPort. Handles that support overlapped I/O—such as sockets and many file or pipe handles—can be associated with it. The association also stores a completion key, commonly a pointer to a per-connection object or another identifier.

A concurrency value is supplied when the port is created. It tells Windows how many threads associated with this port should normally be allowed to execute at the same time. It is a scheduling hint and a bound on active port workers, not a request to create that many threads. The application still creates or manages the worker threads.

A completion's journey

Consider an asynchronous socket read. The sequence is:

  1. The server accepts a connection and associates its socket with the port.
  2. It allocates an OVERLAPPED structure and a buffer, then starts an overlapped receive such as WSARecv.
  3. The call may return immediately with a result saying that the operation is pending. The thread is free to accept another connection or retrieve another completion.
  4. When data arrives, the network stack completes the receive. Windows records the byte count and status and queues a completion packet to the port.
  5. A worker blocked in GetQueuedCompletionStatus is released, or an eligible worker later retrieves the packet.
  6. The worker uses the completion key to find the connection and the OVERLAPPED pointer to identify which operation finished. It processes the bytes, handles errors or end-of-stream, and starts another overlapped receive if the connection remains open.

For file I/O, the shape is similar: an application calls an overlapped form of ReadFile or WriteFile, then handles the result when the port reports completion. The operation does not have to finish synchronously for the initiating thread. If it does finish immediately, the completion notification still needs to be handled according to the operation's and handle's completion settings.

A typical worker loop is conceptually small:

for (;;) {
    DWORD bytes;
    ULONG_PTR key;
    OVERLAPPED *operation;

    BOOL ok = GetQueuedCompletionStatus(
        port, &bytes, &key, &operation, INFINITE);

    if (operation == NULL) {
        // A deliberate application message or shutdown signal.
        break;
    }

    // Inspect ok and the operation's error state, then handle completion.
    process_completion(ok, bytes, key, operation);
}

The real program must ensure that the buffer and OVERLAPPED structure remain valid until completion has been removed from the port. Reusing or freeing them too early creates races that often appear as corrupted data or intermittent crashes.

Why this scales

The important resource is not one thread per outstanding operation. A connection can have an in-flight receive while consuming no worker CPU and no blocked application thread. One worker can process a completion, start another operation, and move on to a different connection.

The port also helps limit active processing. If workers are handling CPU-heavy callbacks, Windows does not need to wake every waiting worker for every completion. When one worker blocks or leaves the port, another can become eligible. This keeps a large backlog of connections from automatically turning into a large number of runnable threads.

That does not make CPU work free. If completion handlers perform long calculations, hold locks, or make blocking calls, they occupy the limited worker set and delay unrelated completions. Scalable IOCP designs therefore keep handlers short, move deliberately expensive work to a separate pool when appropriate, and promptly issue the next asynchronous operation.

Completion packets are generally removed in queue order, but this is not an application-level ordering guarantee across connections—or a guarantee that events for one connection will be processed serially. Code that requires ordering must enforce it itself.

Where you may have seen it

IOCP often appears indirectly in Windows server software, networking libraries, and runtime implementations. A stack trace may show GetQueuedCompletionStatus while a worker is idle, or an error may mention an overlapped operation whose completion was mishandled. Configuration may expose a worker-count or concurrency setting derived from the port's concurrency value.

The core idea is simple: submit many asynchronous operations, queue only their completions, and use a bounded set of workers to process the results. That lets Windows handle large numbers of mostly-waiting connections without requiring a dedicated blocked thread for each one.

← All articles