arthrop0d

Brief articles on varied topics.

The AI Agent Tool-Calling Loop

ai agentsartificial intelligenceapi design

An AI agent’s tool-calling loop is a repeated exchange in which a model requests an external operation, the host application validates and performs it, and the operation’s result is sent back to the model as new context. The model chooses what it wants done; the host decides whether and how it is allowed to happen.

The problem tool calling solves

A language model can generate text, but its learned knowledge is not a live connection to your database, filesystem, ticket system, or payment provider. Without tool calling, an application has to guess when the model wants an operation, extract an instruction from prose, execute it, and paste the result back into a prompt.

That approach is fragile. The model might write “look up order 4815” in a paragraph, use invalid arguments, or mix an instruction with an answer. A tool-calling interface gives the request a structured shape: a named tool plus arguments that should conform to a declared schema.

The important boundary is this:

A tool call is a proposal from the model, not permission to perform an operation.

The host application remains responsible for validation, authorization, side effects, timeouts, and error handling.

Step 1: Declare the available tools

Before asking the model to handle a task, the host sends tool definitions along with the conversation. A definition normally includes a name, a description, and a JSON Schema-like description of its arguments.

For example, the host might declare a tool named get_order whose required argument is a string called order_id. The description should explain the tool’s purpose and important limits. It is guidance for the model, not a security mechanism.

The host also supplies the user’s request and any prior messages. The model then has two broad choices:

A tool call commonly contains a unique call ID, a tool name, and arguments encoded as JSON. The exact wire format varies between model APIs, but the roles are consistent.

Step 2: Receive and inspect the request

Suppose the model decides it needs the order record. The response may conceptually look like this:

{
  "role": "assistant",
  "tool_calls": [
    {
      "id": "call_7f3",
      "name": "get_order",
      "arguments": {
        "order_id": "4815"
      }
    }
  ]
}

The host must parse the response and treat every field as untrusted input. It should check at least:

Schema validation catches malformed types and missing fields, but it does not establish authorization. An argument can be perfectly valid and still be forbidden. For example, account_id: "someone-else" may match the schema while violating access control.

For sensitive actions, the host may require a separate approval step. Sending email, deleting data, issuing a refund, or deploying code should not become automatic merely because a model requested it.

Step 3: Execute outside the model

After validation and policy checks, ordinary application code dispatches the request to the implementation. The model does not call the database or receive network credentials directly.

A provider-neutral host loop looks like this:

while True:
    response = model.generate(messages, tools=tool_definitions)
    messages.append(response.message)

    if not response.message.tool_calls:
        return response.message.text

    for call in response.message.tool_calls:
        result = dispatch_checked_call(call)
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": serialize(result),
        })

This is illustrative control flow, not a promise that a particular model SDK uses these method names. In production, dispatch_checked_call should also enforce deadlines, retry rules, output-size limits, logging, and idempotency for operations that can be repeated.

If execution fails, the host usually returns a structured error or a safe error message as the tool result. It should not expose credentials, stack traces, or unrelated internal data. The model can then explain the failure, ask for missing information, or choose another tool.

Step 4: Return the result as model context

The host sends the tool result back in a new message associated with the original call ID:

{
  "role": "tool",
  "tool_call_id": "call_7f3",
  "content": {
    "status": "shipped",
    "tracking_number": "ZX-2049"
  }
}

The model sees this result together with the earlier conversation and the assistant message that requested the tool. It can now produce a final answer, or request another tool. For example, it might use the tracking number to call a get_tracking_status tool.

This is why tool calling is a loop, not a single model request:

  1. The host provides conversation and tool definitions.
  2. The model returns text or tool-call requests.
  3. The host validates and authorizes each request.
  4. The host executes approved requests.
  5. The host appends the results to the conversation.
  6. The model receives the expanded context and continues.

The loop ends when the model returns an ordinary answer, or when the host stops it because of a timeout, iteration limit, token budget, policy violation, or execution failure.

What to look for in logs

Once you know this shape, familiar operational details become easier to interpret. A log mentioning tool_calls, a tool_call_id, an “invalid tool arguments” error, or a missing tool-result message is describing one stage of this exchange.

A robust implementation records the requested tool, validation outcome, authorization decision, execution duration, and result status—while redacting sensitive arguments and outputs. The model supplies useful intent, but the host is the control plane that makes that intent safe and real.

← All articles