Blog

Field note · Analysis

Build a Local AI Agent With an Audit Trail

A proposed build for people who set up AI agents: builders, researchers, and engineers who deploy AI at customer sites.

  • 11 min read

Michael Turon

In short

To build a local AI agent you can audit, run the model on your own machine, give each agent its own workspace with only the tools its job needs, and record every run in a self-hosted trace store. This is a proposed build. We have not yet run it end to end.

CARE Institute studies where an AI agent may send each request. Our three steps are classify, route, and audit. This post covers the audit step for one person on one computer: how to run an agent locally and keep a record of what it did.

An agent can read a file, plan a task, use tools, and report what it did. That does not mean it should have your whole computer. A useful agent needs a second design that limits its reach and makes its work easy to review.

This blueprint puts the model and the core agent software on your own machine. It adds separate workspaces, clear tool permissions, local traces, and a repeatable test set. A trace is a saved record of one run. Internet access is available when a task needs it, but the core should also run without it. Any cloud-model call is a separate choice that a person approves.

The target is enterprise-lite: enough control to test, recover, and improve on one computer, with clear paths to a team deployment. This is a proposed build. We have not yet run the full stack through a live deployment test.

Many teams face the same tradeoff. A lab wants to keep study data close. A small business wants an assistant that drafts mail but does not send it on its own. A home lab builder wants to see which files an agent touched. A forward deployed engineer, who sets up AI systems at a customer’s site, has to make all of that work in a real setting.

Treat the agent as a product. Define the job, the data it may use, the actions it may take, and the evidence you need before you trust it with more.

What does “local, without an air gap” mean?

Link

A local AI agent runs its model and core workflow on hardware you control. “Without an air gap” means the host may also connect to a network. Both can be true, but a local agent is not a promise that every action stays local.

This build uses three modes:

  • Offline mode: the agent works with cached models and local files while wide-area network access is blocked.
  • Connected mode: the agent may use approved online sources or services, and model inference stays local. Inference is the step where the model answers a request.
  • Cloud escalation mode: a person approves a named request before any data goes to an approved cloud model.

A local model can still call a web tool. A local trace can still hold private data. A local service can still be exposed on a network. Make each path visible, then test it.

This follows CARE Institute’s research focus: understand how sensitive the task is, choose where it should run, and make the result testable.

Why not just use a cloud model?

Link

A hosted model may give better answers than a model on your own machine. Many teams also use cloud services under contract, and that can be a valid rule. CARE does not treat cloud use as forbidden.

A local build is for teams that want more control over data and access. The audit trail in this post matters either way. A reviewer needs to see which route each request took, and why.

The five layers of the local agent stack

Link

The first target is one user on one Mac Studio. The model engine runs on the Mac. Agent workers and the trace service run in containers (isolated packages of software) on the same Mac. A network storage device holds the code repository and backups.

Architecture of the local agent stack: an operator, three agents, a local model server, a trace service, and code with backups.
    • Operator
    • Lab results agent
    • Handoff notes agent
    • Drug reference agent
    • Local model serveroMLX on Mac
    • Local trace serviceLangfuse on Mac
    • Forgejo + backupson network storage
LayerFirst-build role
1. Agent and orchestrationOpenCode runs separate agent profiles and jobs.
2. Routing and decisionsRules decide what stays local, what may use approved network access, and what may escalate to the cloud.
3. Models and inferenceoMLX serves a selected local model on Apple silicon.
4. Tools and executionEach worker gets only the tools, files, and network access its job needs.
5. Memory and recoverySession records, traces, versioned code, and tested backups preserve evidence and support recovery.

Start with three agents. This post uses a healthcare example, taken from the nurse scenario in The Privacy-Routing Problem: a lab results agent, a handoff notes agent, and a drug reference agent. Their data differs. The first two handle patient details, so they use only local files and the local model. The third answers textbook questions with no patient data, so it may use approved online tools. Give each its own workspace, session store, and tool policy. They can share the model engine without sharing all of their files or credentials. In a first build, all three use synthetic records.

The same design can run on other hardware, but each host needs its own memory, network, and image checks. On a Mac, the proposed inference server is oMLX, which exposes model APIs for Apple silicon. Pick a model that fits the machine, and prove it can complete tool calls, not just answer a chat prompt.

How do you give each agent a small, testable workspace?

Link

OpenCode is the agent interface and worker. Point it at the local model endpoint and create a profile for each job. A profile sets roles and rules, but it is not a hard security boundary on its own. The boundary you test must also include mounts, credentials, network rules, and process isolation.

Give each worker the least access that lets a test succeed. In the first pass, each worker reads only its own synthetic files, with shell commands, outside paths, web access, and writes all denied. Add a tool only when a job needs it, and test both the allowed and the denied actions.

Keep the model engine and trace service off the public internet, with authentication, restricted ports, and scoped secrets. Do not mount the host home folder, the container control socket, or storage admin shares into an agent worker. Containers reduce exposure, but they do not make untrusted code safe by themselves.

What should an AI agent audit trail record?

Link

An audit trail records what happened during each run: the agent, model version, task ID, tool call, result, timing, and final status. It also exposes gaps, such as a failed tool call, a slow request, or a source the agent never checked. A chat history alone does not give you this.

The proposed trace store is self-hosted Langfuse. Installing it creates no traces. You have to wire the worker events or an adapter to the trace system, then read back a known run. If the trace service goes down, the loss must be visible. An in-memory queue is not a durable audit log.

Traces can hold sensitive prompts, patient details, legal files, or email text. Begin with synthetic data and metadata. Decide what to redact, who may read the records, how long to keep them, and how to delete them, and test those rules before live data goes in.

Four records do different jobs, and none replaces another:

  • Session history lets a person inspect a chat.
  • Job state shows which step can resume.
  • Git holds versioned code and settings.
  • Backups let you recover after a loss.

A network storage device can host Forgejo for code, with a separate backup store. By default, do not commit keys, raw traces, private files, or model weights. Back up databases and files in a consistent state, restore them to a test location, and check the result.

How do you test and improve an agent with traces?

Link

Build a small test set before you ask the agent to improve itself. Include tasks with known answers, tasks where access must be denied, and cases where a file contains misleading instructions. Save the model, prompt, tool policy, and test versions with each result.

For each change, run the same tests and compare. Track task success, tool failures, unsupported claims, time, and rule violations. Review failures, propose a change, then rerun both the affected tests and a held-out set. A person approves promotion. Logs alone do not make an agent learn, and a model grading its own work does not prove quality.

A practical path to the first working agent

Link

Use one model, one worker, and one synthetic job. Record the machine’s memory, free storage, software versions, model source, and model ID. Leave enough headroom for the host, the model, and the containers to run together.

  1. Check the model server. Start it and verify its health and access rules. Send a plain request, a streamed request, and a tool-call request. A passing tool test needs a real tool response followed by a final answer that uses it. Reject a model that only prints something that looks like a tool call.
  2. Run the worker on a test file. The file holds a fresh receipt. The worker must read it through a tool and return the receipt in its final answer. Then try to read a second worker’s file, run a shell command, and write outside the workspace. Each should fail under the first policy.
  3. Send traces. Send a known trace plus a passing and a failing test result to the local trace store, and read them back from the server. Restart the services and confirm the records remain. Stop the trace service and confirm that a missing record or delivery failure is visible.
  4. Go offline. Cache what the job needs and block wide-area network access, leaving local network paths available. Run the test again. The job should finish without cloud calls. Online lookups wait. Record what passed, what failed, and which versions you tested.

These gates are a foundation. They do not show that every business workflow is ready. Live records systems, online research, and scheduling each need their own adapters, access rules, approval checks, and tests.

Which jobs fit a first agent, and where do people approve?

  • Lab results: the agent reads synthetic results and flags values for review. A person decides what to act on.
  • Handoff notes: the agent drafts. A person reviews and approves before anything enters a record.
  • Drug reference: the agent answers textbook questions with no patient data. Record the source and date of each online lookup.
  • Legal research, a second end market: record the source, date, and citation, and have a person check the law.

How do you move from one machine to a team?

Link

At first, keep the model and workers together. That is simpler to test and has fewer network paths, but it puts pressure on the Mac’s memory. Self-hosted tracing has a real cost too, because the service needs several data stores. Measure it alongside the model instead of assuming it will fit. Langfuse’s deployment guidance describes the components involved.

Later, you can move the workers and trace service to a network storage server or another host while the model stays on the Mac. That frees Mac resources and adds a network dependency. Check the new host’s capacity, image support, storage, encryption, and recovery path first. If the model host goes down, pause the job. Do not silently send private data to a cloud model.

A team deployment adds identity, roles, isolated workspaces, per-user credentials, queue limits, retention rules, and named owners. Add them as measured need appears. Test one user’s data boundary against another’s, and test how the system recovers from a failed host.

What we don’t know yet

Link

This build has not been run end to end, so these questions are open:

  • Whether the full stack fits on one Mac Studio. Tracing needs several data stores, and we have not measured them next to a running model.
  • Whether the chosen local model completes tool calls well enough for the lab results, handoff notes, and drug reference jobs.
  • Which trace fields need redaction, and how long each kind of record should be kept.
  • How the design holds up when it moves from one machine to a team.

We will report results once the gates above have been run and the versions recorded.

Frequently asked questions

Link

Can a local AI agent use the internet?

Yes. A local model can use approved online tools. Control and log the network path and the model path as separate choices.

Is a local AI agent air-gapped?

No. This design works offline for supported local jobs and also allows controlled network access. Run an offline test before claiming a job works without the internet.

Does self-hosted mean secure?

No. Local control gives you more choices about data and access, and it makes you responsible for updates, secrets, permissions, backups, and testing.

What is an audit trail for an AI agent?

It is a record of each run: the agent, model version, task ID, tool calls, results, timing, and final status. It lets a reviewer see what the agent did and where it failed.

Do I need several models or a workflow engine on day one?

No. Start with one model and prove the job. Add model routing or a workflow engine when a measured need, such as durable multi-step work, justifies the cost.

A first agent needs a clear job, a narrow workspace, and a way to show its work. Give it those, test the limits, and keep the results.