technologies · difficulty ◆◆

Harness engineering — the craft of building around a model

An LLM alone is an engine bolted to a concrete floor. The harness is what makes it a car.

A raw LLM with no wiring around it is close to useless — an engine in a crate. Everything that turns that engine into something that runs your business, reviews your PRs, and preps your calls is the harness. And like a car, some people buy it off the lot, some tune it like an F1 machine, and some build it from scratch.

AI-generated draft · reviewed and edited

2026-09-08 · 13 min read

$ harness

4
the lever count: Nov 2024
MCP, open-sourced: Feb 2025
Claude Code launched: Oct 2025
Claude Skills shipped: 26k+ stars
Karpathy autoresearch

What it is

Harness engineering is the practice of building the system around an AI model that makes it useful. The model is the brain; the harness is everything else — the prompt scaffolding, the memory, the tools it can reach, the automation that runs it, and the tests that check its work. The term became popular through Anthropic’s agentic stack, but the idea is tool-agnostic: it applies to Claude Code, ChatGPT, Codex, Cursor, or any agent you wire up.

Why it matters

The failure mode of a bare model is that it can answer but not act, and it forgets everything the moment you close the window. You prompt it with vague instructions, it has no way to reach your calendar or your repo, and every useful thing you got from it was luck. Harness engineering is the fix: you give the agent the context it needs, the tools to act, the automations to run without you, and the evals to know whether its output is any good. The delta between a model and a harness is the difference between a toy and a tool you ship with.

The three levels

There are three tiers, set by where you start. Off-the-shelf apps like ChatGPT and Claude — you tweak the trim: system instructions, memory, a few connectors, your files. Next, Claude Code and its kin — you are tuning a racing car, editing config, writing skills, wiring hooks, and you need real expertise to drive it at speed. At the top, you build your own harness with an API and custom code around the model — total freedom, total responsibility. Most people will live in tier one and two; the skill is understanding which tier you are in and pulling the right levers.

Example

$ # context: CLAUDE.md
# loaded into every conversation, automatically
> Every time you use a skill, end the session with
> one concrete suggestion for improving that skill.

# This single line taught the agent to never let a skill go stale.
# Claude reads CLAUDE.md at the start of every session,
# so the instruction rides along without being re-pasted.

CLAUDE.md (or AGENTS.md for other tools) is the simplest context lever: anything in it is auto-loaded every session. Shaw’s example — one line telling the agent to propose skill improvements — is a self-maintaining harness.

$ # tools: connectors via MCP
# connect Gmail + Google Calendar + HubSpot, then:
> Rank my warmest and coldest leads from the last 30 days.
> Check sales calls, CRM notes, and email threads.

# The agent reads across three sources and answers
# without you exporting anything or pasting a single row.
# This is the difference between context and tools: the agent
# goes and GETS it instead of waiting for you to hand it over.

This is Shaw’s lead-ranking example. Connectors (Claude’s term) are MCP servers under the hood — a standard Anthropic open-sourced in Nov 2024 so any assistant can reach external apps through the same protocol.

$ # automations: scheduled + event-triggered
/schedule "every morning: prep today’s calls"
# git pre-push hook fires Claude Code to review before push

# Scheduled: a daily prep task pulls meeting data, runs web
# search, checks email, writes one Notion page.
# Event: a git hook / Claude hook fires on specific triggers
# e.g. delete the screenshots an agent left in the repo.

These are Shaw's two automation patterns. `/schedule` is time-based — it creates a cloud routine (daily call prep, nightly code review) that runs with a minimum interval of an hour and does not need your machine to be on; `/loop` is the in-session equivalent, and hooks are event-based (on session exit, on push, on any trigger). Hooks only live in the coding agents, not in off-the-shelf chat apps.

Common flags

CLAUDE.md / AGENTS.md: auto-loaded context file — guidance, standards, knowledge, injected every session
skills: a folder of files with a SKILL.md prompt that loads only when relevant (progressive disclosure)
MCP / connectors: standard for giving the agent tools to reach external apps: Gmail, Notion, Google Drive, Slack, Jira, HubSpot
/schedule: time-triggered automation: run a task every morning, every night, on a cron
hooks: event-triggered automation: on session exit, on config change, on push — only in coding agents
evals: tests that tell the agent whether its output is good — the loop that removes you from the review

History

Where the term came from

The word "harness" for the system around a model comes out of the agent-tooling world: it names the layer that wraps a model and lets it act on a repository, and it is used loosely enough that one 2026 paper calls it polysemous. Two engineering posts put it in wide circulation — OpenAI's "Harness engineering: leveraging Codex in an agent-first world" (February 2026), written up from a five-month experiment shipping a product with no hand-written code, and Anthropic's writing on harness design for long-running apps. Shaw Talebi's video "Harness Engineering Explained in 22 Minutes" (July 2026) is a clean public codification of the idea, framing it as the car-to-engine analogy and naming the levers: context, tools, automations — then the bonus lever, evals. The term spread because it names something every power user was already doing but had no word for.

Anthropic’s building blocks in sequence

The stack assembled piece by piece. The Model Context Protocol was open-sourced in November 2024, giving agents a standard way to reach external tools. Claude Code launched as a research preview in February 2025 and went fully general availability in May. Claude Skills arrived in October 2025, bringing the progressive-disclosure pattern — the agent sees the name and description of every skill, but only loads the actual files when a task needs them. The timeline matters because each release moved a lever (context, tools) from manual to standard.

Karpathy’s autoresearch — the evals proof

Andrej Karpathy's autoresearch repo went viral as the proof of concept for evals. Instead of editing train.py himself and iterating by hand, he wrote a program.md file — instructions for an agent to optimize train.py autonomously — and left it running overnight. In Karpathy's own report, it came back with about twenty changes that improved the validation loss, all additive, all of which transferred to a larger depth-24 model. The repo pulled in roughly 26,000 stars in under a week. It is the cleanest public demonstration that building an eval loop beats being the loop.

Under the hood

The two loops: human-in-the-loop vs agent-in-the-loop

Without evals, every interaction is a human loop: you send a request, the agent produces output, you judge it, you send feedback, repeat until satisfied. That caps your throughput at your attention span. Evals replace your judgment for checkable outputs with an automated test. The agent produces output, runs the eval itself, fixes, re-runs, and only hands the result to you once it passes. You review final results rather than every intermediate draft. The loop changes from "you review everything" to "you review what passes the test."

Progressive disclosure — context without the baggage

The genius of skills is that they solve the context-budget problem automatically. Instead of loading every reference, asset, and script into every conversation (a firehose that burns your context window), the agent only sees the name and description of each installed skill up front. When a task matches, it pulls the SKILL.md and the specific files it needs. Your proposal skill’s logo in the assets folder, your Keynote-editing script, your API connector — all available, none of it cluttering the context until it is relevant.

Agent empathy — the design heuristic

Shaw’s closing mental model is worth stealing: before you ship a harness, put yourself in the agent’s shoes. Given the context you provided, the tools you exposed, and the automations you set up, could you complete the task? If yes, the harness is probably good. If no, you know exactly what to improve — more specific context, a missing tool, a hook you forgot. It shifts the question from "how do I do this task" to "how do I make the agent able to do this task, reliably." That is the whole mindset shift of harness engineering.

Fun facts

Pros & cons

pros

+ Turns a vague model into a reliable worker that can reach your data and act on it
+ Skills and context files move knowledge out of your head and into a reusable, versionable form
+ Automations run while you sleep — daily prep, nightly review, scheduled tasks
+ Evals remove you from the loop for checkable outputs, scaling your throughput
+ External context (Notion, Gmail, Drive) means one memory reused across every AI tool
+ Progressive disclosure keeps context windows lean instead of firehosing them

cons

− The deeper you go, the more expertise and maintenance every lever demands
− Hooks and full harness engineering only exist in coding agents, not chat apps
− Garbage in, garbage out — evals only help if your checks actually measure quality
− A poorly tuned harness can hand an agent powerful access it will misuse (and you will not notice)
− The tooling moves fast; a setup that works in February can be outdated by mid-summer

Takeaways

1Map your starting point: off-the-shelf app, coding agent, or fully custom harness — and pull the levers available at that tier
2Start with context: a CLAUDE.md / AGENTS.md file and skills beat hand-pasting instructions into every prompt
3Move knowledge into external tools you can point every assistant at — one source of truth, reused everywhere
4Add tools via MCP/connectors before you write any code; reach apps before you build custom wrappers
5Wire one automation (a schedule or a hook) and one eval, then apply agent empathy: would you succeed with exactly this harness?

Related commands

← all learning