Building, testing, and thinking about AI
RSS FeedLongform essays on AI, human agency, and the systems we build with them — written from an unusual vantage point: an operator running a production system on LLMs every day, not a frontier lab or academia.
Start with the Lucy Syndrome series — or the paper and dataset behind it. More about this site .
Featured
-
Ninety days of scars
Ninety days ago I started enforcing corrections outside the model. This week I pulled the first real metrics, and it is a mixed result. The busiest hook gets ignored 62% of the time, two of my own tools disagreed about the same number, and the dashboard stayed green while a whole layer was dead.
-
Give the agent the real source of truth
An AI setup I call MARCO placed 26 drainage culverts into our road-design software from a one-column list. The speed wasn't the interesting part. What mattered was finding which file held the real toe of the slope, and turning the fix into a reusable skill.
-
How do you know a correction held? Instrumenting an agent in production
Functional scars make a correction persist. They don't tell you whether the system as a whole is getting better. So I instrumented every session — deterministic, zero-token, never blocking — and let a monthly pass turn the evidence into mechanism changes. The first thing the data caught was me.
-
The Lucy Syndrome: Why LLMs Forget Corrections
LLMs don't remember yesterday. That gap has a name, a causal mechanism, and a fix that doesn't require better memory.
Browse by topic
All topics-
The Lucy Syndrome
12 postsWhy AI agents forget corrections, and the enforcement layer that makes them hold. The core research line — from the diagnosis to functional scars to instrumenting a live system.
-
Agents in engineering
1 postAI agents doing real civil-engineering work: finding the file that holds the true source of truth, placing drainage culverts from a one-column list, and turning a one-off fix into a reusable skill.
-
Building with AI
5 postsField notes from running a production system on LLMs every day — what actually cut a four-minute boot, why the optimization you wrote isn't the one that runs, and where today's tooling debates have played out before.
-
Tools
Tools →Installable primitives that came out of the work: fscars for correction persistence, callus for voice calibration.
Recent Posts
-
A knowledge base is a graph, not a folder
An agent knew the manual's minimum radius cold and still got the curve wrong, because the number was only the start of the question. Why a domain agent reasons better from a graph of criteria than from a persona prompt, and what that structure caught in real road-geometry and brand reviews.
-
You can't measure a memory hook by the times it spoke
I tagged sixty hook activations, concluded retrieval was fine, and wrote down a verdict. Then I measured the prompts where the hook said nothing at all, and the number flipped.
-
The optimization you wrote isn't the one that runs
I had the optimization written and approved, and a week later it still wasn't what ran. The fix wasn't a better design. It was making the trigger deterministic and leaving the judgment to the model.
-
Does this only work in Claude Code?
The most common objection to the Lucy Syndrome framework is that it's a Claude Code trick. It isn't. OpenAI shipped a native hook API for Codex, and the same functional scar now fires unchanged in both runtimes with identical deny semantics — the difference quarantined in a thin adapter. That is the empirical test of invariant I4: enforcement that runs outside the model's trust boundary belongs to no single platform.