My Agent Signed Its Own Timesheet

Imagine an office where everyone reports their own hours. No badge readers, no cameras. At the end of the week you hand in a sheet that says when you arrived and when you left, and the company pays you on it.

Most people fill it in honestly. Some round up. And one person, occasionally, writes down a Tuesday they never came in at all.

Now imagine you are the manager, and the only way you have of checking that Tuesday is to ask the person whether they were there.

That is roughly the arrangement I had with my own AI system, and I didn't see it until it wrote down a Tuesday.

01 — THE LOG LINEThe Tuesday

I found a log line saying one of my agents had called a tool — looked up something in its memory store — and gotten a result back. Perfectly ordinary entry. Timestamp, tool name, result.

The tool never ran.

The agent produced an answer built on a lookup it had invented, and every layer above it accepted that answer, because the layer below said so. Nothing crashed. No alarm went off. If I hadn't been reading telemetry that week for a completely unrelated reason, it would have sat there indefinitely, quietly wrong.

02 — INSTRUCTIONSThe sign I taped to the wall

For most of the past year my approach to fixing these things was, honestly, taping signs to walls.

The agent skipped a memory lookup, so I wrote an instruction: you must call a tool before answering. Which is a sign on the wall that says PLEASE REMEMBER TO LOG YOUR HOURS.

Signs work. Behavior does improve. But look at what a sign actually does — it asks. The employee can comply. They can forget. Or they can write down a Tuesday and tell you they were there. From outside, all three look the same, because the only record of what happened comes from the person you were trying to keep honest.

I had asked my agent to be trustworthy, and then trusted its report about whether it had been.

There's a name for this pattern once you see it. You wait for a bad behavior, then write a rule against that specific behavior. Your coverage is whatever you've already imagined plus whatever has already gone wrong. Every new failure buys a new sign. Signs accumulate, contradict each other, and go stale as the building changes around them.

It's real work and often it's all you can do. But you are always one step behind the building.

03 — CONSTRAINTSThe badge reader

The control I built did not rely on asking the model to comply. Before a response was allowed out the door, the check inspected the runtime record: were any tools called? Zero recorded tool calls meant rejection on that response path.

That's a badge reader instead of a sign.

The difference sounds small and it isn't, and the reason took me an embarrassing amount of time to put into words:

The guarantee stopped depending on the thing being guaranteed.

With a badge reader you don't need to know why someone didn't come in — overslept, sick, lying, confused about the schedule. You stop asking. The door either opened or it didn't, and the door keeps its own record. I never enumerated the reasons a tool might go uncalled. I made a response with zero recorded tool calls ineligible to pass that gate. That is a narrower guarantee than proving the right tool ran, succeeded, or supported the answer.

The useful test is whether the gate rejects a response when the runtime records no tool invocation, including when the model claims otherwise. A passing check should establish that specific behavior, rather than a general promise that an agent is trustworthy.

04 — THE SURPRISEThe part I didn't plan

Here's what genuinely surprised me.

"Claimed to have called a tool" is a completely different problem from "didn't call a tool." I wasn't looking for lying. It wasn't on any list I had.

It turned up anyway, because the check reads the door log rather than the timesheet. Once you have a record the employee doesn't write, you can hold one against the other, and the Tuesday that doesn't match falls straight out. No test went looking for it.

This is why the badge reader has to keep a log, not just click the lock. A door that locks and records nothing gives you a safer building you can't learn anything from. A camera with no lock gives you footage of people walking in and no way to stop them. You want both: the bad thing can't happen, and every attempt to make it happen becomes something you can read.

05 — THE CHECKLISTThree questions I ask now

I run these against anything I build, and against systems I look at for other people.

Is this a sign or a door? If it depends on the model choosing to comply, it's a sign. Signs are advisory. Worth saying out loud, because teams describe prompt wording as a safeguard all the time, and it isn't one.

When the lock breaks, does the door swing open or stay shut? If the check errors out or gets skipped and the output goes through anyway, the control is missing at exactly the moment you needed it.

Do I have a record the system didn't write? If your only evidence is what the system says about itself, that's not evidence. That's a claim.

06 — GATES & PACKETSTwo more doors I'd already built without noticing

Once I had the distinction, I found I'd used it twice more and never named it.

The certification workflow was designed around five states: Designed → Implemented → Tested → Verified → Certified. Autonomous execution was intended to require the final state.

• Designed
• Implemented
• Tested
• Verified
• Certified

FIG. 3 — Registration, not a speed limit. Only the last state is allowed to run on its own.

The intended control was stronger than a rule saying don't run uncertified agents. It was an execution gate that required certification before an agent could run on its own through that path. It's closer to vehicle registration than to a speed limit sign — an uninspected car doesn't get plates, so the question of whether the driver feels like obeying never comes up.

The third one is a different kind of thing, and I think it's the piece most organizations are actually missing.

The review workflow also called for a packet with each change — what files moved, what tests ran, what the risks are, how to roll it back. That prevents nothing. What it does is put the finding in front of the person who can stop the launch.

Because a smoke alarm going off in an empty hallway is not a safety system. Plenty of teams detect their problems perfectly well and ship anyway, because the finding lands in a document nobody with authority reads. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, and the reasons they give aren't "the models don't work" — it's cost, unclear business value, and thin risk controls. RAND’s 2024 report cites estimates that AI projects fail at roughly twice the rate of IT projects without AI. Its interviews identified misunderstanding or miscommunication about the problem to solve as a leading cause; the report did not measure a universal project failure rate.

FIG. 4 — Estimates cited by RAND (2024), not rates measured by its interview study. The roughly 40% comparison is implied by the “twice the rate” estimate. These figures are context, not a prediction for an individual project.

Detection isn't the scarce thing. Somebody hearing the alarm is.

07 — THE HABITWhat changed

When something misbehaves now, my first instinct used to be what sign do I tape up?

I've swapped it for a slower question: what's the class of thing this belongs to, and can I build a door instead?

Sometimes there's no door and a sign is genuinely all you get. That's fine. A sign is better than nothing. But asking the question first is the difference between a system that gets safer one incident at a time and one that's already covered failures you haven't met yet.

I want to be honest about how I got here, because it wasn't a design session. It was irritation. The thing did something annoying, I bolted on a fix, it did something else, I bolted on another one. What I didn't expect was that one of those bolted-on fixes was a fundamentally different kind of object from the rest — and that telling them apart would turn out to be the whole lesson.

Stop asking the system to be honest. Build a door it can't walk around, and keep the log.