Trueup Witness · a second instrument for AI agents
Your dashboard said OK.
Your agent crashed.
Trueup Witness checks your AI agent's records against what really happened, lights a DISAGREE warning inside neatlogs when they don't match, and fixes the tracing code that got it wrong.
Wrong records on real Claude and OpenAI Agents calls, before and after Trueup's fixes.
False alarms in 40 normal real AI calls.
Failure rate in neatlogs' daily email vs the real rate: 9 of 81 runs crashed.
Confirmed record errors across 3 tracing tools: neatlogs, Arize and Traceloop.
Fixes sent to Arize, Traceloop and neatlogs. Arize's was reviewed by their head of open source.
The problem
Nobody checks if the dashboard is right.
Teams watch their AI agents through a dashboard. The dashboard shows whatever the tracing code inside the app writes down. When that code gets it wrong, a crashed run is recorded as SUCCESS, and every chart, alert and report built on it is wrong too.
The email said 6.2% of runs failed. The real number was 11.1%.
We read all 81 runs back and checked each one. 9 had crashed; 4 of them were never counted. These were test runs we crashed on purpose. The email added up correctly; the records it was given were wrong.
The reply stopped halfway. The record said it went fine.
A real Claude reply was cut off mid-stream and the app raised an error. neatlogs' record still marked the AI call SUCCESS and listed the whole run as a success.
How it works
A second instrument, and a mechanic.
Planes warn pilots when two readings disagree. Trueup gives your agent the same thing, then repairs the faulty reading at its source.
Witness watches the real AI traffic
It reads the same connection your app uses to talk to the AI provider, and notes what really came back.
Compares it with the neatlogs record
Did the call succeed? Did the run crash? Was the reply recorded? Each reading is checked against the record.
Lights DISAGREE inside neatlogs
It writes a red TRACE DISAGREE step into the same record, where a neatlogs detection catches it, ready to raise an alert.
Fixes the tracing code
Trueup's agent finds the cause, writes the fix and its tests, proves it, and sends it to the people who maintain that code.
Built with
neatlogs, Entire and cfo.ai.
Every Trueup run and every demo run is recorded in neatlogs, and Witness's warning lands in the same record.
Every fix links to the Entire record of the session that made it, so you can see how it was built.
The business plan, built on measured costs: a fix costs about $0.52 in AI spend.
Install
Two lines in your agent.
Witness never changes what your app does, never sends prompt or reply text beyond what neatlogs already records, and turns off with TRUEUP_WITNESS=0.
import trueup.witness trueup.witness.watch() # optional: name one run of your agent with trueup.witness.run("order-1042"): ...
Crash it yourself.
Break a real AI shop bot with one click and watch whether the record tells the truth. Then flip on Trueup's fixes and try again.