The explanation is the part everyone asks for. What it leaves behind is the part that pays off.
I’ve been using the phrase “always-on decision engineer” for a while now, and I’ve noticed that people hear a person. A contractor on call. A senior engineer with a pager and a good memory. That’s a reasonable reading of the words, and it’s the wrong picture. The useful thing about this job is what the work produces, and what’s still around once the work is done.
Two weeks ago I wrote about a Thursday at a home-services company: a booking agent put a burst pipe into a standard slot next Wednesday, and an operator and a developer spent the day working out why. By five o’clock they had something like an answer. It lived in a spreadsheet in someone’s downloads folder and a Slack thread that would scroll away by Monday. The work happened. Almost nothing from it survived.
So instead of describing the role in the abstract, I want to run that same Thursday again with the function in place, and look at what it does at each point, and at what it leaves behind.
It starts before anything looks wrong
The first part of the job happens on every run, including all the ones nobody will ever ask about.
It captures the decision as it was made. That means the customer’s message and the booking, yes, but also the context the agent was actually working from at that moment. On the Thursday, the single most important fact turned out to be a line in the intake guidelines that ops had edited two weeks earlier. Nobody thought of that edit as a change to the agent, so nothing connected it to the agent’s behavior. A decision engineer would treat that instruction change as part of the system, because to someone watching decisions, the instructions are part of the decision.
Then it assesses. Each decision gets held against what that kind of decision normally looks like for this agent: which urgency water-damage messages usually get, how often a same-day request gets pushed out, whether a pattern that was rare last month isn’t rare anymore. For most runs this produces nothing worth saying, and that’s the point.
The value of watching every run is that when a request with “burst” and “ceiling” in it lands in a standard slot, it can stand out against a background, instead of waiting for a customer to call twice.
Most of the job is deciding what not to look into
A good senior engineer doesn’t read every trace. They watch broadly, notice what’s off, and spend their real attention on the small number of things that deserve it. The always-on version works the same way, and I think that’s the most misunderstood part of it.
The tempting design is to reason deeply about every single decision: run a full analysis on every booking, every refund, every routing choice. It sounds thorough. In practice it’s expensive, and it produces so much commentary that the team learns to skim past it, which puts them right back where they started. Continuous assessment is cheap, and it runs on everything. Deeper reasoning comes in only where it creates value: where a decision stands out, matches something that went wrong before, or someone asks directly why the agent did what it did.
On the Thursday, that means the deep investigation happens on the handful of water-damage bookings that broke pattern after the guideline change, not on the forty rows the operator ended up searching for by hand.
An explanation is only as good as its evidence
This is where the Thursday actually fell down, and it’s the step most people skip when they picture the role.
The operator’s final note said: “We think a guideline change caused it.” That sentence isn’t wrong. The trouble is that “we think” is carrying the entire diagnosis, and nobody reading the note can tell what was known and what was guessed.
Part of the job is keeping those separate. Some things were observed directly: the customer’s exact message, the urgency the agent assigned, the booking it created. Some things were reported by the application: which version of the guidelines was in effect, what the scheduling tool returned. And some things have to be inferred: that the new line about “stopped problems” is what tipped a burst pipe into a standard slot. That inference might well be right. It should still be labeled an inference, with the evidence for it and the evidence it’s missing.
The agent itself is the least reliable witness here. If you asked it why it chose standard, it would give you a fluent answer. An agent’s account of its own behavior is one more output, so part of the job is showing what the record supports rather than collecting a better story from the agent.
None of this eliminates “we think.” It puts a boundary around it. The owner can then decide whether to keep the agent running on a clear picture: what’s established, what’s probable, and what nobody knows yet.
Then it explains, in the owner’s terms
On the Thursday, the explanation needed two halves that lived in two different heads. The operator knew what “urgent” means for this business. The developer knew what the agent reads and what changed in the code.
Those two things only met inside a Slack thread, for a few hours, on a day when both people had other jobs.
Doing this well means producing the explanation that normally only exists while both of those people are in the room, and producing it for whoever actually has to act on it. For the owner, that looks something like: after the guideline change on the 12th, the agent began treating water-damage requests as contained whenever the customer said they’d shut off the water. Three bookings since then look affected. Here’s what supports that, and here’s the part we’re still inferring. That’s the “why did it do that?” answer an owner can make a decision on, and it’s a different thing from being told the system is healthy.
What survives the work
Here’s the part I think matters most, and the part the Thursday lost completely.
When the investigation is finished, the function keeps the result. What decision was made, what evidence it rested on, what was observed and what was inferred, the explanation, what the team decided to do about it, and what happened afterward. I call that a Decision Record. It isn’t the transcript of the investigation. It’s the thing an organization would want on file about that decision if anyone ever asked again.
Compare what the Thursday left: a spreadsheet with a hand-built column called “should have been same-day?”, a thread, and a note to the owner whose central word was “think.” If something shaped like that happens again in October, whoever picks it up starts from the runs.
With a record, the second time goes differently. The agent makes a similar call, and instead of an operator searching exports for the word “water,” the function recognizes it: this looks like the pattern from September, here’s what caused it then, here’s what the team changed, and here’s whether the change held. The second investigation starts from the first one’s conclusions instead of from the runs, not because anyone got smarter, but because the first one was kept.
That’s the “remember” part of the job, and it’s what makes the job worth having done in the first place. On a small team, the scarcest thing is rarely someone who can reason about an agent. It’s history: what normal looks like, what went wrong last time, and what fixed it. Nobody on staff has that written down, because nothing was writing it down.
Why this is the part that matters
A lot of organizations running AI today are paying for the same explanations over and over. Every surprising decision gets investigated from scratch by whoever is closest and has the most context, and the result gets consumed the moment it’s delivered. It’s a sensible way to handle one incident. As a way to run a system that makes decisions all day, every day, it doesn’t hold up. The hours are the visible cost. The bigger one is that the organization never gets better at understanding its own AI, because every lesson is spent as it’s learned.
My guess is that the difference between the teams that keep extending their agents and the teams that quietly add a human chaperone after a day like that Thursday will have less to do with who explains a decision fastest, and more to do with whether the last explanation left anything behind. A team that keeps its investigations has something to stand on the next time the owner asks whether to turn the agent off: evidence, precedent, a record of what changed and what held. A team that doesn’t has another Slack thread.
That’s what I mean by an always-on decision engineer. It’s the function that watches every decision, looks closely at the ones that matter, is honest about what the evidence supports, answers in the terms of the person who asked, and keeps what it learned, so the organization doesn’t start from zero on the next decision.
I’m building Nalyqor to be that function, on the belief that the teams running AI in production will need the record at least as much as the answer.