One Thursday, one booking agent, and the people it pulls in before anyone can say why it made the call it made.
This is a composite. The company, the agent, and the day are assembled from a pattern rather than lifted from one team, so none of the details are anyone’s in particular. But if you own an AI workflow at a small company and there’s no ML engineer down the hall, I’d expect most of it to read less like a story than a memory.
You run operations at a home-services company. Forty-odd people, most of them technicians. About five months ago your one in-house developer wired an agent into intake: it reads every request that comes in through the website form, texts, and the shared inbox, decides how urgent it is, books the job into the technician schedule, and sends the customer a confirmation. It was the best thing that happened to the front desk all year. Nobody answers the inbox at 10pm anymore, and nobody needs to.
8:52
The first signal is a support rep forwarding call notes into the ops channel. No alert, and no comment except “??”.
A customer wrote in Monday night: a pipe burst under the upstairs bathroom, water came through the kitchen ceiling, they shut off the main. The agent booked them into a standard slot next Wednesday and sent a friendly confirmation. The customer has had no running water since Monday night and has now called, twice.
The rep has already fixed it by hand. Moved the job, called the customer, apologized. The comment that lands, a few messages further down, is the one the rep types almost as an aside: “think this is the second one this week?”
9:10
You open the dashboard, because that’s what it’s for.
Bookings are on pace. Confirmations sent: all of them. Error rate: flat. Average handling time is actually down a little. Every run for the past week is green, including Monday night’s. By every number anyone set up to watch, the agent did its job.
Then you open the run itself. You can see the customer’s message, word for word. You can see the agent check the schedule, create the booking, send the text. You can see the urgency it assigned: standard. What you can’t see anywhere on the screen is the part you actually need, which is why a message with “burst” and “ceiling” in it came out standard.
You reread the message and notice the line that almost makes sense of it: “We’ve shut the water off so it’s not flooding anymore, but we need someone as soon as possible.” Almost. You can squint and see how something might have read that as contained. You can’t tell whether that’s what happened, or whether you’re just the one doing the squinting.
9:45 to 11:30
The question stops being “why did it do this” and becomes “how many times has it done this,” which, it turns out, is harder.
Nothing you have can answer it. There’s no view for requests that sounded urgent and got booked standard. So you do what operators do. You export two weeks of intake, open it in a spreadsheet, and start searching for words: leak, burst, flood, water, ceiling. Forty-some rows. You read each one and the booking next to it, and build a column by hand called “should have been same-day?”
Some were handled correctly. A few clearly weren’t. Several you genuinely can’t call, and you notice you’re less sure of your own judgment by row thirty than you were at row one. You also can’t tell whether any of this is new, because you never looked at these bookings before this morning. There was no reason to. They were green.
11:40
You message the developer.
The developer built the agent, and is also, this sprint, supposed to ship the customer portal the owner has been promising your largest property-management client since June. The reply comes twenty minutes later and is a single question: “did anything change?”
It’s the right question, and you don’t know the answer. The developer checks what they can see from their side: the prompt in the repository hasn’t been touched in a month, the tools are the same, the model version is the same. From the code’s point of view, nothing changed.
It’s after lunch before you remember the guidelines doc.
1:05
Two weeks ago the owner asked ops to get same-day overtime under control. You added a line to the intake guidelines, the document the agent reads before it books anything: if the customer has already stopped the problem, book the next standard slot. You were thinking about the dripping faucet with the valve closed. You weren’t thinking about a house with the main shut off.
You didn’t think of that edit as a change to the agent. You thought of it as updating a policy doc, and policy docs don’t go through review.
It’s a plausible story. It might even be the story. But you notice that you can’t actually confirm it, and neither can the developer, not today. Confirming it means replaying the flagged requests against the old guidelines, which is developer time the portal doesn’t have. So the explanation stays at “probably,” and you move on, because there are still customers from the spreadsheet who need a phone call.
2:30
The owner has been reading the channel, and asks the only question that really matters to the business:
“Should we turn it off?”
Nobody can answer that with evidence. You can say the agent books a lot of jobs a week and most of them look fine. You can’t say how many look fine because they are fine, and how many look fine because nobody has read them. The developer can say the system is healthy, and mean it, and that doesn’t help, because it’s an answer about whether the agent ran.
So the team does what small teams do in this position, which is to split the difference. You revert the guidelines line, to be safe, and accept that overtime will creep back up. And a support rep will now read every request that mentions water before the agent’s booking stands, until further notice. Nobody writes down what “further notice” means.
5:45
The last task of the day is the note to the owner. It’s the hardest thing you’ll write all week, and it comes out as: “The agent misread some urgent requests as non-urgent. We think a guideline change caused it, and we’ve rolled it back. A rep is reviewing water-related bookings for now.”
Read it again tomorrow and you’ll notice it describes what happened without quite explaining it. “Misread” is a label for the outcome. “We think” is carrying the whole diagnosis.
The spreadsheet sits in your downloads folder. The developer’s reply is in a thread that will scroll out of view by Monday. If something shaped like this happens again in October, whoever picks it up starts from the runs, not from today.
What that day actually was
I want to be careful not to turn this into a lesson about intake guidelines or change management. You could fix both and still have this day, just with a different cause.
What stands out to me is how the explanation was distributed. The operator had the business context: what urgent means for this company, which jobs are worth a same-day truck, what changed in the policy two weeks ago. The developer had the system context: what the agent reads, what it calls, what did or didn’t change in the code. Neither of them could produce the explanation alone, and there was no place the two halves met except a Slack thread, for a couple of hours, on a day when both people had other jobs. On a small team, that’s usually what “nobody can explain what the agent just did” means. The explanation exists only while the right two people are in the same thread, and it’s gone once the thread scrolls away.
It also never looked like an incident, which is part of why it’s so easy to underprice. There was no outage, no page, no postmortem. The cost showed up as a morning of the operator’s time, an afternoon of calls, a developer pulled off the portal, and a sentence with “we think” in it. Plus one cost that doesn’t show up anywhere, which I’d argue is the biggest: by the end of the day, a rep was back reading bookings. The team took back part of the job it had handed the agent, not because anyone proved the agent was bad at it, but because nobody could show it was good at it.
I’ve written about why small teams can’t hire their way out of this, and about how the same question lands on engineering in larger organizations. Those pieces describe this from the outside. From the inside, it doesn’t feel like a staffing gap. It feels like a Thursday.
Where the market actually starts
If you’re building for AI reliability, it’s worth being honest about who has this day, and what they do next.
It’s rarely a platform team or an AI governance function. It’s a head of operations, a support lead, a founder: whoever owns the workflow the agent sits inside. They’re unlikely to go shopping for an observability product afterward, because nothing about the day felt like an observability problem. Every run was there to look at. What was missing was an answer to “why did it do that?” in the terms they think in, fast enough to matter before the owner asks whether to turn it off.
As far as I can tell, the most common outcome of a day like this is retreat. A little more human review, a little less scope, one fewer workflow the team is willing to automate next quarter. The company never decides AI didn’t work. It just quietly stops extending it. I suspect a lot of what reads as small-team AI adoption stalling is really this: agents that worked, hit a day nobody could explain, and got a chaperone afterward.
The teams that keep extending will be the ones who can get from “that booking looks wrong” to a real explanation before the note to the owner gets written. That’s the gap I’m building Nalyqor to close. If you’ve had your own version of this Thursday, I’d like to hear about it.