← All books
The Decision Layer · Volume Two

The Cost of Not Knowing

Waste, risk, and why a new category is forming

Not knowing why your AI decided something isn't an occasional debugging problem. It's a standing cost, and most teams running AI in production are already paying it in three places at once. This book counts all three, and explains why a new category of tooling is forming around them.

Volume One named the gap between what an AI system did and why it did it. This volume follows that gap over months instead of during a single investigation, and asks who pays. The answer shows up in the money, when a cost fix shrinks the bill and the bill drifts back; in the people, when the same two or three engineers get pulled in to explain every failure; and in accountability, when no one can say whether an agent’s decisions were sound.

It is written for the people who run, build, or sign off on AI systems in production. Everything practical in it runs on the stack you already have, and most of it starts with one workflow and one question you can ask this afternoon. It stands on its own if you haven’t read Volume One.

What's inside
  • Check whether a cost fix removed the waste or only moved it
  • Work out why the same few engineers explain every AI failure
  • Price what an AI incident costs to diagnose, with a worksheet
Contents

Four parts, seven chapters, two appendices.

    • Introduction: The Invoice for "Why"
  1. I

    The Savings That Come Back

    • Catching Wasted Spend
  2. II

    The People Who Can Answer

    • The Expertise Bottleneck
    • Why Every AI Incident Becomes an Engineering Incident
    • The ML Engineer You Can't Hire
  3. III

    The Decision Nobody Owns

    • Who Owns the Decision?
  4. IV

    Why a Category Is Forming

    • From Observability to Behavioral Intelligence
    • Buy vs. Build
  5. V

    Closing and appendices

    • Closing: What It Costs to Keep Not Knowing
    • The Buyer's Checklist
    • The Senior-Engineer Tax
From the introduction

Start with an agent most teams running AI will recognize, because they have some version of it. (It's a composite, and it will come back several times in these pages.)

A support team puts an agent in front of its refund queue. The agent reads each request, checks it against the return policy, approves the ones that fall inside it and escalates the rest to a person. For a couple of months it does that well enough that nobody thinks about it, which is the whole point of putting it there.

Then three things happen to it in the same quarter, and none of them looks like an incident.

The Cost of Not Knowing, introduction
About the author

Jamal Jackson is a fractional CTO and the founder of FAI, where he builds production AI systems for companies that need them to hold up. He has been shipping software since 2008 and writing about the decision layer in public since the problem first cost him a week.

More about Jamal →

Also in the series

The argument continues.