Probabilistic Code Needs Human Ground Truth

Pragmatic Sysadmin — Thu, 15 Oct 2026 00:00:00 +0000

Last month I watched a support team demo their new AI deflection wall with pride. A customer asked where their invoice was, and the bot answered — confidently, politely, instantly — with a ticket number that didn’t exist. Nobody caught it for three days. The bot wasn’t lying. It was doing exactly what probabilistic models do: producing the most likely next tokens. The team had just mistaken fluency for truth, and their customers paid for the confusion.

I’ve seen the same mistake in my own terminal. You’ll pull a generated script that passes CI, then watch it break in production because the logic missed an edge case the model never saw. That gap between plausible output and reliable behavior is where the real work lives — and it’s work only a human can do.

Meta on Pragmatic Tech

Probabilistic Code Needs Human Ground Truth