The Security Control That Worked Too Well

The assumption

If you'd asked me to rank the properties of a good security control, "does it block what it's supposed to block" would have been at the top, by a wide margin. Everything else (explainability, auditability, how noisy the false-positive rate is) felt like polish. Nice to have, not the point.

The problem

The disallow-host-network policy from the admission-control experiment is, by the metric I used to rank controls, a great control. It caught every single violation across every test run. Not one bad pod spec got through. If effectiveness were the whole story, this article wouldn't exist.

But when I sat down to write an incident report using only what the policy itself produced, I had almost nothing. Kyverno's policy reports, at default settings, recorded that a resource was blocked and which rule blocked it: not the offending field, not a diff against what a compliant spec would look like, not enough for someone who wasn't already deeply familiar with the policy to understand, from the report alone, what the submitter did wrong.

That's a specific kind of failure: the control worked, and its own record of working was nearly useless to anyone who wasn't the person who wrote it.

What changed my mind

I started imagining the on-call engineer who isn't me: someone paged at 2 a.m. because a deploy pipeline is failing every pod submission in a namespace, with no context about why. The policy report says: blocked, rule no-host-network. It doesn't say which of forty pod specs in that deploy had the problem, what the field looked like, or how to fix it. The engineer now has two choices: page the person who wrote the policy, or start guessing. Neither is what "working" should feel like from the inside.

Effectiveness had been optimized in isolation. Nobody had asked the second question (effective and understandable to whom, under what pressure) because the first question is the one that shows up in a demo.

The uncomfortable part

The uncomfortable part is that fixing this wasn't a matter of writing a better regex or tuning a threshold. It required deliberately adding verbosity to every policy: custom message fields that explain the violation in terms a non-author can act on, structured output that names the exact field and its offending value. And that verbosity has a real cost: policies take longer to write, longer to review, and are easier to get subtly wrong when the message text drifts from what the rule actually checks.

Nobody wants to spend that time on a control that's "already working." That's the trap. The control's success at its primary job is exactly what makes its secondary failure invisible: there's no red X anywhere. Just a quietly unhelpful policy report, waiting for the night someone actually needs it.

What I think now

I've started treating "can someone who didn't write this policy understand why it fired, without asking me" as a release criterion, not a nice-to-have. If a policy passes every test but fails that question, it's not done. It's effective and unaccountable, which is a specific, nameable kind of unfinished.

The takeaway

Effectiveness and accountability are different properties, and a control can have one without the other. The dangerous version isn't the control that fails quietly. It's the control that succeeds constantly while staying silent about why, because success is exactly what makes that silence easy to overlook.