Can a Kubernetes security control prevent an action without generating useful evidence?
Question
When a validating admission webhook blocks a request, does the cluster retain enough evidence to reconstruct what was attempted, by whom, and how many times?
Hypothesis
A denied admission request generates a Kubernetes API audit event, so I expected the default audit configuration to give us everything we'd need for a post-incident timeline without any extra instrumentation.
Setup
A kind cluster running a validating admission webhook (built on Kyverno) that blocks any Pod spec requesting hostNetwork: true. Kubernetes audit logging enabled at the default "Metadata" level, the level most managed clusters ship with out of the box.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-host-network
spec:
validationFailureAction: Enforce
rules:
- name: no-host-network
match:
resources:
kinds: [Pod]
validate:
message: "hostNetwork is not permitted"
pattern:
spec:
=(hostNetwork): "false"
Experiment
I attempted to deploy 12 pods requesting hostNetwork: true from three different service accounts over a 20-minute window, simulating a mix of a misconfigured deploy pipeline and a deliberate privilege-escalation attempt. Then I tried to answer four questions using only what the cluster had recorded: who tried this, how many times, from where, and whether the attempts were increasing in frequency.
Result
The Kyverno policy worked exactly as designed: all 12 pods were rejected at admission. But at the default "Metadata" audit level, the retained audit event recorded the verb, the resource type, the user, and the response code (denied), and nothing about the request body. There was no record of hostNetwork: true specifically, no way to distinguish this denial from any other validation failure, and no record of which container image or namespace was involved beyond what "Metadata" level exposes. Reconstructing intent from the audit trail alone was not possible. Raising audit logging to "RequestResponse" level made every field available (including the full pod spec) but increased audit log volume by roughly 40x in this test, which is the reason most clusters don't run at that level by default.
What I learned
A prevention control and a detection control are not the same investment, and enabling one doesn't give you the other for free. The admission webhook did its job completely: nothing was allowed to run. But if this had been a real privilege-escalation attempt instead of a test, security response would have known an attack was stopped and almost nothing else about it: not the technique, not whether it was automated, not whether it was still being retried. Getting real evidence required deliberately raising audit verbosity for the resources that matter, which is a cost/signal trade-off worth making on purpose rather than discovering during an incident.
Referenced in
The Security Control That Worked Too Well
Effectiveness and accountability are not the same property. A control can have both, or either, or neither, and the failure mode of 'either' is easy to miss.
24 Aug 2026 · 3 min read
The Castle Was Never the Problem
The castle-and-moat model isn't wrong because perimeters are useless. It's wrong because it teaches you to stop asking questions the moment something is denied entry.
22 Jun 2026 · 4 min read