AI in Security: The Guardrail Became the Attack Surface

Two stories are running at once, and most decks only tell the flattering one.

Story one: defenders using models for triage. Alert volumes that used to melt a SOC get clustered, summarised, ranked. Phishing text gets classified. Anomalies get a first pass before a human spends the expensive hour. That is real, and it is the same old lesson — automation for the flood, humans for the weird. Used with logs, limits, and a person who can override the model, it is a force multiplier. Used as a replacement for a security function, it is a new way to miss the slow incident.

Story two: the guardrail grew a mouth. Coding agents read the repo, the env file, the internal wiki. They suggest IAM that "just works." They paste a secret into a prompt because you asked them to debug. They widen a CORS policy with polite comments. The code looks like a senior wrote it. The threat model looks like nobody did. Plausible secure-looking code is still the threat. The new twist is that the assistant is also a path: context windows as exfiltration, tools as hands, a helpful default as a privilege escalation with grammar.

The collision is the 2026 job. The same class of model that drafts your WAF rule can draft the bypass-shaped feature if the prompt is a deadline. Security teams that only scan yesterday's CVEs will miss today's harness. App teams that only ship velocity will donate the incident.

Practice is blunt. Identity, money, encryption, and migrations stay human-reviewed — not "the agent plus a rubber stamp." Least privilege for tools: the agent does not need production data to write a test double. Redact. Prefer fake customers in local context. Do not paste live secrets into chat; if the product cannot boot without them in the thread, the product is the bug.

Treat AI-assisted review as a second reader, not a court. A model catching a missing await is lovely. A model blessing an auth change is a vibe. Keep scanners on the merge. Keep a short threat model next to any feature that stores a person or moves a rand. Rate limits and abuse cases are product safety. They are not a backlog garnish.

The guardrail is now part of the surface. Inventory it. Constrain it. Make refusal cheap — a developer should be able to say no to an agent the way they say no to a dangerous PR. If your security story is a logo in the footer and a model in the IDE, you do not have a security story. You have a hope.

Defenders should still use the models. That is not a contradiction. Use them where the flood is inhuman: triage, clustering, first-pass summarisation of a thousand similar alerts. Then put a person on the residue — the slow, the novel, the thing that looks like a business process and is actually a theft. The failure mode is letting the same fluency that drafts a playbook also close the ticket.

In product engineering the translation is: let the agent draft the rate limiter. Let a human who has been abused by a bot decide the numbers. Let the agent propose the auth check. Let a human who has been burned by a wrong claim own the merge.

Hope is not a control. Review is.