A Look Into Safety and Security: When Plausible Code Is the Threat

Safety used to mean catching crashes. Security used to mean patching known bugs. AI changes the threat model because the danger now often looks professional.

Generated code can introduce insecure defaults with polite variable names, leak secrets through conversational context, broaden permissions "just to make it work," and invent APIs that almost match your stack until production.

This piece is a hard look at safety and security when machines write faster than teams review.

We walk through concrete failure modes: auth that trusts the wrong claim, payment flows that skip idempotency, databases without least-privilege thinking, logs that casually store PII, and dependency suggestions nobody vetted.

The agent harness is a new class of risk even when the generated TypeScript is "fine." Repo rules, session hooks, and workspace tasks are instructions with privilege. Secrets in context are secrets in transit. A model that can read .env to "debug locally" has already widened the blast radius. You do not need a villain for this. You need a helpful default.

Then we turn it into practice — a defensive checklist for AI-assisted teams: never paste production secrets into chat; require human review on identity, money, encryption, and migrations; run scanners as habit, not theatre; keep threat models short and real for sensitive features; treat rate limits, abuse, and graceful failure as product safety, not backlog leftovers.

Give agents the least filesystem and credential access that still lets them work. Prefer redacted logs. Prefer typed boundaries around auth so a fluent hallucination cannot quietly swap "verify" for "decode." If a change touches Clerk, Stripe, or a migration, a human who can draw the flow owns the merge.

Safety is also product language. A crash is honest. A 200 that charged twice is a story you tell a lawyer. Generated code loves the happy path because the prompt described the happy path. Your job is to describe the other paths until the model is bored of them.

The goal isn't fear. It's making AI velocity compatible with duty of care.

If you wouldn't bet a customer's trust on a black-box paragraph, don't ship it.