AI Coding Tool Security Risk for Engineering Leaders

The research on AI-generated code security reaches engineering teams as anecdote before it reaches them as data. Here’s the data.

Three independent studies from the last 24 months document a consistent pattern: AI coding assistants produce code that passes developer review at higher rates than it deserves, the security gap doesn’t close as models get larger, and most engineering organizations have no formal process for managing the risk. All three problems have known solutions. The first step is understanding the actual scope.

What AI-Generated Code Security Research Shows

BrassCoders’s N=15 AI-generated code corpus detected at least one security finding in all 15 files when scanned with the full 12-scanner stack — a controlled research sample, not a deployment-scale dataset. The corpus is published at coppersun.dev/benchmarks: 15 Python files generated from security-relevant prompts, with reproducible scan instructions. The external research operates at substantially larger scale and reaches similar conclusions.

Veracode’s July 2025 GenAI Code Security Report tested over 100 large language models across 80 distinct security-relevant tasks. The headline number: 45% of AI-generated code samples fail security tests. Broken down by language, Java failed at 72%, C# at 45%, JavaScript at 43%, and Python at 38%. Cross-site scripting tests failed at 86% of relevant test cases across the tested models. The finding that matters most for engineering leaders: newer and larger models showed no consistent security improvement over earlier models. Scaling the model doesn’t close the vulnerability gap.

Perry et al. (CCS 2023, 47 participants) ran a controlled experiment assigning developers either an AI coding assistant or no assistant. On an ECDSA signing task, 3% of the AI-assisted group produced secure code versus 21% of controls (p=0.039). The ECDSA task is a useful probe because correct implementation requires parameter choices subtle to non-cryptographers — the nonce k must be random and never reused, and an incorrect implementation looks structurally correct to reviewers who don’t know that constraint. The study used codex-davinci-002, a 2022-era model; frontier models today produce more fluent code. Whether they’ve closed the cryptographic correctness gap is an open research question. Veracode’s 2025 findings suggest the gap persists across model generations.

Checkmarx’s Future of AppSec 2025 report (1,519 respondents, August 2025) measured the organizational response: 98% of organizations reported a security breach attributable to vulnerable code, and only 18% had governance policies in place for AI coding tools. Adoption is ahead of governance by 82 percentage points.

Why AI Assistants Produce Insecure Code

BrassCoders’s N=15 corpus shows three bug categories recurring across the AI-generated files: SQL injection via format-string query assembly, hardcoded credentials in configuration and initialization code, and unsafe cryptographic calls. All three share the same structural origin — the AI generates code that satisfies the stated prompt and passes local tests, with no model of the production attack surface the code will face.

SQL injection through format-string assembly is the clearest case. A Flask database query generated from “write a user lookup by ID” produces:

cur.execute("SELECT id, name, email FROM users WHERE id = %s" % user_id)

This query returns correct rows during development. Tests pass. The vulnerability — SQL injection via user-controlled user_id — only manifests when an attacker sends a crafted string to the endpoint. Standard tests don’t exercise that path unless the suite was written specifically to test injection resistance. BrassCoders flags this as CRITICAL via Bandit B608 and Semgrep taint analysis. The fix is parameterization:

cur.execute("SELECT id, name, email FROM users WHERE id = ?", (user_id,))

Hardcoded credentials follow the same pattern. A configuration module generated from “write an HMAC signing key setup” produces SECRET_KEY = "s3cr3t-signing-key-change-me". A literal string satisfies the prompt; it works during development; it commits to version control when the developer runs git add. BrassCoders’s SecretsScanner flags it HIGH and redacts the credential value from YAML output — the code snippet and file path appear, but not the literal secret. The fix: SECRET_KEY = os.environ.get("HMAC_SECRET_KEY", "").

The cryptographic case requires more context-reading. hashlib.md5() without usedforsecurity=False triggers Bandit B324 because MD5 is broken for authentication and signing — collision attacks let an attacker substitute a different input that produces the same hash. For content deduplication (asking “are these bytes identical?”), collision attacks are irrelevant. The AI generates code correct for the stated use case. The scanner correctly flags the structural pattern. Whether the finding is a real vulnerability depends on what the digest is used for downstream. That’s context the scanner can’t read. The AI reviewer handles it.

The Overconfidence Problem

BrassCoders catches SQL injection, hardcoded credentials, and command injection patterns at commit time — before the code enters a review cycle where Perry et al.’s overconfidence finding becomes expensive. The study’s core result was explicit: AI-assisted developers rated their code as more secure than controls rated theirs, while actually writing less secure code. The model’s apparent confidence transfers to the developers reviewing its output.

This is a review-calibration problem, not a one-time training fix. Fluent, well-structured AI-generated code carries a lower friction signal than code that looks uncertain or inconsistent. The hesitation that catches a suspicious SQL interpolation — “wait, is that injectable?” — is quieter when the surrounding code looks professional. The mechanism isn’t carelessness. It’s what well-formatted, syntactically correct code communicates to reviewers trained to trust that signal.

Two practical implications follow for engineering teams. First, peer review processes for AI-generated PRs should include an explicit scanning step, not only visual review. The scanner catches structural patterns that visual review misses under overconfidence conditions. Second, PR reviewers should know they’re reviewing AI-generated code — the Perry overconfidence effect is partly a calibration issue, and reviewers who know the source may apply more scrutiny to the output.

Veracode’s 2025 State of Software Security report found average remediation time for security flaws has increased 47% over five years, to 252 days. A bug that ships in an AI-generated commit runs the same remediation sequence as any other production security bug: detection, log audit to bound the exposure window, credential rotation if the window is uncertain, compliance notification if regulated data was in scope. The commit velocity AI coding tools provide doesn’t compress that timeline. What compresses the timeline is when the bug is caught. At commit, it’s seconds. At production, it’s months.

The Governance Gap

BrassCoders’s scanner output goes to Claude Code or Cursor for AI-assisted triage — the AI reviewer reads the YAML, source-verifies each finding against the original file, and classifies each as real vulnerability or false positive with fix-ready advice. That division of labor (BrassCoders as deterministic pattern-matcher, AI reviewer as context-aware triage layer) is the technical architecture. The governance question is whether that architecture is required before AI-generated code can merge.

The Checkmarx 2025 data makes the gap concrete: 82% of organizations with active AI coding tool usage have no defined process for what review AI-generated commits require. The tooling problem is solvable in an afternoon. The governance problem is a management decision about what process is required.

Three decisions define a minimal AI coding governance policy:

What scans run before merge. At minimum, a static-analysis scan covering the categories AI assistants structurally miss: SQL injection, hardcoded credentials, unsafe subprocess calls, and phantom imports from AI hallucinations. BrassCoders OSS core covers all of these with no account required.

Who reviews scanner findings. Automated scanning produces findings; a human closes the loop. The AI reviewer (Claude Code, Cursor) does context-aware classification. The developer confirms. Without a defined owner, findings queue unreviewed.

What the verified false-positive process is. Some findings are false positives in context. .brassignore captures these decisions so they don’t consume review cycles on future scans. A glob rule like file_dedupe.py suppresses all findings from a specific file; a type rule like :brass.python.taint.sql-injection suppresses a specific Semgrep check project-wide. Without a suppression process, teams tune out the scanner after the first wave of noise.

These three decisions don’t require new tooling. They require the management decision that AI-generated code has a defined review process. That decision is what the 18% who have governance policies have made.

Building a Security Gate for AI-Generated Code

BrassCoders installs in one command and runs offline with no account required — the OSS core catches the structural bugs AI coding assistants produce without any data leaving the developer’s machine.

pip install brasscoders
brasscoders scan /path/to/project

The scan emits YAML to .brass/findings.yaml, structured for Claude Code or Cursor consumption. Each finding includes severity, detector, file path, line number, and code snippet. The AI reviewer reads the file, verifies each finding against the source, and produces a triage output: real bug with fix advice, or false positive with reasoning. The scanner is the deterministic first layer. The AI reviewer is the context-aware second layer.

CI integration runs the scan on every commit:

# .github/workflows/brasscoders.yml
steps:
  - name: BrassCoders scan
    run: pip install brasscoders && brasscoders scan .

BrassCoders Paid ($12/developer/month) adds a semantic deduplication pass and project-signature reranking that reduces a 1500+ finding raw scan to approximately 300 prioritized findings. The enrichment pass processes already-redacted findings and a project signature derived from README, manifest, and entrypoint metadata — no raw source code leaves the machine. For teams where 1500 raw findings make AI-assisted triage impractical at scale, the enrichment pass is what makes the process work.

The engineering leadership version of this is binary: the gate runs in CI before merge or it doesn’t. Perry, Veracode, and Checkmarx describe what happens when it doesn’t. The gate costs one afternoon to wire and runs in seconds per commit.