The Secrets Your AI Assistant Might Leak (And How to Catch Them)

BrassCoders detects around 20 canonical secret formats — AWS access keys, GitHub personal access tokens, OpenAI keys, Stripe live keys, JWT tokens, PEM-encoded private keys, Anthropic keys, Slack tokens, NPM auth tokens, SendGrid keys, Mailgun keys, DigitalOcean tokens, Twilio credentials — plus a high-entropy fallback that catches the formats it hasn’t seen before. The reason this matters: AI coding assistants embed secret-shaped strings in generated config files, example scripts, and test fixtures more often than developers expect, and the secret in your AI-generated .env.example looks just like a real secret to a downstream attacker.

This post covers why AI tools generate credential-shaped output, what specific formats are worth scanning for, how entropy-based detection catches the formats nobody has cataloged yet, and how BrassCoders’s two-boundary redaction prevents the detector itself from becoming a leak surface.

Why AI Generates Code With Embedded Secrets

BrassCoders treats embedded-secret hallucinations as a known AI failure mode, not an edge case. Models that have seen millions of code examples in training have also seen millions of API keys, tokens, and passwords embedded in those examples; in the absence of an explicit safety policy at generation time, the model reproduces credential-shaped strings as plausible filler when generating config files, example scripts, or test fixtures.

The pattern happens in three recurring contexts. First, example code generation: ask the AI for “an example .env file for a Stripe + SendGrid + AWS setup” and the model produces a file with what look like Stripe live keys (starting sk_live_), SendGrid keys (starting SG.), and AWS access keys (starting AKIA). The keys are syntactically valid — the model knows the format perfectly because it saw thousands of real ones in training — and a junior developer might commit the file without realizing the AI invented credentials that look real enough to fool security automation.

Second, test fixture generation: ask the AI to “write a test that verifies authentication” and the model fabricates a JWT token, a PEM-encoded RSA private key, or a session cookie value. The fabricated credentials are usually high-entropy and structurally correct. Some get committed to repos as “test data” and end up triggering security scanners weeks later.

Third, completion-context leakage: an AI assistant operating on a partial file may complete a config line based on patterns it saw nearby. If your real .env is open in the same editor session and the AI has context, the completion may include credential-shaped strings borrowed from that context — sometimes the real credentials, sometimes a hallucinated variant that’s been altered just enough to look plausible. GitGuardian’s annual State of Secrets Sprawl report has tracked AI tooling as a contributing pattern in the growing rate of public-repo secret leaks.

The mitigation is structural: scan every diff for credential-shaped strings before it’s committed, treat every hit as a CRITICAL finding, and never assume the developer noticed.

The 20+ Secret Formats BrassCoders Detects

BrassCoders ships with detection coverage for around 20 canonical secret formats, drawn from a combination of an established upstream secret-detection library and BrassCoders-specific patterns added for the formats AI tooling tends to fabricate most often.

The upstream library is Yelp’s detect-secrets, which provides the historical-format coverage; the BrassCoders additions target the API ecosystems where AI hallucinations cluster. The full active list as of release 2.0.4:

• Cloud provider credentials: AWS access keys (AKIA..., 20 chars), AWS secret access keys (40-char base64), DigitalOcean personal access tokens, Twilio account SIDs and auth tokens.
• Developer-tooling credentials: GitHub personal access tokens (classic ghp_... and fine-grained github_pat_...), NPM auth tokens, Anthropic API keys, OpenAI API keys.
• Payment and messaging credentials: Stripe live keys (sk_live_...) and publishable keys, SendGrid keys (SG....), Mailgun keys, Slack bot tokens and webhook URLs.
• Cryptographic material: PEM-encoded private keys (RSA, DSA, EC, OpenSSH), JWT tokens (three-segment base64 with eyJ... prefix), and PGP private key blocks.
• Web app credentials: Generic high-entropy strings in environment-variable assignments, hardcoded password literals, and common session secret patterns.

Each pattern is a regex tuned to match the format’s canonical structure. The patterns are intentionally conservative: it’s better to occasionally flag a non-secret high-entropy string for human review than to miss a real credential. False positives cost a developer 30 seconds of dismissal; false negatives cost a credential rotation and a security postmortem.

The upstream detect-secrets library handles a broader range of historical formats (Slack legacy tokens, Datadog keys, deprecated AWS formats) that BrassCoders picks up automatically when detect-secrets ships an update. BrassCoders’s value-add is the AI-tooling-tuned subset: the formats that LLMs hallucinate or borrow from context most often.

How Entropy-Based Detection Works

BrassCoders’s entropy-based detector catches secret-shaped strings that don’t match any cataloged format. The detector computes Shannon entropy on each candidate string; values above a threshold (typically 4.5 bits per byte) signal randomness consistent with credentials rather than human-readable text.

Shannon entropy is a well-established information-theoretic measure of randomness in a string. A coin-flip sequence has entropy 1 bit per flip. A uniformly-random 256-character base64 string has entropy close to 6 bits per character. A human-readable English sentence has entropy around 1-2 bits per character because letters follow predictable patterns (Q is usually followed by U, common bigrams dominate). Real credentials sit between these two endpoints — they need to be high-entropy enough to resist guessing, which puts them squarely in the band where the entropy detector flags.

The implementation in BrassCoders: for each string literal in scanned code, the detector computes the Shannon entropy. If the entropy exceeds the threshold AND the string appears in a suspicious context (right side of an = in a config file, value of a key named password / secret / token / key / api_*, hardcoded in source as a constant), BrassCoders surfaces it as a finding. The context check matters because uniformly-random data appears in plenty of legitimate places — UUIDs, content hashes, binary blobs — that aren’t credentials.

What entropy detection catches that pattern matching misses: rotated formats (when a vendor changes their key format, the old regex misses but the entropy is unchanged), internal-format credentials (your company’s custom JWT, session-token format, or internal SSO bearer that no public regex knows about), and the credentials AI assistants invent that look novel but high-entropy. The detect-secrets library’s keyword detector implements this pattern at the upstream layer; BrassCoders tunes the thresholds for AI-tooling-shaped output specifically.

How BrassCoders Redacts at Two Boundaries

BrassCoders redacts secrets at two boundaries: the scanner masks matched values at detection time, and the YAML writer strips known-sensitive metadata keys at serialization time. The redaction is the same whether output is being written locally to .brass/ or sent to the BrassCoders Paid plan’s hosted gateway.

The scanner boundary handles the immediate masking. A credit-card-shaped number like 4111-1111-1111-1111 gets masked to 4111****1111 before any BrassCoders-owned object holds the full value. An AWS access key gets the type recorded but never the value — the finding includes type: AWS_ACCESS_KEY and a file path, not the literal AKIA... string. JWT tokens, PEM-encoded private keys, and other high-entropy credentials get similar treatment: the type and location persist; the secret value does not.

The YAML writer boundary is the belt-and-suspenders layer. For any finding whose type is PRIVACY or whose detector is on the secret-leak allowlist, the writer strips matched_text, code_snippet, context_line, raw_match, and context from the serialized output. This catches anything the scanner forgot — if a future scanner forgets to mask at the source, the writer’s allowlist still strips it before serialization.

Why two boundaries: defense-in-depth. The single-layer pattern (mask at the source) fails the moment any scanner is added or modified without the mask logic being updated. The two-layer pattern means the writer is a known choke point through which every finding flows, regardless of which scanner produced it. Adding a new secret-detecting scanner doesn’t require touching the writer; existing redaction applies automatically.

The data-handling guarantees: the same redacted output goes to disk and to the gateway. No “this version is for our servers, that version is for the local file” split. Whatever you can read in .brass/security_report.yaml is exactly what the gateway sees. For the full data-flow detail, see What BrassCoders Sends to Its Servers (And What It Doesn’t).

How to Set Up Secret Scanning in CI

BrassCoders’s secret scanning runs by default with every brasscoders scan; no flag needed. CI integration is a one-line addition: run the scan, grep the output, fail the build if any HARDCODED_SECRET findings appear.

A minimal GitHub Actions workflow:

name: BrassCoders secret scan
on: [pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pipx install brasscoders
      - run: brasscoders scan .
      - run: |
          if grep -q "HARDCODED_SECRET\|HARDCODED_CREDENTIAL\|PRIVATE_KEY" .brass/security_report.yaml; then
            echo "Secret detected in code. Failing build."
            cat .brass/security_report.yaml
            exit 1
          fi

For pre-commit hooks, the same pattern in .git/hooks/pre-commit:

#!/bin/sh
brasscoders scan --offline . > /dev/null
if grep -q "HARDCODED_SECRET\|HARDCODED_CREDENTIAL\|PRIVATE_KEY" .brass/security_report.yaml; then
  echo "Secret detected. Rejecting commit. Review .brass/security_report.yaml"
  exit 1
fi

The --offline flag in the pre-commit hook keeps the scan from triggering any network calls (including the optional package-hallucination check). For pre-merge in CI, drop --offline if you want both checks running.

What this defends against: every secret-shaped string the AI puts in your diff, every example credential a junior developer copy-pastes from generated code, every test fixture that contains a too-real-looking token. What it doesn’t defend against: secrets already in git history (use gitleaks or detect-secrets in scan-historical mode for that), secrets stored elsewhere on disk, or secrets passed via legitimate channels like environment variables that aren’t in source.

---

The hardest secret to detect is the one that looks legitimate enough that nobody questions it. AI-generated credentials are designed (by training) to look exactly like real credentials. Scanning for them is the only reliable defense.

Install BrassCoders with pipx install brasscoders and run brasscoders scan against your project. For the broader context on AI code review failure modes, see AI Code Review: The Practical Guide for 2026.