Where Your LLM App Actually Leaks Data

A source scanner catches PII written into your code — a hardcoded email, an API key in a string literal. An LLM app leaks somewhere a scanner can’t look: at inference, when the model emits a piece of sensitive data it was handed through context or retrieval, a string that was never in your source to begin with. BrassCoders scans source; governing what the model returns is a runtime control, and this is one more place worth knowing exactly where the two layers divide.

The leak you’re picturing and the leak that actually happens are at different layers, and a tool that conflated them would give you false confidence at the point you most need clarity.

What a Scanner Catches Versus What the Model Emits

BrassCoders flags PII and secrets that live in the source: a credential in a config file, a customer email in a test fixture, a high-entropy string that parses like a key. Those are structural, and they get caught the same way on every scan. A leak from an LLM application is a different event — the model receives sensitive data at runtime and includes it in a response, and that data was never a literal in your code.

Picture a support bot with access to a customer database. The account number it discloses to the wrong user isn’t hardcoded anywhere; it flowed in through a retrieval step and back out through the model’s answer. Scan the whole repository and you’ll find nothing, because there’s nothing to find. The disclosure is a property of what the model was allowed to touch and return, not of the code you wrote.

Why the Leak Isn’t in the Code

BrassCoders reads static source, and an inference-time disclosure has no static form. The model assembles each response from its context window — the system prompt, the conversation, whatever documents a retrieval step pulled in — and any sensitive value in that context can surface in the output. The sensitive string is data flowing through the system, not an artifact committed to the repository.

OWASP tracks this as LLM06, sensitive information disclosure, in its Top 10 for LLM Applications. The failure is that the model returns data the user shouldn’t see: another tenant’s records, secrets from the system prompt, PII from a retrieved document. None of it is a pattern in code. All of it is a runtime property of what the model can reach and what it’s allowed to say.

Where This Sits Next to Prompt Injection

BrassCoders indexes this in the same family as prompt injection: both are runtime attacks on the model rather than patterns in source, and they often chain. Prompt injection is how an attacker steers the model; sensitive-information disclosure is frequently what gets exfiltrated once it’s steered.

That pairing is why neither is a scanning problem. An injected instruction tells the model to read a secret and send it somewhere; the secret leaves through the output channel. A source scan sees neither the instruction (it arrived in untrusted content at runtime) nor the disclosure (it happened at inference). The whole exchange lives above the layer a pattern scanner can reach.

Where the Controls Live

The controls that stop runtime disclosure are runtime controls. Output filtering and DLP inspect what the model returns before it reaches a user and redact or block sensitive fields. Least-privilege access on retrieval keeps the model from fetching records it has no business touching. Context minimization keeps sensitive values out of the prompt in the first place, so they can’t surface in a completion.

Each of those operates on the running system, at inference, on the data path. A static scan of committed source has no visibility into any of them, and configuring them is the work that actually closes the exposure. A scanner that claimed to prevent runtime leakage would be inferring a runtime property it never observes.

What BrassCoders Covers

BrassCoders owns the source side of data handling: PII patterns written into code, hardcoded secrets, credentials in string literals, high-entropy values flagged deterministically on every scan. That layer is real, it’s structural, and AI assistants populate it regularly by pasting a key or an email into a completion.

The two layers are complementary. BrassCoders clears the secrets and PII that made it into your source; runtime filtering governs what your model discloses at inference. Run the scan so nothing sensitive ships in the code, and configure output governance so nothing sensitive ships in a response.

pip install brasscoders
brasscoders --offline scan /path/to/your/project

The scan catches the secret in your source. The data your model leaks at runtime is the output layer’s job, and a tool worth trusting draws the line where the layers actually meet.