When AI Invents Libraries: Detecting Hallucinated Imports

BrassCoders’s OSS core scans for hallucinated package imports with a single flag: brasscoders scan --check-package-hallucination. The flag is opt-in because the check is the only path in the OSS core that makes outbound network calls. The pattern matters because AI coding assistants generate imports of packages that don’t exist at rates measured by published research, and a typosquatter who registers those hallucinated names as malware turns the hallucination into a supply-chain attack.

This post walks through why hallucinated imports are a different kind of failure than other AI mistakes, what the published research says about how often they happen, how the typosquat attack chain works in practice, and how BrassCoders’s hallucination check fits into a CI workflow.

What Is a Hallucinated Import

BrassCoders treats a hallucinated import as a finding type that requires its own detector: an import or require statement referencing a package name that doesn’t resolve on the relevant registry. The AI writes from fastapi_users_pydantic import UserManager when only fastapi-users is published on PyPI; the syntax is valid; the IDE doesn’t complain; the failure surfaces at install time or — much worse — when a typosquatter has registered the hallucinated name as a malware-bearing package.

Why the hallucination happens: LLMs generate plausible-sounding package names by pattern-matching against the millions of imports they saw during training. A model that’s seen fastapi-users 50,000 times and pydantic 200,000 times can confidently produce fastapi_users_pydantic as a “combined” package because the structural pattern looks correct. The model has no grounding in whether the combined package actually exists; it’s just producing the most-likely next token given the prompt.

Why static analyzers miss it: traditional Python linters check syntax, type annotations, and code style. None of them verify that an imported name corresponds to a published package on PyPI. Pylint doesn’t. Bandit doesn’t. Pyflakes doesn’t. The verification requires a network call, and most linters are explicitly offline tools.

BrassCoders surfaces hallucinated imports as a security finding, severity high, with the package name and the source line. The downstream AI consumer (or human reviewer) then decides whether to replace the import with a real package, remove the code entirely, or accept the risk.

How Often Models Hallucinate Packages

BrassCoders’s position on hallucinated-package rates: the absolute percentage depends on the model and the benchmark, but the rate is high enough across published research that any team relying on AI-generated code without an existence-verification step is shipping a real risk. The specific numbers shift as models improve and as benchmarks change; the pattern is durable.

The most-cited research is Lasso Security’s 2024 analysis of LLM-generated package suggestions. Their methodology: prompt commercial code-completion models with realistic refactoring tasks, capture every imported package, and check each one against the relevant registry. The findings document hallucinated-package generation as a consistent failure mode across major models, with rates measurable enough to constitute a category of risk rather than an occasional bug.

The rate is higher in some contexts than others. Long-context refactoring (where the model fills in plausible-adjacent libraries from training memory) tends to produce more hallucinations than short single-file completions. Less-popular languages and frameworks produce higher rates than mainstream ones because the model has less ground-truth coverage. Newer libraries are more dangerous than older ones because the model’s training data lags reality.

What the research doesn’t tell you: the rate at your specific company, on your specific codebase, with your specific prompt practices. The only way to know is to instrument your AI-augmented PR pipeline with a hallucination check and measure.

Why Hallucinated Imports Are a Supply-Chain Risk

BrassCoders treats hallucinated imports as supply-chain vulnerabilities, not stylistic errors. The attack chain is straightforward: a malicious actor monitors AI-generated code (via published GitHub repositories, AI-tool usage research, or direct model behavior testing), identifies frequently-hallucinated package names, registers those names as typosquatting packages on PyPI or npm with embedded malware, and waits. When AI-generated code reaches a developer’s environment or a CI runner that does pip install or npm install, the typosquat package gets fetched and its install-time hooks execute.

This isn’t theoretical. PyPI’s security advisory feed and the npm security advisories database both track typosquatting as one of the largest attack surfaces in their ecosystems, with hundreds of confirmed malicious-package incidents per year predating AI tools. AI hallucinations accelerate the attack surface by providing the typosquatter with target names — instead of guessing what common typos developers make, the attacker watches the LLMs themselves and registers the names the LLMs invent.

The malware delivery, once you accept the attacker has the package up, can be anything that runs at install time: credential exfiltration from ~/.aws/credentials or ~/.ssh/, persistent backdoors, cryptominers, ransomware seeds. The Python ecosystem’s setup.py and the JavaScript ecosystem’s preinstall/install hooks both run arbitrary code at install time by design, and there’s no sandboxing. Snyk’s Vulnerability Database maintains an actively-updated catalog of confirmed malicious packages — a useful reality check for any team that hasn’t yet wired up supply-chain scanning.

Mitigation requires catching the hallucination before the install. By the time pip install runs, the attacker has already won — running the hallucination check at code review time (or as a pre-commit hook) is the only reliable defense.

How BrassCoders’s Check Works

BrassCoders’s package-hallucination check parses every import statement in your scanned files, extracts the package name, and queries the target registry — PyPI’s JSON API at pypi.org/pypi/<name>/json, npm’s registry at registry.npmjs.org/<name>, or Go’s pkg.go.dev/<name> — to verify the package exists. A non-200 response is a hallucination signal; a 200 means the package is registered on the registry.

The check is opt-in via the --check-package-hallucination CLI flag because it’s the only path in the OSS core that makes outbound network calls. BrassCoders’s default posture is offline-first: scans complete with zero network traffic unless you explicitly turn on this one check. The flag respects --offline: passing --offline overrides the opt-in back to off.

What gets sent over the wire: the bare package name (fastapi-users, lodash, github.com/spf13/cobra). No source code, no surrounding context, no project name, no telemetry. PyPI sees nothing different from a normal pip search query.

What gets returned: a JSON object from PyPI or npm if the package exists, a 404 if it doesn’t. BrassCoders discards the JSON content and only retains the existence signal. The package name plus the existence boolean are what go into the finding record.

What happens with private packages: if your code imports an internal package that isn’t published to the public registry (e.g. mycompany_auth), the check will flag it as hallucinated. This is a known false-positive pattern. Mitigation: pass --internal-packages mycompany_auth,mycompany_billing to whitelist names the check should skip. Or simply don’t enable the check on repos that import a lot of private packages.

How to Use It in CI

BrassCoders’s hallucination check fits into a CI pipeline as a pre-merge gate. Run brasscoders scan --check-package-hallucination against your branch and fail the build if any finding type matches HALLUCINATED_IMPORT. The added latency is 100-500ms per imported package depending on registry network latency, which usually adds 5-30 seconds to a typical scan.

A minimal GitHub Actions workflow:

name: BrassCoders scan
on: [pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pipx install brasscoders
      - run: brasscoders scan --check-package-hallucination .
      - run: |
          if grep -q "HALLUCINATED_IMPORT" .brass/security_report.yaml; then
            echo "Hallucinated package import detected. Fail the build."
            exit 1
          fi

The same shape works in GitLab CI, CircleCI, Jenkins, or any other runner — the only requirement is a Python 3.10+ environment and network access to PyPI/npm. The check shouldn’t run inside an air-gapped environment; pair it with regular brasscoders scan (without the flag) for those cases.

What about catching hallucinations at code-generation time, before they hit a PR at all? That’s the right end-state. Until your AI assistant of choice runs hallucination verification on its own output (and most don’t yet), the pre-merge scan is the load-bearing defense.

---

The supply-chain attack chain that hallucinated imports enable is the kind of thing that gets a security org calling at 11pm. The mitigation is a one-line CI step. The trade-off isn’t close.

Install BrassCoders with pipx install brasscoders and add the check to your PR pipeline. For the broader context on AI code review failure modes, see AI Code Review: The Practical Guide for 2026.