LiteLLM Got Hacked. Here's Your AI Supply Chain Audit Checklist.

LiteLLM — the open-source universal LLM proxy that thousands of AI applications depend on — just had its "SolarWinds moment."

On March 24, 2026, security researchers discovered that litellm==1.82.8 (and likely 1.82.7) on PyPI contained a credential-stealing payload that exfiltrated SSH keys, AWS credentials, Kubernetes secrets, environment variables, shell history, and even crypto wallet files to an attacker-controlled server. The malicious code didn't require importing LiteLLM — it executed automatically the moment Python started, thanks to a .pth file injected into the package.

The attack vector? A poisoned Trivy dependency in LiteLLM's CI/CD pipeline that leaked the project's PYPI_PUBLISH token. The attacker used that token to push compromised versions directly to PyPI.

596 points and 244 comments on Hacker News. And the irony is thick: the tool everyone uses to abstract away LLM complexity became a single point of failure for the entire AI middleware stack.

What Actually Happened: The Kill Chain

Here's the attack chain, reconstructed from the GitHub security issue and the maintainer's response on HN:

Step 1: Trivy Compromise. The attacker compromised a version of Trivy — a popular container vulnerability scanner — that LiteLLM used in its CI/CD pipeline. This is the upstream attack: infect a security tool to access the targets that trust it.

Step 2: PYPI_PUBLISH Token Exfiltration. The poisoned Trivy variant extracted the PYPI_PUBLISH token stored as an environment variable in LiteLLM's GitHub CI pipeline. This token had enough permissions to push new package versions to PyPI.

Step 3: Malicious Package Publication. Using the stolen token, the attacker published litellm==1.82.8 (and modified 1.82.7) containing a file called litellm_init.pth.

Step 4: Automatic Execution via .pth. Here's the clever part. Python's site module automatically executes .pth files found in site-packages/ on interpreter startup. No import litellm required. If the package was installed, the payload ran every time Python started — including in CI/CD runners, Docker containers, and production servers.

Step 5: Credential Harvesting. The payload — double base64-encoded to evade naive scanning — collected:

• All environment variables (API keys, database passwords, tokens)
• SSH keys (private keys, authorized_keys, known_hosts)
• Cloud credentials (AWS, GCP, Azure, Kubernetes configs)
• Git credentials and Docker configs
• Shell history (bash, zsh, mysql, psql, redis)
• Crypto wallets (Bitcoin, Ethereum, Solana, and more)
• SSL/TLS private keys
• CI/CD secrets (Terraform, GitLab CI, Jenkins, Drone)

Step 6: Encrypted Exfiltration. The harvested data was encrypted with AES-256 (random session key), the session key was encrypted with a hardcoded RSA-4096 public key, and the package was exfiltrated to models.litellm.cloud — note: NOT the legitimate litellm.ai domain.

Who's Affected

If you're in the AI/ML space, the blast radius is significant:

• Any team using LiteLLM as an LLM proxy — LiteLLM is the go-to tool for routing requests across OpenAI, Anthropic, Cohere, and dozens of other providers. It's in thousands of production stacks.
• CI/CD pipelines that install LiteLLM — Docker builds, GitHub Actions, GitLab CI runners that pip install litellm during the affected window.
• Development machines — Any developer who ran pip install litellm or uv add litellm and got version 1.82.7 or 1.82.8.
• Downstream dependencies — Any package that lists litellm as a dependency and pulled the compromised version during a build.

The attack window was limited (the versions were yanked quickly), but the damage model is binary: if you installed the affected version, all secrets on that machine were exfiltrated.

Your AI Stack Audit Checklist

Here's the practical part. Whether or not you use LiteLLM, this attack exposes patterns that apply to every AI stack.

1. Check If You're Directly Affected

# Check installed version
pip show litellm 2>/dev/null | grep Version

# Check for the malicious .pth file
find $(python3 -c "import site; print(site.getsitepackages()[0])") \
  -name "litellm_init.pth" 2>/dev/null

# Check pip install history / requirements files
grep -r "litellm" requirements*.txt pyproject.toml setup.py Pipfile 2>/dev/null

If you find litellm_init.pth or had version 1.82.7 or 1.82.8 installed at any point, assume full credential compromise and proceed to step 2.

2. Rotate Everything — No Exceptions

If you were affected, rotate credentials in this order (highest risk first):

1. Cloud provider credentials — AWS access keys, GCP service accounts, Azure service principals
2. PyPI / npm / registry tokens — to prevent the attacker from publishing on your behalf
3. SSH keys — regenerate all key pairs, update authorized_keys on all servers
4. Database passwords — especially if they were in environment variables
5. API keys — every LLM provider key (OpenAI, Anthropic, Cohere, etc.), Stripe, Twilio, everything
6. Kubernetes secrets — rotate and re-deploy
7. Git credentials — regenerate personal access tokens

3. Pin Dependencies and Verify Hashes

This is the single most impactful change most AI teams aren't doing:

# pyproject.toml — pin EXACT versions with hashes
[project]
dependencies = [
    "litellm==1.82.6",  # Known good version — NEVER use >= or ~=
    "openai==1.68.0",
    "anthropic==0.49.0",
]

Better yet, use pip-compile with hash checking:

# Generate locked requirements with hashes
pip-compile --generate-hashes requirements.in -o requirements.txt

# Install with hash verification
pip install --require-hashes -r requirements.txt

Or with uv:

# uv lock generates hashes automatically
uv lock
uv sync

4. Isolate CI/CD Secrets

The root cause of this attack was a PYPI_PUBLISH token stored as a broad CI/CD environment variable accessible to every step in the pipeline — including Trivy, which had no business seeing it.

Fix this:

# GitHub Actions — BAD: token available to all steps
env:
  PYPI_TOKEN: ${{ secrets.PYPI_PUBLISH }}

# GitHub Actions — GOOD: token only in publish step
jobs:
  test:
    steps:
      - run: pytest  # No access to PYPI_TOKEN
  
  publish:
    needs: test
    environment: pypi-publish  # Separate environment with approval gates
    steps:
      - uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_PUBLISH }}

Even better: use PyPI Trusted Publishers which use OIDC tokens instead of long-lived API keys. No token to steal.

5. Audit Your AI-Specific Dependencies

AI stacks have uniquely deep dependency trees. A typical LLM application might pull in 200+ transitive dependencies:

# Count your transitive dependencies
pip install pipdeptree
pipdeptree -p litellm | wc -l

# Scan for known vulnerabilities  
pip install pip-audit
pip-audit

# For uv users
uv pip audit

Pay special attention to:

• LLM client libraries (openai, anthropic, cohere, together) — high-value targets
• Vector databases (chromadb, pinecone-client, weaviate-client)
• ML frameworks (torch, transformers, diffusers) — enormous dependency trees
• Eval/monitoring tools (langsmith, langfuse, promptfoo)

If you're building with AI, you should know about the best AI APIs and their security postures — not all providers handle credential management equally.

6. Consider Alternative LLM Routing

LiteLLM isn't the only LLM proxy. If this attack shakes your confidence, evaluate alternatives:

For many teams, the answer might be simpler than a proxy swap: just use the provider SDKs directly. If you're only using 2-3 models, a thin abstraction layer in your own code is fewer dependencies, fewer attack surfaces, and code you control.

7. Implement Secret-Memory Isolation for Agents

This attack highlights a broader issue: AI agents that handle credentials need proper secret isolation. The NanoClaw Agent Vault — which hit HN the same day as the LiteLLM compromise — represents the emerging approach: agents can act on your behalf without raw credential access.

The principle: agents should never see plaintext secrets. Credentials live in a vault. The agent requests actions (not keys), and the vault executes authenticated API calls on the agent's behalf. If the agent's context window is compromised — or the underlying package is compromised — the secrets aren't there to steal.

This is the same principle that makes AI agents replacing SaaS both exciting and terrifying: more autonomous agents mean more credential surface area to protect.

Dustin Ingram from the Python Software Foundation walks through PyPI's supply chain security model — the exact infrastructure that was exploited in this attack. Required viewing if you publish or consume Python packages.

The Bigger Picture: AI's Supply Chain Problem

This isn't just a LiteLLM problem. The AI ecosystem has a structural vulnerability that traditional software didn't:

AI stacks are dependency-heavy by nature. A typical web app might have 50-100 transitive dependencies. A typical AI application — with model clients, vector databases, eval frameworks, and inference engines — can have 300+. Each one is an attack surface.

AI packages handle secrets by default. Unlike a CSS library or a date formatting utility, AI packages routinely handle API keys, model endpoints, and user data. A compromised AI package isn't just running arbitrary code — it's running arbitrary code in an environment rich with high-value credentials.

The "move fast" culture compounds the risk. The AI space moves at breakneck speed. New model providers, new frameworks, new tools — weekly. Teams adopt packages quickly, often without security review. The same urgency that makes AI exciting makes it vulnerable.

Understanding how LLM proxies and gateways work is essential context for evaluating whether you need one — and how to secure it if you do.

The OpenAI acquisition of Astral (the team behind uv and Ruff) adds another dimension: when your package manager is owned by an AI company, the lines between "tool" and "attack surface" blur further. Not because OpenAI is malicious — but because concentration of control in the Python toolchain means a single compromise has wider blast radius.

What Comes Next

The LiteLLM maintainers are handling this transparently — Krrish's HN updates have been refreshingly human ("I'm sorry for this") compared to the usual corporate crisis-speak. They've deleted impacted versions, rotated all keys, and are scanning for additional compromise vectors.

But the broader lesson isn't about LiteLLM. It's about the AI industry growing up on security:

1. PyPI Trusted Publishers should be mandatory for any package with >10K weekly downloads
2. CI/CD secret isolation needs to be a first-class concern, not an afterthought
3. Dependency hash verification should be the default, not an opt-in
4. Agent credential management needs purpose-built solutions like vault-based isolation
5. Build artifacts, not live installs — your production systems should never depend on PyPI being up or uncompromised

If you're building AI applications — especially with coding assistants that install packages on your behalf — this is the wake-up call. The AI stack is a high-value target. Secure it like one.

This article will be updated as the LiteLLM team publishes their full postmortem. Follow the GitHub issue tracker for real-time updates.

For more on building secure AI applications, see our guides on AI safety and ethics, the best AI APIs for developers, and running AI locally to reduce your attack surface.