LiteLLM’s supply chain was compromised and budget enforcement fails silently under concurrent load

What you expect

LiteLLM is a widely-used Python SDK and AI gateway providing unified OpenAI-format access to 100+ LLM providers. With 40K+ stars and adoption at Stripe, Google, and Netflix, the expectation is that PyPI packages are clean, security posture is production-grade, and budget enforcement reliably caps spend.

What actually happens

Supply chain attack via compromised Trivy scanner (March 24, 2026)

PyPI versions 1.82.7 and 1.82.8 contained a credential-exfiltrating malware payload. The attack vector was a compromised Trivy CI/CD scanner that injected a .pth file (litellm_init.pth) into the package build. .pth files execute on every Python startup — not just import litellm — meaning any environment that had these versions installed was affected on all Python process starts, not only when LiteLLM was used directly.

The payload exfiltrated:

• Environment variables (API keys, tokens, secrets)
• SSH keys from ~/.ssh/
• AWS, GCP, and Azure credentials
• Kubernetes tokens
• Any other secrets accessible in the runtime environment

Exfiltration target: models.litellm.cloud, a fraudulent domain controlled by the attacker. Per LiteLLM’s security blog, the compromised packages went live on March 24, 2026 at 10:39 UTC and were available on PyPI for roughly 40 minutes before PyPI quarantined them. Mandiant forensics was engaged. The attack is attributed to the TeamPCP threat actor, the same group behind prior supply chain attacks against open source packages (Snyk analysis).

The official LiteLLM Docker image was not affected (pinned dependencies). Safe version ranges: <=1.82.6 and >=1.82.9.

Seven security advisories in 35 days (April–May 2026)

A coordinated disclosure batch produced 7 GitHub Security Advisories. Two are Critical severity:

Advisory	Severity	Impact
GHSA-jjhc-v7c2-5hh6	Critical	Authentication bypass via OIDC userinfo cache key collision
GHSA-53mr-6c8q-9789	High	Privilege escalation via unrestricted proxy configuration endpoint
GHSA-69x8-hrgq-fjj8	High	Password hash exposure and pass-the-hash authentication bypass
GHSA-xqmj-j6mv-4862	High	Server-Side Template Injection in /prompts/test endpoint
GHSA-r75f-5x8p-qvmc	Critical	SQL injection in Proxy API key verification
GHSA-v4p8-mg3p-g94g	High	Authenticated command execution via MCP stdio test endpoints
GHSA-wxxx-gvqv-xp7p	High (CVSS 8.8)	Sandbox escape in custom-code guardrail (v1.81.8–v1.83.9; patched v1.83.10+)

The sandbox escape (GHSA-wxxx-gvqv-xp7p) affects the POST /guardrails/test_custom_code endpoint and enables arbitrary code execution via bytecode-level techniques. Requires proxy-admin credentials but no physical access. Fixed by replacing the custom sandbox with RestrictedPython in v1.83.10+. CVSS 8.8.

The MCP stdio advisory (GHSA-v4p8-mg3p-g94g) is specifically relevant for builders using LiteLLM as an MCP gateway — the stdio test endpoint allows authenticated command execution.

Budget enforcement loses 75–93% of counter increments under concurrent load

Issue #20977 (closed as not_planned as of May 2026) documents a race condition in the cache increment path that underlies budget counter, TPM/RPM, and rate-limit tracking. Under 4+ concurrent threads with 50 increments each (200 total expected), actual recorded increments are approximately 51 — a 75–93% loss rate (#20977). At 16 concurrent workers, the in-memory rate limit cache shows 92.8% data loss. At 500 concurrent sessions, budget enforcement is effectively non-functional.

The fix (PR #20979 — atomic cache increments via threading.RLock + diskcache.transact()) remained open and unmerged as of May 2026, blocked by an unresolved reviewer concern.

Multi-replica deployments compound this: each replica maintains independent spend counters. There is no cross-replica coordination. A customer with a $100 cap running against 5 replicas can spend up to $500 before any single replica triggers enforcement.

No exception is raised when the budget is exceeded due to race conditions — the gateway silently accepts traffic beyond the configured cap.

Upstream Retry-After header is not forwarded to clients

Issue #21553 (closed as not_planned as of May 2026) documents that when an upstream provider returns HTTP 429 with a Retry-After header, the LiteLLM proxy forwards it only under the prefixed name llm_provider-retry-after and omits the bare Retry-After header. Clients using the OpenAI SDK pointed at the proxy expect the bare header, so they never see the provider’s timing and do not back off — amplifying load on the rate-limited provider. PR #21648 (fix attempt) was closed unmerged.

What this means for you

Any Python environment that installed LiteLLM v1.82.7 or v1.82.8 — even transiently — should be treated as potentially compromised. The .pth payload ran on every Python startup, not just when LiteLLM was imported. This means credentials in that environment (API keys, SSH keys, AWS/GCP/Azure credentials, K8s tokens) may have been exfiltrated to an attacker-controlled endpoint, regardless of whether LiteLLM was actively being used at the time.

For operators running LiteLLM in production: the budget enforcement race condition means any concurrency above 4 threads renders stated spend limits unenforceable. Budget caps in multi-replica deployments multiply the exposure by the replica count. This is not a corner-case edge condition — it’s the normal production operating mode for any team using LiteLLM at meaningful scale.

The seven security advisories represent a pattern, not a one-time incident: 2 Critical and 5 High severity vulnerabilities disclosed over ~34 days (April 3–May 7, 2026), with coordinated disclosure ongoing as of the time of writing.

What to do

1. Check your install history: Run pip show litellm and verify the version is not 1.82.7 or 1.82.8. Check pip freeze logs from March 2026 deployments. If either version was ever installed, rotate all credentials that were accessible in that environment.

2. Pin to safe versions: Use litellm>=1.82.9 in your dependencies. Safe pre-compromise ranges: <=1.82.6 or >=1.82.9. Official Docker images were safe.

3. Patch to v1.83.10+ for the sandbox escape: If using the custom-code guardrail (POST /guardrails/test_custom_code), patch to v1.83.10-stable or later. The sandbox escape (GHSA-wxxx-gvqv-xp7p) requires proxy-admin credentials but enables arbitrary code execution.

4. Do not rely on in-memory budget enforcement above 4 concurrent threads: Use an external shared state (Redis) for budget counters if your deployment runs more than 4 concurrent LiteLLM threads. Multi-replica deployments require a shared budget store regardless of concurrency level — independent replica counters cannot enforce a shared cap.

5. Treat rate limit budgets as soft limits in production: Until PR #20979 (the cache increment atomicity fix) merges, budget enforcement is statistically unreliable under real production concurrency. Build cost alerting at the provider billing level, not at the LiteLLM proxy level.

6. Add retry-backoff protection at the load balancer: Since the LiteLLM proxy does not forward the bare Retry-After header to clients, SDK clients will not naturally back off on provider rate limiting. Add jitter and maximum-delay bounds at the infrastructure layer (ALB, nginx), or read the llm_provider-retry-after header explicitly, rather than relying on the standard SDK retry path.

Falsification criterion: This finding would be disproved by evidence that the advisory GHSAs were erroneously issued and rescinded, that the compromised packages were not distributed through PyPI (contradicted by Snyk and LiteLLM’s own blog), or that issue #20977 was resolved and the fix confirmed by independent testing showing correct counter atomicity under concurrent load.

Evidence

Tool	Version	Evidence	Result
BerriAI/litellm	v1.82.7–v1.82.8	independently-confirmed	Supply chain attack via compromised Trivy scanner — litellm_init.pth payload exfiltrated env vars + credentials to models.litellm.cloud; TeamPCP attribution; 3.4M daily downloads affected; Snyk analysis
BerriAI/litellm	All proxy deployments	independently-confirmed	GHSA-jjhc-v7c2-5hh6 (Critical) OIDC auth bypass; GHSA-r75f-5x8p-qvmc (Critical) SQL injection in API key verification — published April 2026
BerriAI/litellm	v1.81.8–v1.83.9	independently-confirmed	GHSA-wxxx-gvqv-xp7p (High, CVSS 8.8) — sandbox escape in custom-code guardrail; arbitrary code execution via bytecode; patched in v1.83.10+
BerriAI/litellm (budget race)	v1.83.x	source-reviewed	Issue #20977 — 75–93% cache-increment loss (budget/TPM/RPM tracking) under 4+ concurrent threads; 200 expected increments, ~51 recorded; closed as not_planned, May 2026
BerriAI/litellm (budget fix)	v1.83.x	source-reviewed	PR #20979 — fix unmerged as of May 2026; blocked by reviewer concern
BerriAI/litellm (Retry-After)	v1.81.9	source-reviewed	Issue #21553 — upstream provider Retry-After header forwarded only as llm_provider-retry-after, bare header omitted, so SDK clients don’t back off; closed as not_planned, May 2026

Confidence: empirical — 4 environments reviewed, 3 independently confirmed via published GHSAs and LiteLLM’s own security blog. Independent confirmation via Snyk analysis and 7 published GitHub Security Advisories.

Strongest case against: The supply chain attack window was narrow (~40 minutes before PyPI quarantine, per LiteLLM’s security update), meaning most developers on a given machine would not have pulled the compromised versions unless their dependencies were re-resolved in that window. LiteLLM’s response was rapid (Mandiant engaged, packages quarantined, blog post same-day), and the affected version range is narrow and well-documented. The budget enforcement race condition may only matter at concurrency levels above 4 threads, which is not a constraint for small-scale deployments. Some of the security advisories require proxy-admin credentials, limiting the attack surface to authenticated endpoints. Teams running the official Docker image were not affected by the supply chain attack.

Open questions: Has PR #20979 (budget atomicity fix) merged in a post-v1.85.0 release? Are there additional advisories beyond the 7 published through May 7, 2026 (disclosure process was stated as ongoing)? Does the custom anthropic_tokenizer.json introduce billing discrepancies vs. Anthropic’s native token-counting endpoint?

Seen different? Contribute your evidence — share a repro or counter-example and we’ll review it against this finding. Reader evidence is what keeps these findings accurate.