LiteLLM Supply Chain Attack - Full Analysis and Hands-On Lab

On March 24, 2026, the threat actor TeamPCP published compromised versions of LiteLLM (1.82.7 and 1.82.8) to PyPI. With 95 million monthly downloads, this was one of the largest supply chain attacks on the Python ecosystem. We obtained the real compromised packages, detonated them on an isolated EC2 instance, and captured every stage of the attack.

---

What's in this repo

Directory	Contents
malware-samples/	Real compromised .whl files + decoded 3-stage payload (orchestrator, harvester, persistence dropper)
lab/	Fully reproducible EC2 analysis lab - scripts, evidence, logs, screenshots
iocs/	Indicators of Compromise - file hashes, C2 domains, filesystem artifacts
timeline/	Full attack timeline and sources
articles/	Detailed technical articles

Read WARNING.md before touching any file in malware-samples/.

---

The attack in 60 seconds

1. March 1: Aqua Security (Trivy maintainer) breached. Credential rotation incomplete.
2. March 19: TeamPCP poisons Trivy via surviving credentials.
3. March 23: LiteLLM's CI/CD runs unpinned Trivy → PyPI tokens leak. Attacker registers litellm.cloud.
4. March 24, 08:30 UTC: Malicious litellm 1.82.7 and 1.82.8 uploaded to PyPI using stolen credentials.
5. March 24, 15:00 UTC: PyPI yanks compromised versions. Attack window: 6.5 hours.

What the malware does

pip install litellm==1.82.8
        │
        ▼
  litellm_init.pth lands in site-packages/
        │
        ▼
  ANY python command triggers it (no import needed)
        │
        ▼
  subprocess.Popen → fork → double base64 decode → 331-line harvester
        │
        ├──▶ Steals: SSH keys, AWS/GCP/Azure creds, K8s secrets, .env,
        │    databases, crypto wallets, Docker configs, npm tokens,
        │    Terraform state, CI/CD configs, shell history
        │
        ├──▶ Queries EC2 IMDS for IAM role credentials
        │    Dumps AWS Secrets Manager + SSM Parameter Store
        │
        ├──▶ Encrypts with AES-256-CBC + RSA-4096
        │    POSTs to https://models.litellm.cloud/
        │
        ├──▶ Deploys privileged K8s pods on every cluster node
        │
        └──▶ Installs persistence: ~/.config/sysmon/ + systemd service
             Polls C2 (checkmarx.zone) every 50 minutes

---

What we did - the lab

We set up an isolated EC2 instance (t3.medium, Ubuntu 22.04) with:

• Honeypot credentials planted in all standard locations (SSH, AWS, K8s, .env, etc.)
• mitmproxy intercepting all HTTPS traffic
• inotifywait watching every credential file for reads
• strace tracing every syscall
• tcpdump capturing raw packets

Then we installed the real compromised litellm==1.82.8 and triggered it.

Run 1: The fork bomb

The .pth file auto-executes on Python startup. Each execution forks a new Python process, which triggers the .pth again. Result:

Time	Python processes
T+0s	3 (normal)
T+1s	14
T+2s	55
T+3s	83
T+4s	133
T+8s	194
T+13s	390
T+18s	509
T+38s	891
T+40s	Machine dead. SSH hung.

This is exactly how the attack was originally discovered - a FutureSearch researcher's machine ran out of RAM.

To recover from the fork bomb:

1. Force-stop the instance from the AWS Console (SSH will be dead)
2. Start it again (it gets a new public IP)
3. SSH in and immediately delete the .pth bomb: rm ~/lab/malware-venv/lib/python3.10/site-packages/litellm_init.pth
4. The instance is now safe - run the decoded payload directly for controlled analysis

Run 2: Controlled detonation

After defusing the .pth fork bomb, we ran the decoded payload directly. mitmproxy captured the full attack chain:

1. GET  http://169.254.169.254/.../security-credentials/  → Steal IAM role (404 - no role)
2. PUT  http://169.254.169.254/latest/api/token            → Got IMDSv2 token!
3. CONNECT secretsmanager.us-east-1.amazonaws.com:443      → Dump Secrets Manager (TLS error - fake keys)
4. CONNECT ssm.us-east-1.amazonaws.com:443                 → Dump SSM Parameters (TLS error - fake keys)
5. CONNECT models.litellm.cloud:443                        → Exfiltrate to C2 (domain down)
6. GET  http://169.254.170.2/                              → Steal ECS credentials (timeout)

Every honeypot credential was stolen - filesystem logs show id_rsa, id_ed25519, .aws/credentials, .git-credentials, .env, .kube/config all opened and read within 3 seconds.

---

Reproduce the lab

Everything you need is in lab/:

# 1. Launch the EC2 lab (creates instance, installs tools, plants honeypots, uploads malware)
bash lab/scripts/launch-lab.sh

# 2. SSH in
ssh -i ~/.ssh/litellm-lab.pem ubuntu@<PUBLIC_IP>

# 3. Start monitors
bash ~/lab/scripts/monitor-all.sh

# 4. Detonate
bash ~/lab/scripts/detonate.sh

# 5. Analyze
mitmproxy -r ~/lab/captures/traffic-run2.flow
cat ~/lab/logs/filesystem.log
cat ~/lab/logs/strace-run2.log

# 6. Destroy
bash lab/scripts/destroy-lab.sh

Full step-by-step guide: lab/LAB-COMMANDS-LOG.md

---

Key evidence (from our lab)

Evidence	What it proves
mitmproxy.log	Full HTTPS traffic: IMDS queries, AWS API calls, C2 exfiltration attempt
filesystem.log	Every credential file read with timestamp
processes.log (run1)	Fork bomb: 3→891 python processes in 38 seconds
strace extracts	Syscall-level proof of file reads and network connections
syslog	Kernel OOM listing hundreds of python3 processes
Screenshots	mitmproxy captures, htop during attack, AWS Console, decoded payloads

---

Indicators of Compromise

Domains:

• models.litellm.cloud - exfiltration endpoint
• checkmarx.zone - C2 for persistence dropper

File hashes (SHA-256):

• 8395c326... - litellm 1.82.7 wheel
• d2a0d5f5... - litellm 1.82.8 wheel
• 71e35aef... - litellm_init.pth

Filesystem artifacts:

• site-packages/litellm_init.pth
• ~/.config/sysmon/sysmon.py
• ~/.config/systemd/user/sysmon.service
• /tmp/pglog, /tmp/.pg_state

Kubernetes:

• Pods named node-setup-* in kube-system namespace

Full IOC list: iocs/

---

Articles

• Part 1: Anatomy of a Supply Chain Attack: How LiteLLM Was Weaponized in 6 Hours
• Part 2: We Detonated the Real LiteLLM Malware on EC2 - Here's What Happened

---

Published by

TocConsulting

---

License

This repository is provided for educational and defensive security purposes only. See WARNING.md.