LiteLLM drops Delve after a malware scare, and AI gateway teams should pay attention

LiteLLM is moving away from compliance startup Delve and redoing its certifications with Vanta plus an independent auditor, after a rough stretch that included a credential-stealing malware incident tied to its open source project.

That matters because LiteLLM sits in a sensitive part of the stack. It’s a widely used AI gateway for routing traffic across providers like OpenAI, Anthropic, Azure, and Bedrock. If you run one of these gateways, you’re handling API keys, prompts, outputs, tenant metadata, logs, and usually a messy layer of retries, caching, tracing, and access controls. A compromise there can leak a lot, fast.

There’s a second issue too. LiteLLM’s move is a fairly blunt rejection of the way some startups have sold compliance as a shortcut. Security teams were already wary of vendors that blurred the line between collecting evidence and validating it. Now they have fresh reason to be.

Why this hit hard

According to public statements from LiteLLM leadership on March 30 and April 1, the company plans to leave Delve, adopt Vanta for compliance automation, and use its own independent auditor for re-certification. That came after allegations that Delve misled customers with questionable audit evidence and auditor relationships that looked uncomfortably close. Delve’s founder denied the accusations and offered free re-tests, but the damage was already done. A whistleblower added more alleged receipts over the weekend, which made things worse.

On its own, that would already be ugly. But LiteLLM had also just dealt with a malware incident in its open source project, reportedly aimed at stealing credentials. Once there’s an active security incident, badges and certificates stop looking like procurement paperwork. People start asking whether the underlying assurance was real.

For LiteLLM, switching vendors and rerunning the process was the only serious option.

AI gateways are a soft target

A lot of teams still talk about AI gateways as thin proxies with decent dashboards. That undersells the risk.

A LiteLLM-style gateway usually does all of this:

• normalizes APIs across multiple model providers
• routes requests and handles failover
• enforces quotas and rate limits
• stores or brokers provider API keys
• logs prompts, outputs, token counts, latency, and traces
• scrubs some sensitive data, or says it does
• sometimes caches model responses
• often plugs into vector stores or app middleware

That’s a lot of trust concentrated in one layer. If malware gets into the codebase, build process, dependency tree, or shipped artifacts, there are plenty of places to pull useful material from. Environment variables. CI tokens. Cloud credentials. Provider keys. Local config files. Cached requests. Session data from maintainers’ laptops.

Even when production is locked down reasonably well, open source infrastructure still leans on developer endpoints, CI runners, package registries, and release pipelines. Those are often softer targets.

There’s no public root-cause report for the LiteLLM incident, so guessing at the exact entry point would be sloppy. The usual paths are familiar enough:

• typosquatted or malicious dependencies in npm or PyPI
• dependency confusion against private namespaces
• compromised GitHub Actions or CI tokens
• poisoned release artifacts or container images
• malicious pull requests hidden behind obfuscated code or install hooks

None of this is exotic. Serious infrastructure teams know the risks. Plenty still act like a passed audit answers them.

It doesn’t.

Compliance tooling has a place

It does not prove software integrity.

Security buyers should stop acting like this is murky.

Platforms like Vanta, Drata, and Secureframe are automation products. They collect evidence, track controls, run checks, and help teams keep a SOC 2 or ISO 27001 program in order. That’s useful. It saves time. It also makes life easier for small security teams.

But those products aren’t the audit.

A proper SOC 2 Type II still depends on an independent CPA firm reviewing the controls and testing whether they actually operated over time. ISO 27001 certification still depends on an accredited certification body and a functioning ISMS, with internal audits, surveillance, and risk management. If the same commercial relationship starts shaping both the evidence collection and the supposedly independent judgment, the whole exercise gets shaky.

That’s why LiteLLM’s new setup matters. Vanta handles the operational plumbing. A separate auditor handles the formal validation. Basic separation of duties. Teams should have demanded that from the start.

The market got lazy here. Startup buyers liked compliance sold as a neat package because it simplified procurement. Founders liked it because it produced reports and logos quickly. Procurement teams often treated the PDF as the finish line. Convenient, yes. Serious, no.

Ask for controls you can inspect

If you’re evaluating an AI gateway right now, ask for software supply chain evidence before you get impressed by certification badges.

A useful short list:

• signed releases with sigstore or cosign
• build provenance through in-toto attestations
• a credible path toward SLSA-aligned build practices
• SBOMs published per release and checked in CI
• dependency pinning with hashes
• OIDC-based short-lived credentials in CI instead of long-lived secrets
• strict CI and production egress controls
• mandatory 2FA, signed commits, and token scoping for maintainers

Those controls can be inspected.

If a vendor says it secures its release pipeline, ask whether deploys verify signatures before running containers. If it says it manages dependencies carefully, ask whether pip install --require-hashes or equivalent integrity checks are standard practice. If it says its CI is locked down, ask whether GitHub Actions still has standing cloud credentials sitting in secrets.

A lot of teams will answer with some version of “we’re working on it.” Fine. That’s at least honest. The important part is whether they understand the gap and have a credible plan to close it.

Why this stings for AI middleware vendors

AI infrastructure has a trust problem that many teams still downplay.

A model gateway sits between user traffic and upstream model providers. That means it sees raw prompts, completions, token usage, routing logic, and tenant-level metadata. In enterprise deployments, those prompts can include PII, internal code, legal material, support transcripts, proprietary instructions, and the usual pile of operational junk that ends up in chat workflows.

So when an AI gateway gets hit, the blast radius can include:

• provider API keys that attackers can use for access or billing fraud
• customer prompts and outputs
• internal model routing policies
• cached completions containing sensitive context
• logs and traces that were supposed to be scrubbed but often aren’t

The line that it was “just the open source project” won’t calm many buyers. Open source is often upstream of the hosted product, the self-hosted install, or both. If trust in the development pipeline takes a hit, enterprise customers won’t politely separate those concerns.

They’ll pause evaluations, send longer security questionnaires, or move to a vendor that can answer harder questions without fumbling.

What teams should change now

If you build on LiteLLM, use a similar gateway, or run your own, this is a good time to tighten the boring controls that actually limit damage.

Start with the supply chain:

• verify signatures on containers and release artifacts before deployment
• publish an SBOM for every release
• pin dependencies with hashes
• use ephemeral CI runners where possible
• move CI auth to cloud OIDC and delete long-lived keys
• lock down outbound network access from CI jobs

Then deal with secrets:

• store provider keys in a proper KMS or vault
• rotate them on a schedule and after contributor offboarding
• scope them per tenant or per service instead of using one giant shared secret
• treat logs as sensitive data stores, because that’s usually what they become

And if you’re buying rather than building, push vendors on specifics. Ask who performs the audit. Ask whether the auditor is selected independently. Ask for evidence of artifact signing, provenance, and release hygiene. Ask how they protect maintainer accounts and release pipelines. Vague answers are still answers.

The correction is already happening

This episode will probably push more AI infrastructure vendors to clean up their release process. Good. Some were overdue. Open source popularity and enterprise trust were never the same thing.

It will also make the compliance startup market more uncomfortable. Also good. Too many buyers treated automation software as if it could manufacture assurance on its own. It can’t. At best, it helps organize proof. The rest still depends on independence, process quality, and technical controls that hold up when someone actually tries to break in.

LiteLLM’s switch to Vanta plus a separate auditor won’t erase the malware incident. It does show the company understands what credibility has to look like afterward. For teams building AI middleware, that’s the part worth watching. The bar moved up a little. It needed to.