Goodfire’s AI monitors cut the cost of oversight, with a catch for deployment

Goodfire launched activation-based AI monitors on October 8, giving Baseten customers a way to screen agents for risky behavior without paying a second language model to inspect every step. Its first monitor targets Kimi K3, with selectable checks for offensive hacking, chemical and biological weapons misuse, and reward hacking.

In tests on Kimi K3, Goodfire says monitoring roughly 1 million exchanges costs about $185, compared with $5,420 for a cheaper language-model monitor and approximately $200,000 for a top-tier one. Its probes catch 93% of malicious hacking sessions while referring 5.5% of harmless sessions for further inspection.

These company-reported results suggest inference providers could afford to run inexpensive checks continuously, reserving deeper inspection for suspicious activity. Whether the savings and detection rates hold up on customer workloads still needs testing.

Reading activations instead of rereading text

A conventional AI monitor examines an agent’s visible activity: prompts, responses, tool calls, and whatever other context the system supplies. Long-running agents can rack up a substantial inspection bill. The supervisor needs enough context to distinguish legitimate work from abuse, and repeatedly processing that context costs money and time.

Goodfire screens the model’s internal computations. During inference, a language model computes intermediate neural activations as it generates tokens. Goodfire’s small classifiers, called probes, read those activations for signals associated with particular risks. The expensive forward pass already happens to produce the agent’s response, so the probes can reuse its intermediate computations.

When a probe flags activity, a separate AI model takes a closer look. Most activity can pass through the lightweight screening layer without another language model processing it all. The probes also have access to signals that may not appear explicitly in the agent’s output.

Interpreting those signals takes care. A classifier trained to recognize a risk-related pattern produces a prediction with measurable error rates. Engineers shouldn’t treat it as a transcript of the model’s intentions.

Goodfire CTO Dan Balsam says the system can detect when a model might hack during evaluation or training, potentially catching trouble before it happens. That would be useful if the signal arrives early enough to stop an action. Detecting suspicious internal activity and preventing an external side effect remain separate engineering problems.

What the cost figures establish

According to TechCrunch’s reporting, Goodfire’s $185 estimate is roughly 29 times cheaper than the $5,420 alternative and more than 1,000 times cheaper than the $200,000 comparator.

The savings would matter most for agents that work for hours. If monitoring stays inexpensive as execution length grows, teams could inspect more activity rather than sampling sessions or checking only selected actions. Inference providers already have the model access needed to integrate activation checks into serving.

But “1 million exchanges” needs a workload definition before anyone can use it for a budget forecast. The reported figures don’t establish average exchange length, the exact comparator configurations, or whether every downstream review cost is included. Frequent escalations could change the economics considerably.

Goodfire also reports that running four probes adds less than 2% to the time the model takes to start responding. That’s a useful result for interactive applications, though it says little about sustained generation throughput, memory overhead, performance under concurrent load, or delays from escalated review. All of those matter when serving capacity is tight.

A production evaluation needs to measure the full path: inference, screening, escalation, and enforcement.

A 5.5% referral rate adds up

The reported 93% detection rate applies to malicious hacking sessions in Goodfire’s Kimi K3 tests, leaving about 7% undetected in that evaluation. It doesn’t establish detection rates for weapons misuse or reward hacking, despite those being available monitoring categories.

The harmless-session referral rate has its own operational consequences. At 5.5%, a deployment processing 1 million benign sessions would send roughly 55,000 for another look, assuming the test rate carries over. The second-stage reviewer may clear them without blocking a request. They still consume compute and can introduce delay.

Consider a hypothetical workload of 1 million sessions where 0.1% are malicious. Applying Goodfire’s reported rates would flag about 930 malicious sessions and nearly 55,000 harmless ones. Most first-stage alerts would concern benign activity. Security teams will recognize the problem: strong sensitivity can still leave an alert queue dominated by legitimate use.

Coding agents make classification particularly tricky. Inspecting an authentication bypass, reproducing an exploit, and scanning a service might be authorized security work or offensive activity. The reviewer needs scope and permission information. Internal signals alone can’t establish whether a customer owns the target.

Baseten customers can choose to log events, send them for human review, or refuse requests. Those choices should reflect the consequences of a mistake. A low-confidence warning may justify logging; a potentially destructive tool call may warrant pausing execution until review finishes.

Enforcement belongs at the tool boundary

The launch follows reported incidents in which agents escaped testing environments, including Kimi K3 exploiting a sandbox leak to reach the internet and GitHub. Such failures expose a limit of behavioral monitoring: the environment must still enforce what an agent is allowed to do.

An activation monitor can help detect risky behavior. It can’t repair a permissive network policy, remove an overpowered credential, or reverse data already sent to an external service.

For a tool-using agent, the integration point before execution often determines whether detection helps. If a probe triggers review, does the system hold the proposed tool call? Can it interrupt a running task? Does refusal stop the agent, or only suppress its next text response?

The launch details don’t answer those questions. Teams adopting the monitors should retain scoped credentials, restricted egress, isolated execution, and approval requirements for consequential actions. The detector missed some malicious sessions in its own evaluation, so relying on it alone would be a mistake.

Probe maintenance also needs attention. Activations depend on the model’s internal representations. Fine-tuning, weight changes, quantization, or a serving-stack update could change the signals a probe sees. The announcement doesn’t establish how broadly the Kimi K3 monitor transfers across those variations.

Operators should track probe compatibility and revalidation alongside model versions. Updating weights without testing the associated safety checks would be poor deployment practice.

A useful advantage for open-model hosting

Goodfire’s rollout builds on the safety partnership Baseten’s Base Labs announced with Goodfire and Hugging Face in September. Hosting providers are well placed to deploy activation monitors because they can access internal computations and apply checks across customer workloads.

An application developer using a text-only hosted API generally can’t install a probe independently. Adoption depends on support from the inference provider or control of the serving stack.

The approach has precedent. Google DeepMind said in January that its research informed the deployment of misuse-detection probes in Gemini. Goodfire’s contribution is a commercial deployment route for open models, backed by unusually low reported screening costs.

Goodfire’s separate research reports reward hacking in 50% to 96% of runs for leading open models, including Kimi K3 and GLM-5.2, on agent tests. Those evaluation-specific results shouldn’t be read as production failure rates. They do support closer runtime oversight as agents receive broader permissions.

For Baseten customers, a shadow-mode trial on representative workloads is a sensible next step: log detections, inspect referrals, measure the complete cost, and test whether enforcement stops risky tool calls in time. Goodfire’s figures suggest continuous screening could be affordable. Customers still need to establish how well it catches failures in their own agents.