Meta’s Scale AI bet already looks messy, and that says a lot about how frontier models get built

Meta put roughly $14.3 billion into Scale AI in June and brought Scale founder Alexandr Wang, plus several executives, into its new Meta Superintelligence Labs. The logic was clear enough: get closer to a major data supplier, move faster on model development, and try to close ground on OpenAI, Google DeepMind, and Anthropic.

A few months later, it already looks awkward.

At least one Scale executive who followed Wang to Meta, Ruben Mayer, has left. Mayer says he departed for personal reasons and disputes some reporting around his role, but the timing still matters. Meta’s core AI group, reportedly called TBD Labs, is also said to be buying data from other vendors, including Surge and Mercor, both major Scale rivals. Some researchers reportedly prefer the quality they’re getting there. Meta disputes that Scale has a quality problem.

That quality fight matters more than the executive churn. It points to the harder truth underneath this whole market: frontier AI training now leans on a supply chain that’s expensive, fragile, and hard to standardize. A lot of it comes down to expert human supervision.

The bottleneck moved

For years, the AI scaling story was simple. Bigger models, more GPUs, more web data. Then the obvious limits showed up. Public internet text is noisy. Web-scale pretraining gives you a broad base model, but it doesn’t reliably produce systems that reason well, use tools cleanly, follow nuanced policies, or hold up in high-stakes domains.

The next gains come from post-training data.

That usually means things like:

• step-by-step reasoning traces for math, code, science, and planning
• multi-turn tool-use sessions with function calls, retrieval, browser actions, and code execution
• safety and red-team examples with actual policy nuance
• multimodal supervision tying text to images, audio, and video with solid temporal and spatial alignment
• domain-specific annotations from people who know law, medicine, finance, or cybersecurity

That’s a very different business from old-school data labeling. Bounding boxes and sentiment tags can be pushed through huge contractor pools and majority voting. Reasoning supervision can’t. You do not want three random annotators averaging their way toward a proof, a malware triage task, or a differential diagnosis.

You need expert labor, clear rubrics, layered review, and traceability. That costs more and moves slower. It also means vendor quality stops being a commodity claim and starts affecting model performance.

Why Meta is still shopping

Meta’s behavior makes sense. Owning part of a data vendor doesn’t remove the need to test other vendors.

If Surge or Mercor are delivering better coding traces, cleaner preference data, or tighter expert review in some domain, Meta would be reckless to ignore that because it owns part of Scale. That matters even more if its own model work is under pressure. Llama 4 reportedly disappointed internally. The company has been reshuffling leadership, recruiting hard, and spending heavily on infrastructure, including the reported $50 billion Hyperion data center project in Louisiana.

At that scale, bad supervision data gets expensive fast.

A weak SFT corpus puts a low ceiling on everything that follows. If the rationales are shallow or the annotations are inconsistent, downstream methods like DPO, IPO, KTO, or reward-model tuning can end up optimizing junk preferences. You can waste a lot of GPU time polishing the wrong target.

That’s why engineers should care about the vendor argument. It affects training signal quality.

What “good data” actually means

Labs talk about “high-quality data” constantly and usually leave it there. In frontier post-training, the term should mean something pretty specific.

First, the rubric has to be good. If you’re collecting reasoning traces, reviewers need decomposed criteria: logical soundness, completeness, whether errors are localized, whether a tool was used appropriately, whether the final answer is verifiable. “Looks good” isn’t a rubric.

Second, agreement needs to be measured properly. Raw agreement is weak. Teams that take annotation quality seriously track metrics like Krippendorff’s alpha or Cohen’s kappa, often with graded scoring instead of binary labels, because many tasks are genuinely ambiguous.

Third, you need gold sets and calibration. Seed the queue with examples whose outcomes are already known. Watch how annotators score them. Recalibrate often. Drift is constant, especially when the task mix changes or model-assisted workflows start shaping reviewer behavior.

Fourth, provenance matters. If you can’t trace an example back to annotator_id, rubric_version, trace_id, source license, and the model used to draft or pre-score it, audits and reproducibility get much harder. For teams shipping into enterprise or regulated environments, that’s a product risk.

The training stack now looks a lot like data engineering, labor ops, and evaluation science tied together. “Collect labels, train model” is an outdated picture.

Model-assisted labeling can quietly poison the data

Most large labs already use model-assisted labeling. A strong model drafts an answer, and a human expert edits it, scores it, or adds an explanation. It’s efficient. It also creates a nasty failure mode.

If the assisting model is mediocre in a domain, annotators can get anchored by bad suggestions. If teams don’t track the model version or control how much its output shapes final labels, they can build datasets that inherit the assistant’s blind spots. That shows up later as stylistic overfitting or brittle reasoning.

You can see the pattern when models start rewarding verbosity because verbose answers dominated the “good” examples, or when they learn to imitate polished rationales that sound smart but don’t solve hard tasks. Benchmarks like SWE-bench, GPQA, MMLU-Pro, ARC-AGI, and strong math evals expose that pretty quickly.

Synthetic data helps. Self-play helps. Distillation helps. You still need human anchors with real domain expertise.

Scale’s problem goes past Meta

The Meta relationship also seems to have made Scale less attractive to other major customers. Reporting says OpenAI and Google stopped working with Scale after Meta invested. That tracks. Once a key supplier is financially and strategically tied to a rival frontier lab, neutrality becomes hard to trust, no matter what formal walls exist on paper.

Scale has also reportedly laid off around 200 people in data labeling and shifted attention toward government work, including a $99 million U.S. Army contract. From a business perspective, that may be sensible. Government contracts bring money and a different kind of defensibility. But it also suggests the commercial frontier model market is getting tougher.

The market is splitting.

One side wants elite expert networks, strong security posture, auditability, and high-end post-training work. The other still wants cheaper, high-volume annotation. Those are different businesses. A vendor trying to do both can end up muddled.

What engineering teams should take from it

If you run model training, evals, or AI product quality, Meta’s situation is a useful warning.

Single-vendor dependence is risky. Multi-vendor data pipelines are becoming standard for a reason. Teams want bake-offs, overlapping scopes, and domain specialization. One provider may be best at coding traces, another at multilingual safety reviews, another at medical annotation under tight compliance rules.

The implementation details matter a lot more than the vendor pitch deck. A few practical points stand out:

• Standardize your data schema. jsonl with fields for messages[], rationale, tool_calls[], evaluation, policy_flags[], provenance, and trace_id is a solid starting point.
• Version everything. rubric_version, prompt templates, gold-set revisions, and any assistant_model_sha used in labeling should be tracked.
• Run blinded acceptance testing against downstream metrics, not vendor self-reports. Check coding Pass@1, SWE-bench, math accuracy, tool-use success rate, and safety regressions.
• Put quality terms in contracts. Inter-rater reliability thresholds, leakage tests, rework clauses, audit rights, and latency SLAs should be standard.
• Reserve your hardest tasks for scarce experts. Use uncertainty sampling or disagreement sampling to route work instead of treating every example the same.

Security and compliance deserve separate attention. If your vendor can’t explain how data is segmented, who touched what, whether annotators are vetted, and what certifications or controls support those claims, don’t wave it through because the sample outputs look decent. Frontier training data increasingly includes proprietary workflows, internal tools, and sensitive edge cases. A sloppy annotation pipeline can turn into a data leak with a nice interface on top.

Meta’s partnership with Scale was supposed to simplify one part of the AI race. Instead, it exposed how messy that layer has become. The next gains won’t come from compute alone. They’ll come from better supervision data, tighter controls, and a less naive view of how replaceable expert human judgment really is.