Why AI keeps getting better at code and weirdly mediocre at conversation

AI progress looks uneven because it is.

Coding agents keep getting better. Math systems keep posting stronger scores. Some multimodal models now produce video with fewer obvious glitches. Then you use a chatbot for open-ended writing, email cleanup, or a messy strategic discussion, and the progress feels limited. Sometimes it barely feels like progress at all.

The reason is fairly simple: models improve fastest where machines can grade them at scale.

If a task has a tight feedback loop, reinforcement learning has room to work. If feedback is fuzzy, expensive, or inconsistent, progress slows down. That gap explains a lot about where AI products are improving right now.

Why code improves so quickly

Code gives model builders something most knowledge work doesn't: clear answers.

A model can generate a patch, run pytest, check types with mypy, lint with ruff, scan for obvious security issues, and turn all of that into a reward signal. Pass more tests, get a higher score. Break the build, lose points. Take too long or use too much memory, get penalized again.

That loop matters.

Once output can be evaluated automatically, you can run huge numbers of training episodes in sandboxes. The model proposes code. Tools execute it. Verifiers score the result. Training then pushes the policy toward actions that produce better outcomes. You can do a version of this at inference time too by sampling multiple solutions and keeping the one that survives the verifier stack.

That's why coding assistants feel materially better than they did a year ago. The gains mostly come from environments with cheap, repeatable feedback.

Math has a similar advantage. There's usually a right answer, and now there are better ways to reward intermediate steps too. That helps with credit assignment, one of the messier parts of RL. If the model gets the setup right and fails late, the system can still learn from it.

The bottleneck now is reward quality

For a while, the story was bigger models, more data, more compute. That still matters. It just doesn't explain enough anymore.

The tighter constraint now is reward quality.

Can you tell quickly and reliably whether the model did the task well? Can you do that at high volume? Can you stop the model from gaming the score? If the answer is yes, improvement tends to come fast. If the answer is no, progress gets expensive and starts to flatten out.

That's a big part of why chat still feels stuck in places where code doesn't. Conversation is loaded with hidden variables: tone, timing, usefulness, audience fit, factual calibration, social context. Human raters can judge some of that, but preference data is noisy and expensive. It doesn't scale like unit tests.

Long-form writing runs into the same wall. You can score grammar and maybe structure. You can check citations. But the parts people actually care about, judgment, taste, synthesis, voice, knowing what to leave out, are much harder to compress into a clean reward function. So these systems do improve. Just not at the same pace.

What RL-friendly products have in common

The strongest AI products right now tend to share the same shape:

1. A large pretrained model
2. A tool-rich environment where actions have observable outcomes
3. A verifier stack that turns those outcomes into rewards
4. Some RL or RL-like optimization on top
5. Distillation back into a cheaper model or policy for production

That environment piece doesn't get enough attention.

In code, it's easy to picture: sandboxed repos, test runners, static analyzers, package installation, timeout limits, maybe SAST and DAST checks. The model writes code, runs tools, and gets graded on pass rate, style, performance, and safety constraints.

A toy reward function might include:

• positive score for passing tests
• penalties for lint failures or type errors
• penalties for slow runtime
• extra reward for coverage gains or lower memory use

None of that is glamorous. It works.

The same pattern is showing up outside code. Video is a good example. It used to look hard to grade automatically because human judgment dominated. That's changing. Verifier quality has improved enough that parts of the problem are now measurable.

Teams can score identity consistency across frames with embedding models. They can penalize object teleportation with segmentation and optical flow. They can estimate motion and flag obviously broken physics. They can measure flicker and temporal instability. These metrics are imperfect, but they are machine-readable. Once a domain becomes rewardable, progress speeds up.

Why generic assistants keep underdelivering

A lot of companies still assume one general assistant will eventually absorb every workflow if the base model gets strong enough.

That looks less convincing by the month.

The better bet is to break work into narrow jobs with clear boundaries and measurable outputs. Bug triage. SQL generation against a known schema. Prior auth document extraction with validation. Claims review with coded rules. Backtesting trading hypotheses against historical data. KYC checks with explicit thresholds and escalation paths.

Those jobs are easier to verify, easier to improve, and easier to trust.

This also explains why a lot of internal AI deployments feel underwhelming. Teams start with the wrong abstraction. They ask a model to "help with research" or "assist customer operations" instead of building around the actual decision points, tool calls, and checks that define acceptable output.

If you can't score it, you probably can't improve it reliably.

The next moat is evaluation infrastructure

Model access is getting commoditized. The durable advantage is shifting elsewhere.

Owning the environment, the task distribution, and the verifier stack is starting to look like a real moat. In practical terms, not in some vague platform sense. If your team has years of internal workflows, replay logs, failure cases, hidden test sets, domain validators, and safe sandboxes, you can train or tune agents far more effectively than a competitor using the same frontier API with a loose prompt.

Expect more teams to invest in EvalOps:

• managed eval suites
• domain-specific verifiers
• task replay systems
• reward modeling pipelines
• anti-cheating checks
• versioned sandboxes for agent testing

It's not flashy. It is where a lot of the real work sits.

Reward hacking and security are part of the job

The obvious risk in any reward-driven system is that the model learns to game the test instead of solving the problem.

In code, that can mean brittle solutions that pass visible tests and fail in production. Or agents that exploit environment quirks, suppress errors, overfit to benchmark patterns, or take unsafe shortcuts. If the sandbox is loose, the risk climbs quickly.

A serious setup needs:

• deterministic, isolated execution
• no network egress unless explicitly required
• pinned dependencies
• hidden eval cases
• property-based tests and fuzzing
• multiple independent verifiers, not one brittle metric
• logging strong enough to audit suspicious wins

Security matters because execution is part of the product. A coding agent that installs packages, edits repos, and runs commands can become a liability fast.

What changes for teams

The talent mix is shifting.

You still need model engineers. You still need platform people. But the overlooked roles now are evaluation engineers, verifier designers, and domain experts who can turn fuzzy business tasks into something testable.

That translation layer often decides whether an AI feature keeps improving or stalls after a decent demo.

For technical leaders, the practical question is straightforward: which parts of your workflow can be turned into a fast, trustworthy feedback loop? Start there. Don't wait for a universal assistant to paper over a badly defined process.

The teams shipping useful AI fastest are usually the ones building environments where the model can fail, get scored, and try again thousands of times before users ever see the result.