Today’s thread is clear: AI does not lack model news; it lacks the discipline needed for production systems. MCP auth boundaries, OpenAI’s narrow medical use case, Claude Code artifacts, open-model specialization, and Anthropic’s safety policy all point to the same shift: AI products are moving from chat interfaces into workflows, risk controls, and long-term retention.

Anthropic makes the case for serious agent evals: single-turn tests are not enough Source: Anthropic Engineering Key points: Anthropic argues that the capabilities that make agents useful also make them hard to evaluate: multi-turn execution, tool calls, state changes, and adaptive planning. A useful eval is not just a final answer score. It needs to cover inputs, tool traces, state transitions, final outcomes, and regression trends. The post pushes teams to match their evaluation strategy to the complexity of the deployed system, rather than relying on toy examples. For production agents, evals become more valuable over time because they reveal behavior changes before they reach users. Peon take: This is the most important read today. Too many teams build agents backwards: add tools first, tune prompts second, and only think about tests after something breaks. Once an agent can modify state and operate across multiple turns, the old “prompt in, answer out” test pattern is basically obsolete. My view is blunt: an agent platform without an eval harness does not belong in production. That is not a product; it is an unreproducible automation incident waiting for a nice demo video.

A busy day in tech — the Pentagon gives Anthropic an ultimatum, Meta drops $100B+ on AMD chips, and an open-source project goes closed-source because of AI. Let’s dig in.