Daily Brief

When should a company move from prompting to post-training its own models? Fireworks AI co-founder and CEO Lin Qiao lays out the full progression at Sequoia Capital’s Own Your Intelligence event, from prompting and RAG to supervised fine-tuning, preference tuning, reinforcement learning, and distillation. And she explains which technique solves which problem. Lin also covers the pitfalls teams hit along the way: prioritizing data quantity over quality, relying on vibes instead of systematic evals, sloppy RL environments, and reward hacking. She shares how companies like Cursor have used post

RL environments have become the hottest topic in AI training data. But what are they, exactly? At Sequoia Capital’s Own Your Intelligence event, Mercor CEO Brendan Foody breaks down the three components: worlds (the messages, docs, and files of a real project), apps (high-fidelity clones of tools like Salesforce and Google Workspace), and tasks (prompts paired with rubric-based verifiers). He walks through a real legal environment built with lawyers from top firms, and shares post-training results showing dramatic gains on domain-specific tasks from modest compute. Brendan also covers why hum

A Qwen thinking model was taking up to 80 turns to submit on SWE bench. Applied Compute wanted it wrapping up by turn 40 and got the submit tool call rate from 22% to 60% with test pass rate flat. The interesting part is the mechanism: because the rollout was conditioned on an old production trace that never called the tool, the teacher never touched the tool call tokens at all. It moved the reasoning path toward the call instead, and the call followed. Sam Denton's frame is a grid. One axis is how online the traces are, from a single dump of production traces to a unified engine where servin

Build the thousand example eval suite everyone tells you to build, switch harnesses, and 80% of it stops meaning anything. Ben Hylak's complaint is that eval advice is still written for the chatbot era, back when you knew the answer to nearly every question a user would ask. His reframing is that the useful question is not what issues your agent has, since it will have effectively infinite issues, but which ones matter. That turns on the gap between your ceiling, the most impressive thing your agent can do, and your floor, the worst. The floor is what breaks trust: recommending a competitor, d

A profile ChatGPT keeps on Shlok Khemani says he travelled to Turkey in 2025. He never has. The memory came from conversations where he was choosing between Turkey and Thailand, he went to Thailand, and the profile kept both with overlapping dates. What bothers him is not the mistake but the incuriosity: nothing notices the conflict, and the evidence to settle it was sitting in his email as flight and hotel bookings. He calls that a product problem, not a technology one. The rest is a year of reverse engineering how consumer memory systems are built, all of it his reading from the outside rat

Does your agent get dumber after the first compaction? After the second? You cannot read that off the code, only off the traces, and there are far too many to read yourself. So LangChain points agents at the traces of other agents and asks exactly that, alongside questions like where users got upset and what a different model would have done at the same step. Vivek Trivedy's argument is that observability and continual learning are the same problem in different clothing, because an agent acting in an environment produces the only real record of what happened, and that record is the substrate e

An agent is a harness orchestrating a model and context. If you want to own your intelligence, you probably want to own all three. LangChain co-founder and CEO Harrison Chase focuses on the harness: the loop that brings the right context to the model at the right time, and how to customize it with middleware, hooks, and sub-agents. At Sequoia Capital's Own Your IntelligenceI event, Harrison explains when an off-the-shelf harness is enough and when to build your own (the more out-of-distribution your task, the more customization you'll want) and why even custom harnesses should keep in-distrib

Today's AI models keep getting smarter, but every conversation still feels like their first day on the job. Trajectory co-founder Arjun Karanam calls this the experience gap: models are improving on IQ, but not on experience. Continual learning aims to close it, turning the trillions of tokens of agent interactions that get thrown away into signal that makes agents better with every use. At Sequoia Capital’s Own Your Intelligence event, Arjun lays out four goals for companies that want to get there: full traceability (including sub-agents and corrective behavior like edits and retries), evals

Full episode: https://www.youtube.com/watch?v=-RXD4bTuFTo Me on twitter: https://x.com/dwarkesh_sp