Daily Brief

A single mid range GPU can turn half a million tokens per second into embeddings in the low tens of milliseconds, where a managed endpoint costs orders of magnitude more and takes hundreds. Daniel Svonava calls embeddings the no brainer entry point. His real subject is what comes after. A small model fits on one GPU two or three generations old, and for a specific task it is now at or beyond the frontier, which is flattening while small open models climb. You do not prompt one 27 billion parameter model for everything; you slice the workload into tasks and pick the model trained for each, so a

Byung-Gon Chun's team invented continuous batching, now standard across the industry, and the work that followed inspired one of the most widely used open source serving frameworks. So when he says agents have changed the economics of inference, he knows the tooling from the inside. His demonstration: the same coding agent, the same task of building a tower defense game, run once on a closed frontier model and once on an open weight model. Both finished at a usable level. The open weight run came in roughly five and a half times cheaper. That is the promise of open weights. But the model is on

Once in roughly a thousand prompts, the model returned gibberish. No crash, no warning, and high confidence, which made it an engineering problem, not a quality one. It happened only in vLLM, only under load, and only with Jamba, AI21's hybrid of attention and Mamba layers. Asaf Gardin and Yuval Belfer could not reproduce it with prompts, so they starved it: dropping vLLM's GPU memory utilization from ninety percent to twenty, at temperature zero, made request 8,854 return nonsense every time. From there they built a logprob comparison against a plain reference implementation, cleared the pref

An RMS norm layer does almost none of the arithmetic in a transformer, yet a single decode step can launch it around 33 times, and a GPU is fast at math and slow at everything else: starting work, moving data, waiting. That gap is what FlashNorm attacks. Filip Makraduli wrote the paper with Nils Graef, and the idea fits in two lines of algebra. Fold the norm's gain into the projection weights offline so one matrix absorbs both. Defer the scalar divide so the matrix unit and the vector unit run at once instead of one idling for the other. And in newer architectures that normalize twice in a row

Full episode: https://www.youtube.com/watch?v=6AgOfiZOWiY Me on twitter: https://x.com/dwarkesh_sp

Full episode: https://www.youtube.com/watch?v=6AgOfiZOWiY Me on twitter: https://x.com/dwarkesh_sp

法治建设，它不可能一蹴而就，无法毕其功于一役，法治的每一个进步背后都有许多沉甸甸的故事，这些故事不应该被人遗忘，因为牢记历史就是为了避免重蹈覆辙。 参考文献： 1.陈永生，我国刑事误判问题透视———以20起震惊全国的刑事冤案为样本的分析，中国法学2007年第3期 2.对我国死刑复核程序之检讨———以中国古代及国外的死刑救济制度为视角，陈永生，比较法研究2004年第4期。