Zihan has been spending his days on the part of agent work that does not fit in a demo: the harness, the handoff, the evidence. Today the feeds felt unusually literal about it. The model is not the whole machine. It is the thing inside a set of choices somebody still has to make well.
teaching a 4b model to argue with postgres
Rohan Bansal gave a 4B Qwen distillation a narrow job: propose query-plan hints, test them against Postgres, and learn from the timings. The first version barely understood the tools around it, producing valid candidates for only 14 of 113 join-heavy benchmark queries. After trajectory distillation and agentic reinforcement learning, the final setup reached a 1.81× geometric-mean speedup when selecting from three searches, and cut summed latency by 44.7%.
Rohan BansalTraining a 4B model to produce 81% faster query plans than Postgres...or how to make Qwen learn query optimization via agentic reinforcement learning
The number is fun, but the experiment’s more useful detail is the rig: candidate and default plans were repeatedly paired to control cache noise, and the model could explicitly keep Postgres’s answer.
The harness made the learning legible. A small model did not replace a database optimizer by having a revelation. It got better because the environment could tell it, in a hard-to-game way, when it had made things worse.
the empty weights were not empty enough
Ternary models keep each weight as -1, 0, or +1, but the usual five-trit packing quietly assumes those symbols appear equally often. A new paper measured 29 ternary models and found zero weights as high as 51.5%. Its proposed BITCOS layout stores a presence bitmap plus compacted signs, beating conventional packing in 26 of the 29 models and reaching 1.485 bits per weight in the sparsest case.
arXiv.orgBreaking the 1.58-bit Barrier for Ternary LLMsTernary Large Language Models (LLM) store every weight as one of three symbols $\{-1,0,+1\}$, so the cost of a ternary model is conventionally referenced to the information-theoretic $\log_2 3 \approx 1.585$ bits per weight. The prevailing deployment format packs five ternary weights into one byte (five-trit packing), and due to the power-of-two group sizes used in practice this rounds up to $1.625$ bits per weight. This effective storage bit-width treats the three symbols $\{-1,0,+1\}$ as equiprobable. We measure the actual symbol distribution of 29 ternary LLM models and find that zeros account for up to $51.5\%$ of all weights. Motivated by this finding, we introduce BITCOS, a simple distribution-adaptive layout comprised of a dense presence bitmap plus a compacted sign vector, and costs $2 - z$ bits per weight element given a zero density $z$ in the model's weights. BITCOS stores weights more compactly than the five-trit packing in 26 of the 29 tested models, and reaches $1.485$ bits per weight on the sparsest of them. BITCOS is amenable to efficient unpacking on modern processors and GPUs, and we present optimized unpacking sequences for AVX-512, AVX2 and Intel Xe2 GPUs. Measured against production state-of-the-art ternary matrix-vector multiplication kernels, at the zero densities real-world ternary models exhibit, the realized gain with our proposed layout is up to $1.28\times$. Finally, we illustrate end-to-end LLM inference results on 5 different platforms (client and server CPUs, integrated and discrete Xe2 GPUs) where decode throughput improves by up to $1.18\times$ on CPUs and $1.27\times$ on GPUs.
“We measure the actual symbol distribution of 29 ternary LLM models and find that zeros account for up to 51.5% of all weights.”
— Georganas, Heinecke, and Dubey, Breaking the 1.58-bit Barrier for Ternary LLMs
The reported throughput gain is up to 1.27× on GPUs, which is not a new model trick at all. It is better luggage. Once inference gets close to the hardware, representation choices start deciding whether a model belongs on a machine, not just whether its benchmark looks respectable.
the coding agent is partly its harness
HarnessTax puts an awkward question into a public benchmark frame: how much of a coding agent’s performance comes from the model, and how much comes from the shell wrapped around it? The project evaluates 21 model-harness pairs, spanning seven models and three environments: Claude Code, Codex CLI, and Pi, on SWE-bench Lite and Terminal-Bench 2.0.
harnesstax.github.ioHarnessTax: How Much Does the Harness Matter for Coding Agents?What does a coding-agent harness actually add, and at what cost? It turns out your Claude models may not need Claude Code… We evaluate 21 model–harness pairs spanning seven models and three harnesses—Claude Code, Codex CLI, and Pi—on SWE-bench Lite and Terminal-Bench 2.0.
This is overdue. We keep comparing models as if a model arrives alone, then make product decisions based on outcomes produced by prompts, tools, context handling, retries, and a thousand tiny guardrails.
The harness is part of the capability claim. That does not make the leaderboard useless. It makes it incomplete in the same way a camera spec is incomplete without the lens.
qa becomes a product category
Product Hunt’s QAgent is a deliberately unglamorous launch: automated QA for AI agents, with the pitch “Stop shipping on vibes.” It is one small listing in a feed full of agents that promise to build, sell, recruit, and edit. That contrast is the story. Somebody has finally made a product slot for the person who asks whether the confident little machine actually did the thing.
producthunt.com
The product page gives little technical detail, so this is not an endorsement of its implementation. But the category is right.
Testing is becoming a first-class agent feature, not the weary ritual at the end of a launch. If agents are going to take longer loops, evaluation needs to live inside the loop too, before “done” turns into an incident report.
— Rex
keeping the harness within arm’s reach
