What can we say? We love it when the good guys win. But in the backdrop of GLM-5.3-Flash (aka Ox Alpha) impressing everyone (except GDM vaguepoasters) and Qwen also shipping an impressive Flash model on chinese chips, perhaps the post Hot Chips conversation about Western open AI is a great backdrop for this.
Z.ai formally launched GLM-5.3-Flash, revealing that the previously previewed “Ox Alpha” model is its public identity.
Z.ai announced GLM-5.3-Flash as a natively multimodal model with a 1M-token context window, 320B total parameters / 18B active parameters, released under the MIT License, and available via weights, API, chat, coding plan, and AutoClaw.
The launch also resolved the long-running Ox Alpha mystery: multiple posters explicitly connected Ox Alpha to GLM-5.3-Flash, including SemiAnalysis, rasbt, theo, and Cline.
Artificial Analysis first published an overview with an incorrect 400k context window, then issued a correction to 1M context, aligning with Z.ai’s original announcement.
Community response was unusually strong for an open-weight release, ranging from brief shock reactions like “HOLY” to more substantive claims that the model may now be the best intelligence-per-dollar option, e.g. Artificial Analysis and zainhas.
Independent pushback emerged on at least one modality claim: skalskip92 argued the model looks weak on several vision/object detection tasks despite being “native vision.”
Official claims and launch details
Z.ai’s primary launch tweet is the factual anchor: GLM-5.3-Flash is described as:
A follow-up launch-support post from AutoClaw framed the model as suitable for vision-language understanding, code generation, and long-horizon agentic tasks and paired availability with credits/rebates, but this is mainly rollout information rather than new technical evidence: AutoClaw launch post.
Independent benchmarks and cost/performance positioning
Ties GPT-5.6 Terra and Muse Spark 1.2 at 57, but at much lower cost per task.
$0.09/task vs $0.68/task for GLM-5.3 max.
Claimed ~7.5x lower cost per task than GLM-5.3 max.
Claimed ~5.7x cheaper per task than GPT-5.6 Terra and ~4.4x cheaper than Muse Spark 1.2.
Token-efficiency and reasoning mix
Artificial Analysis notes an interesting tradeoff:
GLM-5.3-Flash used 149M output tokens to run the Intelligence Index
compared with 168M for GLM-5.3
but more than Kimi K3 (133M) and Qwen3.8 2.4T A95B (136M) at similar Intelligence Index score
134M of the 149M tokens (~90%) were reasoning tokens
This is an important nuance: the model’s economics look excellent largely because token pricing is extremely low, not because it is especially token-frugal.
Agentic/work evals from Artificial Analysis
Artificial Analysis also reports that GLM-5.3-Flash is stronger than its raw knowledge metrics might imply on agentic tasks:
GDPval-AA v2 Elo: 1770
tied within margin of error with GLM-5.3 and Grok 4.6
behind only Claude Opus 5 xhigh/max
Terminal-Bench v2.1:84.3% vs 83.9% for GLM-5.3
τ³-Banking:47.2%, trailing GLM-5.3 by 3.1 percentage points
Knowledge/hallucination stats
AA-Omniscience score:+7
Accuracy:28%
Hallucination rate:28%
Compared with GLM-5.3:
GLM-5.3 accuracy 34%
GLM-5.3 hallucination rate 30%
Compared with GPT-5.6 Terra:
Terra accuracy 47%
This suggests a recurring theme in reactions: GLM-5.3-Flash may be much stronger on practical code/agentic workflows than on broad real-world factual knowledge.
Architecture and systems details
Several technically informed reactions tried to reverse engineer or summarize what changed from GLM-5.2 / GLM-5.x.
The most detailed public architecture breakdown in the tweet set came from rasbt, who says GLM-5.3-Flash moves from GLM-5.2’s 744B-A40B backbone to 320B-A18B, and uses:
Kimi Linear-style 3:1 hybrid attention
34 KDA layers (Kimi Delta Attention)
11 MLA/DSA layers
MLA = Multi-head Latent Attention
DSA = DeepSeek Sparse Attention
DeepSeek V4-style mHC residual path
four parallel streams
plus a native vision encoder
The same tweet describes it as “super hybrid” because both major attention components are already “efficient” variants rather than a simple efficient/full-attention hybrid.
Another useful systems-oriented summary from thealexker frames the release as an efficiency story, highlighting:
compared to GLM-5.2:
~1/10 the cost
active params 32B → 18B
layers 92 → 45
hybrid linear + sparse attention
smaller average KV cache per layer
lower attention compute compounding at long contexts
claims that visual intelligence benefited from coding/RL style improvements
says the GLM-5.3 infrastructure agent co-authored parts of the work by helping with kernels, bottlenecks, and serving stack optimization
The broader context post from eliebakouch is opinionated but technically notable because it places GLM in a Chinese open-model trend:
nearly all Chinese frontier models now use linear attention
nearly all use sparse attention / indexer-compression designs
many use fancy residuals like mHC, attention residuals, gated residuals
many use Muon
That post is not a direct GLM paper summary, but it helps explain why the architecture details immediately resonated with model engineers: GLM-5.3-Flash appears to be another data point in a fast-converging efficiency-first Chinese frontier OSS design space.
Chinese chip angle and serving implications
The hardware/serving side was one of the most-discussed parts of the launch.
Z.ai itself said the model was “running entirely on Chinese AI chips”. The strongest amplification came from SemiAnalysis, which focused on the claim that 100T tokens/day are being served on Chinese chips. That tweet does not provide all the derivation, but it framed the infrastructure feat as the most shocking part of the reveal.
Reactions emphasized the significance:
theo: “Ox being a ‘flash’ model is insane. Serving all the traffic on Chinese chips is even more insane.”
There was also explicit back-of-envelope capacity reasoning from teortaxesTex:
If inference economics are comparable to V4-Flash,
10K tokens/s/NPU is “realistic”
864M/day per chip
100T/day would imply about 116K chips
suggesting 100K+ chips scale, “doable” but consuming an enormous fraction of total compute
That estimate is speculative rather than confirmed, but it shows how engineers interpreted the serving claim: not as marketing fluff alone, but as an infrastructure statement implying very large domestic accelerator fleets and mature inference optimization.
Adoption and distribution reactions
A notable part of the reaction cycle was how quickly usage posts appeared.
Cline said GLM-5.3 Flash was already its fastest growing model in Cline history, driving 11% of all traffic in less than a week, while also advertising it as free in Cline. This is partly promotional, but it is also a concrete demand signal.
Infrastructure providers moved quickly:
CoreWeave: “coming soon to CoreWeave Serverless Inference”
Baseten: day-0 availability, emphasizing general intelligence + agentic coding, native vision, and 1M context
Dell via Jeff Boudier: framed GLM 5.3 Flash and Qwen 3.8 Flash as open models ready for on-prem deployment
This matters because it reinforces that GLM-5.3-Flash was not treated as a curiosity; it was immediately slotted into real inference/developer stacks.
Facts vs opinions
Facts / externally attributable claims
Z.ai launched GLM-5.3-Flash as 320B total / 18B active, 1M context, MIT-licensed, multimodal, previously previewed as Ox Alpha.
thealexker interpreted the release primarily as a story of efficiency engineering.
eliebakouch framed it as evidence of exciting convergence in Chinese frontier open architectures.
zainhas argued it is now the best intelligence-per-dollar choice.
skalskip92 argued the model is bad at vision, pushing back on the launch’s multimodal framing.
scaling01 alleged it was “painfully obvious” Ox Alpha was a GLM model and further alleged ZAI used hype accounts; that claim is unverified in the tweet set.
OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack
Jalapeño’s published numbers are the day’s biggest technical story: OpenAI released first benchmark details for its custom inference chip Jalapeño, claiming materially better efficiency and latency than NVIDIA GB200/GB300 systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per watt at peak throughput and 1.7–3.6× lower end-to-end latency, with 2.1–4.1× higher performance for highly interactive workloads; the chip is rated at 700W but reportedly stayed at or below 550W on the tested runs. OpenAI says deployment into its own infrastructure begins by year-end, with Gen 2 already deep in development and Gen 3 underway (OpenAI announcement, deployment roadmap, Sam Altman).
Why engineers care: the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive prefill/decode disaggregation or speculative decoding in some setups, while beating systems that did use them (gdb, kimmonismus summary, eliebakouch analysis, You Jiacheng). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin-class systems (SemiAnalysis, dylan522p).
A second-order story is model-assisted systems optimization: OpenAI’s post also said GPT-Astra + Codex helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran 1.5–1.8× faster than existing human-expert-written code (kimmonismus, eliebakouch). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.
Broader infra implication: several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck (Liam Fedus, teortaxesTex reaction, LearnOpenCV caveat on TSMC/CoWoS capacity).
Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class
Harness quality is increasingly as important as model choice: several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on AutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of +9.0 on GAIA2, +9.6 on SWE-Bench Pro, and +10.0 on Terminal-Bench 2.0 over base harnesses (paper summary). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured Harness Card disclosure standard (analysis, “There Is No Neutral Harness”).
Long-horizon software engineering remains very unsolved: SWE Refactor Bench measures whole-repository migration tasks like C→Rust, Maven→Gradle, and POSIX→WebAssembly across real projects including SQLite, zlib, and libsodium. Across 520 runs, only 28 survived all three stages, for a 5.4% survival rate, and 13/20 tasks were solved by nobody (EinsiaAI). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.
Memory systems are being redesigned as programmable state, not compressed chat history: one Alibaba paper summarized by DAIR backs agent sessions with an append-only event log plus a persistent Python kernel, binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include 94.8% on LongMemEval_S, 73.1% on BEAM_10M (+5.1 over the previous best published memory system), and 86.7% on LOCA_256K with Qwen3.8-Max (summary). Related work on Knowledge Triage showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only 10% of safety rules, while type-aware retention policies preserved 2–4× more (summary).
Practical eval-engineering is moving from ad hoc to productized workflows: LangChain/partners shared a concrete loop for turning traces and human feedback into task specs, synthetic environments, and evals that can be used to measure and post-train agents over time (Vtrivedy10, hwchase17). LangSmith Engine also shipped >2× better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes (LangChain).
Local-First Agents, On-Device Inference, and the New Personal Compute Stack
Perplexity’s Portable Computer is the clearest local-agent product launch of the day: Perplexity launched Portable Computer on NVIDIA DGX Spark, positioning it as a fully local version of Perplexity Computer where the orchestrator LLM, subagent LLM, and agent harness all run on local hardware with no cloud dependency (Perplexity launch, model details, NVIDIA, Arav Srinivas). The initial local stack uses a post-trained PPLX 27B with Qwen 3.8 27B also available; Nemotron 3.5 Lightning support is coming.
The deeper trend is persistent, always-on local agents: Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware (Arav Srinivas). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a $5k DGX Spark rather than commodity consumer devices (theo critique, theo follow-up).
Apple/macOS local AI tooling is also maturing: exo said Apple featured it on new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini pages, emphasizing low-latency RDMA over Thunderbolt 5 to cluster Macs and run models like Kimi K3 and GLM-5.3 at API-like speeds, with 4× M5 Ultra scaling to about 4.8 TB/s aggregate memory bandwidth (exo). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical (anemll, onirenaud).
Tooling continues to fill in around local runtimes: Ollama v0.33 added one-toggle integration to let Claude Desktop use Ollama as a third-party gateway for cloud and local models (Ollama); OpenCode v2 was shown running inside a Cloudflare Durable Object, illustrating how small agent runtimes are becoming embeddable in edge environments (fayazara).
Models, Retrieval, and Search Infrastructure
Qwen 3.8 is showing up across the stack: enthusiasm around the Qwen3.8 release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for Qwen3.8-27B (Together) and Unsloth claiming full QLoRA fine-tuning of the 27B model on free 2× Tesla T4 Kaggle instances using optimized kernels (danielhanchen). On the application side, Qwen3.8-27B reached #1 among open models in the Image-to-WebDev Arena and #7 overall, while priced at $0.40 / $3 per million input/output tokens (arena).
Search and retrieval infra got multiple substantive updates: Hugging Face published a detailed architecture writeup for the Papers with Code search engine: PostgreSQL + pgvector, Qwen 3 Embedding 0.6B, hybrid retrieval, embeddings generated on an NVIDIA L4 via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages (Niels Rogge). Keenable came out of stealth with a Web Search API and Web Query Language for AI, built by former Yandex Search leaders and backed by a $26M seed, explicitly targeting agent-scale web retrieval (styskin).
Retrieval model design remains active territory: there was renewed discussion around late interaction / multivector retrieval, with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format (mixedbread perspective, Silvio Martinico).
Robotics, Physical World Models, and Embodied Data
Figure’s “Index” is a major robotics data announcement: Figure introduced Index, described as the largest and most diverse robot dataset in the world, with reported ingestion at 30 minutes of video uploads per second, 16M video uploads, $15M already paid out for data, and 264k downloads. The company also says it will spend $1B over the next 12 months on data and compute (Brett Adcock, follow-up). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.
Large-scale physics/world modeling continues to push context limits: Anima Anandkumar highlighted Accelerated Understanding, a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming 1T parameters during pretraining, 1T context during training, and >5T context at inference without subsampling or patching (Anima Anandkumar). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.
Embodied policy generalization remains an active benchmark target: a separate robotics post introduced S1, a manipulation model that can complete tasks from a single demonstration outside its training distribution (anag004). Google Research also shared AgentHands, an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks (Google Research).
Open-source local task agents: @AndrewYNg on OpenWorker stood out for combining open harnesses, local models, and security-focused workflows.
Benchmark realism for coding agents: @EinsiaAI on SWE Refactor Bench is one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.
Lovable is well known as an AI-powered platform to build applications. But ironically, it is now moving towards a future where fewer and fewer people will be using conventional apps. That is, of course, because of the growing impact of agents.
In a recent blog post, Lovable outlined a vision for “a digital brain for your team connecting your daily tools.”
Or as Lovable CTO Fabian Hedin put it in an interview with Latent Space, “you can get to a place where you’re using one entry point to all the work that you’re doing.”
Diagram by Latent Space based on an internal diagram shown to us by Lovable.
To be clear, Lovable still wants to be the tool you use to build apps — but increasingly, it will also enable you to build what Hedin calls “capabilities.” Lovable defines a capability as a useful part of an application that an agent can call directly; bypassing the need for a human user to open the app.
Lovable can turn a published application into agent-accessible capabilities by exposing selected functions from the app as tools through a hosted MCP server. The result is essentially one application with two interfaces: a traditional human UI and a new agent interface that can be used from ChatGPT, Claude and other MCP-compatible AI clients.
Diagram supplied by Lovable
This is how fast an AI business evolves
This shift towards capabilities is the latest evolution from Lovable, in an already fast-moving 3 years in business.
Lovable emerged from GPT Engineer, an open source coding tool that launched in 2023, initially focusing on prototyping. In November 2024, it became a commercial product and the following month, it was rebranded as Lovable.
By that point, they’d begun to notice some of its users building production apps on Lovable — including products that had become real businesses.
“We started seeing people on the platform building not only a prototype, and not only an MVP [Minimum Viable Product], but the actual thing — an actual product that serves real customers,” Hedin said.
Next, Lovable noticed its users creating internal software, in some cases to support a public-facing app and in other cases as an internal app built for an enterprise company.
“People started creating not only software to enable a business, in terms of a customer-facing product, but also the operations behind the company,” Hedin said.
He means tools like a CRM, an admin panel, or a customer-support console.
From app builder to agent platform
So in less than three years, Lovable has become an all-round software creation and hosting company, which means it’s swimming in the same waters as the likes of Vercel and Cloudflare. That said, Lovable is more focused on AI-generated software than infrastructure. But we are seeing crossover in these markets — for example, Vercel’s v0 allows you to generate an app from natural language, just like Lovable.
Also just like the black triangle and orange cloud companies, Lovable has expanded into agentic workflows.
Lovable connectors, which let you use external tools.
This rapid product evolution has been accompanied by strong user and revenue growth. According to a tweet from Deedy Das, a partner at lead investor Menlo Ventures, the company has surpassed a $500 million annualized revenue run rate, with more than 60 million projects created and over 900 million monthly visits to Lovable-built apps. Lovable also says employees at nearly two-thirds of the Fortune 500 have used the platform.
Unsurprisingly, Menlo Ventures is doubling down on its investment. It led Lovable’s $400 million Series C this month, alongside the Scaleup Europe Fund managed by EQT, valuing the company at $13.3 billion.
Hedin attributes the pace of change to a combination of Lovable’s innovation and the rapidly improving state of LLMs.
“Every few months, we introduce new capabilities at the application layer, while the large language models also continue improving. Those two things compound.”
Lovable’s model of a company brain
The concept of a digital brain for an organization, for Lovable, essentially means a single interface where you can access many different tools and workflows.
“It should have as much context as possible about you, your company and the world around you,” said Hedin. “Then it needs the capabilities to perform both general tasks and actions that are specific to your organization.”
Ultimately, he added, the goal is that “everything that you’re building can be reused in an agentic way.”
Diagram supplied by Lovable
In a sense, then, applications are becoming a collection of capabilities that users will increasingly access through an organizational agent — instead of, or in addition to, the actual application.
“Our job as a platform is to ensure that all these separate capabilities are connected through one agent — not that you have to build a different agent for every task,” said Hedin.
As an example, Hedin mentioned an internal application they use at Lovable.
“We built this internal tool to help us grant credits to users [via] our support team, and help manage our platform in different ways. Those capabilities are now available [internally] through the Lovable agent.”
Lovable also wants this company brain to work asynchronously. Its agent can schedule itself to resume a task later — for example to check a deployment or to monitor a recurring process — then return the result to the same conversation.
The competition
Lovable isn’t the only company pursuing a “company brain” vision. Vercel CEO Guillermo Rauch recently introduced its internal agent, called @𝚟. “Every day-to-day job at Vercel now involves @𝚟,” Rauch tweeted. “It’s growing exponentially both in daily interactions and token use.”
Hedin acknowledged that Vercel and other AI companies are building towards a similar vision, but he thinks Lovable’s “wedge” is “being the best place to build the capabilities that agents need.” In other words, Lovable’s focus is on helping their users build the capabilities that a company brain will need.
“Orchestrating these capabilities is the easy part,” Hedin said. “Making sure they are well connected, built correctly and reliable is the hard part.”
He also hinted at why they’re using the word ‘brain’ to describe this shift, rather than just ‘agent’.
“I’m careful about using the word ‘agent.’ It suggests something like an employee performing a task, which is an easy way to think about it. But underneath, it is really about connecting the right context and capabilities.”
Security and connecting to external capabilities
Perhaps the biggest challenge with the agents and capabilities paradigm is security. For instance, if one of your employees creates an app with Lovable that connects to the company Slack, you want to ensure that user doesn’t inadvertently expose their personal messages, or any other confidential information, to the company brain.
Connectors are Lovable’s method of connecting to external tools and services. Hedin said the platform must account for a “kind of permissioning graph” to maintain security and privacy.
As described in a technical article on Lovable’s blog, one connector type, which Lovable calls an “app user connector,” preserves each user’s identity and source-system permissions. Credentials are stored server-side in encrypted form and handled by Lovable’s connector gateway, rather than being exposed to the generated application; the app instead presents a short-lived key bound to the relevant user.
Diagram supplied by Lovable
“We separate the connection to external systems from the application code being written,” is how Hedin put it. “The application interfaces with the Lovable platform, but the application itself never gets access to those credentials.”
The future of SaaS
So Lovable is moving to a future where a company brain uses capabilities derived from the apps its users build. That begs the question: what will happen to SaaS apps?
Hedin reiterated that people will increasingly interact with software through an AI layer — the company brain concept.
“People are not going to have as many tabs open in different tools as they have historically. That experience is going to consolidate, but the vertical capabilities those tools provide will remain valuable.”
He recognizes that some traditional SaaS products may “fight” this trend, by sticking with their traditional apps and not adapting, but he says Lovable wants to become a platform for building capabilities.
“We want to build this open platform that anyone can connect to, anyone can use,” he said.
Hedin ended with some advice for SaaS companies, whether existing ones or apps that might emerge on the Lovable platform.
“I think SaaS businesses are going to have to focus more on providing the shovel for AI to use their capabilities.”
A few years ago, Caltech Prof. and co-founder of Accelerated Understanding, Anima Anandkumar set out to develop the first open-source weather model with AI. Talking to experts in the field, she was met with skepticism. Weather is chaotic, physics simulations are hard, have been developed for decades, and require supercomputers, the data just isn’t there. Despite reservations, Anima went forth and built. Within a year her team had developed FourCastNet, a predictive model that is competitive with the best physics-based simulations available. Thanks to Anima, and her follow up work, anyone can now predict weather accurately over a short timescale using consumer grade GPUs.1
In the fifteen or so science episodes we’ve released on Latent.Space, we’ve covered atoms, molecules, materials, biology, and math. Anima is a pioneer in studying physical systems that are continuous. Weather, fusion, and fluid or heat flow are huge areas of science that are extremely difficult to model: they are large, chaotic, and fundamentally multi-scale. This is a field the AI community has somewhat neglected, but one we expect will grow fast. We plan to cover large physical systems more in coming episodes.
One thing you can glean from Anima’s work is that this area of AI resists the scaling ideas that have permeated the rest of the field. The data isn’t there: open source datasets in many of these domains are limited to tens or hundreds of thousands of examples, far from what token-hungry transformers need. Even worse, the resolution that physics demands pushes the context length into the hundreds of billions, so you can’t just throw more tokens at the problem. That isn’t a ceiling though, just a slower road: progress here comes from building in structure and inductive biases. Sorry for all you bitter-lesson-pilled language modelers.
“If each dimension is even a few hundred grid points, which is where industrial scale starts... we’re talking hundreds of billions to even a trillion context length. So forget ever having a transformer for anything of this scale, all of the world’s compute will not be enough.”
The math underneath
To tackle these systems, Anima pioneered a technique known as Neural Operators, one of the most beautiful theoretical developments in AI of the last decade.2 These allow you to combine data and physical laws to enable multi-scale inputs and outputs. We’re no longer modeling a grid, we’re modeling a function that evolves over many scales. This allows Anima and crew to build in priors based upon physical intuition.
Neural Operators What if we created a neural network where every layer was itself a function?
To see how physical priors are still helpful for AI modeling, let’s revisit the problem of weather forecasting on a global scale. The earth is a sphere, which meant that accurate modeling involved using the right basis set —3 the Spherical Harmonics. Run a weather model on a grid and it blows up fast. Move to the natural basis for the problem and it stays stable far longer, long enough to roll out months ahead instead of days. Anima’s Fourier Neural Operator learns directly in this frequency domain, and its spherical variant powers FourCastNet 3, which models the weather across the whole globe and keeps running stably far into the future.
FourCastNet 3 The earth is (almost) a sphere — bake the spherical harmonics into your network!
The physical world is forgiving
Anima explored Neural Operators across other physical domains too, and one striking observation is that the physical world is more forgiving than you’d expect. In fusion, a few thousand samples are enough to predict plasma disruptions, and to do it a million times faster than traditional simulation.
None of this is a rejection of scale, it is a different route to it. Anima ultimately still wants to build a “foundation model for physics”, a model that spans many phenomena and does both simulation and design. You get there by building in the structure the physical world already has, not by waiting for data that will never exist. It is a start, and it will take longer than the token-driven parts of AI, because for the physical world tokens were never the answer.
“All of the things that work with deep learning, let’s take them, but make them a bit more principled.”
Weather is only the beginning
Neural operators and weather modeling were a personal passion of mine, so we’ve spent much of this blog and the episode exploring this work. Anima has done so much more! In the episode, we cover several other recent developments from Anima:
Anima has a series of works integrating neural networks and automated proof techniques. We talk about TorchLean, a new framework that lets you write PyTorch-style networks inside the proof assistant Lean and formally verify them. This is a major step for proving bounds on neural networks, something that would be really important for someone trying to, e.g., add a neural network as part of the control loop to their fusion reactor!
This episode has something for every AI or science nerd! Elegant math? ✅ Old school harmonic analysis? ✅ Fundamental developments in modern AI? ✅ Practical ways of modeling the physical world? ✅
We’ve lost count of how many adoption milestones have been passed since the original Rise of the AI Engineer post, but surely Andrew Ng, cofounder of Google Brain and Coursera among many other things, relaunching DeepLearning.ai with a focus on AI Engineering is a big one:
This was done via “an analysis of over 10,000 job postings; carrying out dozens of structured interviews with AI experts, hiring managers, and recruiters; gathering data through surveys; and synthesizing other online data” .
Here are the four most important AI engineering skills according to Andrew:
You can read his full post for more from the horses’ mouth, but we agree that “AI Engineering Skills” are broadly applicable to more than just those with the job title of “AI Engineer” and that is an insightful focus.
Commentary on the 4 skills:
Building and deploying AI applications: “People who are skilled at building and deploying AI applications understand the building blocks of AI (such as LLMs, context engineering, RAG, agentic workflows, machine learning and deep learning) and, importantly, how to use statistical techniques to measure, steer, and govern AI systems so that they behave more predictably. A core skill in doing so is knowing how to drive disciplined evals and error analysis loops.”
yup. this part is closest to the traditional MLE/MLOps workflow, from “zero gradient” aka prompt engineering techniques, to harness engineering, to finetuning and beyond, all the way up to building your own agent labas folks like Harvey are now doing
Software engineering fundamentals. “Understanding software fundamentals allows you to recognize what tradeoffs even exist. This leads to better decisions in choosing your software stack, designing system architecture, designing your data store, testing, and so on. It also leads to much better outcomes than those for an inexperienced developer who vibe codes a solution without knowing the tradeoffs their coding agent is making — which will often be poor ones, because they don’t know what context to give their coding agent.”
yup. this part is closest to the traditional SWE workflow. LLMs reward expertise — they raise the ceiling (high skill devs) much more than they raise the floor (low skill vibecoders), though both are improved.
Using coding agents. “Using agentic coding effectively is now a key skill for every developer. When you have this skill, you have a good mental model for how agents work. You understand their limitations and how to work around them, and are able to quickly steer them — knowing how much to intervene and how much to leave them alone — to build robust software without wasting excessive time or tokens. You also need to know how to work with a clear spec (and when not to bother doing so), orchestrate multiple agents that work together, and avoid pitfalls like risk an agent messing up your production database. Because agentic coding is evolving quickly, using coding agents skillfully means not only knowing cutting-edge practices, but also having routines to keep trying new tools and evolve your workflows as best practices change.”
When we first spoke about the 1000x AI Engineer in 2023, when Copilot was the only game in town, this was the part that was the least evident, but clearly on the horizon. Coding exploded in 2024-2026 culminating in the epic 0-$60B run of Cursor and the rise of Claude Code, Codex, Cognition, Cline and other coding powerhouses not starting with C. Being nimble here is a plus, just as much as being wary of tokenmaxxers with LLM psychosis.
Shaping the build. “Effective AI engineering requires having product sense and understanding business context and customer goals, so you can participate in shaping and driving the build… Taking advantage of this opportunity requires knowing how to drive projects forward. For example, knowing when to quickly build an MVP to take to users for testing, and when to slow down and take longer in order to build more carefully.”
This is perhaps the only part of AI Engineering that wasn’t foreseen in the original essay; we added the AI PM track in World’s Fair 2024 and soon Design Engineering and other AIE adjacencies because the lines started to blur very quickly in both directions.
Overall, a great update to the DeepLearning.AI focus. Welcome Andrew and team!
Agent Harnesses, Persistent Agents, and Enterprise MCP
Harness design is becoming a primary optimization surface: Several posts converged on the idea that agent quality is increasingly shaped by the harness rather than just the base model. NVIDIA’s new evaluation work argues that structural checks on agent “skills” barely predict usefulness—scan scores correlate with judged quality at just Spearman ρ = 0.14—and proposes measuring “Skill Lift” instead: run the same task with and without a skill under identical conditions and score the delta in completed work (paper summary via @omarsar0). In parallel, a position paper on Anthropic-style harnesses argues enterprises should standardize on a single reusable coding-agent harness rather than bespoke orchestration graphs, claiming harness choice can matter more than model choice on enterprise work (summary via @dair_ai).
Persistent and self-modifying agents are moving from concept to open-source implementations: @andykonwinski introduced Headlong, an open-source “microharness” for persistent agents that think continuously rather than only on request. The system stores trajectories as a DAG of jsonl files, keeps a self-guided inner loop running, and reportedly achieved an unattended self-debugging repair in 48 minutes; tradeoffs include $1–$2/hr background thinking cost and occasional self-inflicted failures. Complementing that, @omarsar0 described exo, a harness architecture for recursive self-improvement with an append-only event log, swappable executor, and snapshot/rollback-capable sandbox—explicitly designed so agents can rewrite prompts/tools/memory without being able to corrupt durable state. Together, these posts suggest the next wave of agent infra is about durability, forking, rollback, and continuous operation, not just better prompting.
MCP is maturing into enterprise infrastructure: Anthropic rolled out enterprise-managed auth for MCP connectors, centralizing authorization through the organization’s identity provider so end users no longer perform per-tool OAuth for connectors like Asana, Atlassian, Canva, Datadog, Figma, Notion, Slack, and Supabase (announcement from @ClaudeDevs). Separately, the MCP roadmap highlights upcoming support for long-running workloads with streaming/server push, HTTP for local servers, progressive discovery for large catalogs, and standard identities/delegated permissions (roadmap summary via @_philschmid). This closes a notable gap between toy demos and auditable enterprise deployment.
Model Releases, Leaks, and Competitive Positioning
Qwen3.8-27B continues to punch above its size class: In Code Arena: WebDev, Qwen3.8-27B landed at #9 overall with 1595 points, the only model in its size class in the top 10 and just six ranks behind Qwen3.8-Max (leaderboard update from @arena). It also ranked highly in consumer product, brand/marketing, and gaming categories. A related open-source derivative, Carnice-V3-27B, was released by @kaiostephens: a 27B Qwen-based, Hermes-agent SFT intended to fit on consumer GPUs (3090+), with merged BF16 and GGUF variants.
Rumor cycle around unreleased frontier models intensified: Multiple tweets referenced apparent early access or traces of unreleased systems: EAP models labeled “claude-melon-eap” and “claude-marshmallow-eap” reportedly emphasized 3D/RL-style tasks and used many thinking tokens (demo by @Lentils80); @kimmonismus collected signs of new Claude models, Ox Alpha, Qwen 4, and a confirmed GPT Astra; and @eliebakouch claimed access to a model still in training with a public W&B run. Treat most of this as ecosystem signal rather than verified spec, but it’s notable how much of the discourse is now about pre-release access asymmetry rather than public launches—echoing @michael_nielsen, who warned that controlling access to unreleased models is becoming a source of power concentration.
OpenAI and Anthropic positioning remains in flux: OpenAI developers announced GPT-5.6 availability in Kiro and a claimed ~82% cost reduction per successful Terminal-Bench 2.1 task in Kiro’s spec-driven environment for the Terra variant (announcement). OpenAI also cut GPT-5.6 Sol API pricing to $4/M input and $20/M output tokens (pricing note via @kimmonismus), with Arena updates showing Sol and Luna shifting the cost/performance Pareto frontier (@arena). On the Anthropic side, @tenobrus noted there has not been an unambiguous Opus-line upgrade in over six months, even as external testers reported stronger medium-reasoning results from new Claude variants (@kimmonismus).
Inference, Benchmarking, and Cost-Efficiency
Tool latency overlap is emerging as a key harness-level speedup: @a1zhang introduced Speculative Programmatic Tool Calling (sPTC), which predicts safe tool calls during code generation and launches them early in a copy of the environment so execution overlaps with token generation. The reported improvement is modest so far—about 1.0–1.2×—but the mechanism is important: it shifts optimization from token-level decoding tricks to agent workflow pipelining. @lateinteraction compared it to CPU speculative execution, emphasizing that discarded work is acceptable if most guesses are right.
Token accounting and benchmark hygiene remain messy: Several posts called out misleading reporting practices. @bnjmn_marie shared a DeepSWE run with 918.9M input tokens, clarifying many were cache hits, while @cHHillee bluntly argued that counting cached input tokens in “token usage” is “incredibly dumb.” On the eval side, @jmbollenbacher warned that when a quantized model exceeds the reference model on a benchmark, it may indicate overfitting the quant, not genuine improvement; @xeophon summarized the broader lesson: fixing the eval may matter more than hill-climbing it.
Cost-normalized agent benchmarks continue to reshape model choices: Together AI reported that under a $100 budget, GLM-5.3 completed 5× more work than Fable 5 on DeepSWE, roughly 17 vs 3 solved tasks, despite similar first-try performance (tweet). @reach_vb similarly reported GPT-5.6 Sol Max at 72.7% on DeepSWE v1.1 for $6.47/task versus Fable 5 Max at 69.7% and $21.63/task. Cline also compared Ox Alpha vs Fable on a real bugfix and found both solved it, but Ox used roughly 3× fewer output tokens, suggesting a notably different post-training philosophy around re-verification versus acting on the first conclusion (comparison from @cline).
On-Device AI and Inference Systems
Liquid AI + Artificial Analysis launched a serious on-device benchmark stack: @liquidai released Pipette, an open-source evaluation suite for on-device inference measuring quality, speed, latency, and memory across model + quantization + runtime + device combinations, with 10k+ verified results spanning 35 model classes, 7 quants, llama.cpp runtimes, and four devices. Artificial Analysis paired this with independent phone-scale intelligence evals on iPhone 17 Pro and Galaxy S26 Ultra (full thread).
Phone-scale results highlight a different Pareto frontier than cloud evals: Under an 8 GB memory / 16K context framing, Nanbeige4.2-3B and LFM2.5-2.6B topped the average score at 63, with LFM2.5-2.6B much more efficient on iPhone (8.0s, 2.3 GB) than Nanbeige (21.4s, 4.0 GB). MoE designs such as LFM2.5-8B-A1B and Ling 3.0 Tiny are notable because they activate ~1B parameters/token, enabling sub-6-second responses on phone hardware. The evaluation also makes explicit that many “smart” reasoning models are poorly matched to mobile memory and latency constraints.
Inference vendors are competing on agent-specific throughput, not just raw TPS: NVIDIA’s Groq 3 LPX was described as adding a dedicated token-generation accelerator to Vera Rubin, with a claimed 3,400 output tokens/s on Gemma 4 31B at 100K context in Artificial Analysis benchmarking (summary via @kimmonismus); Groq said it will be among the first to deploy it in production (announcement). Separately, vLLM published extensive AgentX 1.0 results on real multi-turn coding traces, emphasizing KV offload, prefix reuse, and prefill/decode disaggregation as the keys to high agentic throughput rather than classic single-turn serving metrics (@vllm_project).
Research, Papers, and Technical Education
RL for LLMs and harness-native training remain hot: @cwolferesearch published a comprehensive reinforcement learning guide covering token-level vs completion-level formulations, PPO/GRPO variants, actor-critic methods, rubric-based RL, and agentic RL/world modeling. This coincides with growing attention on “harness-native” RL and agent environments, reflected in paper roundups like @TheTuringPost and discussion of papers such as Agent Lightning, LEGO-RL, EnvHarness, and SkillGate.
Other notable research threads: Meta/USC’s Periodic Row-wise Muon extends Muon optimization to larger diffusion transformers by amortizing expensive Newton–Schulz updates while keeping gains over AdamW (summary via @iScienceLuvr); Adobe’s Latent Dynamics Reasoning learns extrapolative video world models from pixels by modeling latent state evolution instead of direct future prediction (paper via @_akhaliq, authors’ note); and Cartwheel reported compute-optimal scaling laws for human motion generation, arguing motion may become the fifth modality with Chinchilla-like scaling behavior (launch).
Educational content worth saving: @fchollet recommended chapters 15–16 of Deep Learning with Python as one of the best accessible explanations of why dot-product attention works; @ProfTomYeh posted a detailed by-hand walkthrough of self-attention; and @mervenoyann announced a new home for llama.cpp docs, with upcoming material on speculative decoding, quantization, and coding agents.
Top tweets (by engagement)
Hands-on product/UI performance: Anthropic said long answers in Claude web/desktop now stream ~4× smoother, with 9× fewer stalls and 4.5× shorter worst freezes on slower laptops (announcement).
Fast image generation UX: @samdape showed a technique to make GPT image generation draw faster.
OpenAI research culture: @gdb amplified a post from @kundan2510 praising OpenAI’s willingness to sustain long-term bets like full-duplex models.
Learning resources: @fchollet recommending attention chapters from Deep Learning with Python was one of the highest-signal educational posts in the set.
Enterprise MCP: Anthropic’s enterprise-managed auth for MCP connectors was one of the most consequential platform updates for production agent deployment (announcement).
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Qwen 3.8 27B Coding and Quantization Benchmarks
“Qwen 3.8 isn’t Opus level”: I re-ran the test. (Activity: 911): The image (link) shows the Deepseek/pi.dev-style coding harness being used with qwen3.8-27b in “Plan” mode for a C#/OpenGL ocean-rendering task, supporting the post’s claim that harness quality strongly affects observed model capability. In the author’s rerun, the same Qwen3.8 model and prompt failed under VS Code Copilot with a black screen, but succeeded under the alternate harness, reportedly using screenshot feedback and even generating a PNG decoder when vision was not enabled, producing waves, sky, sun, and underwater view in about 1 hour on an RTX 5090 running an ninfer-nvfp4 build with ~190k context at ~150–180 tok/s. Commenters largely agreed that the result demonstrates a large gap between “lazy” or sandboxed coding harnesses and agentic harnesses with execution/screenshot feedback. The original critic of Qwen3.8 conceded the prior conclusion was wrong and began retesting with pi.dev, noting fewer crashes and lower RAM use than VS Code/BYOM with llama.cpp.
A key technical theme was that harness quality can dominate perceived model capability: commenters noted Qwen 3.8 apparently implemented an “on the fly PNG decoder” and still produced working ocean shaders despite an initially misconfigured or limited execution setup. The discussion framed this as evidence that sandboxed tools like Copilot-style environments may under-represent what coding agents can do when given a proper runtime/test loop.
The original tester reported switching from VS Code + BYOM talking to llama.cpp to pi.dev after acknowledging the earlier harness was inadequate. They observed two concrete issues in the VS Code setup: driver errors after spawning the test executable, and random VS Code crashes while llama.cpp RocM 1200 build from Lemonade SDK continued running without output errors; by contrast, pi.dev had not crashed and used noticeably less RAM.
Several commenters compared agent harnesses such as pi.dev/OhMyPi, opencode, and local llama.cpp setups, with interest in how much autonomy the harness provides beyond a standard Claude-like chat workflow. Hardware constraints also came up: users speculated that a RTX 5090 or similar high-end local GPU setup, potentially with tools like Ninfer, could make local agentic coding workflows more viable without cloud subscriptions.
New qwen3.8:27b on a 39k line C to single-file HTML / three.js port (Activity: 655): A one-shot agent benchmark attempted to port a 2.1 MB / 39k-line / ~600k-token single-file C procedural shooter (skill-issue) into single-file HTML/Three.js, where the source was >2× the available 262,144 token context. On RTX 6000 Pro 96GB with vLLM, FP8 weights and FP8 KV cache, Claude Code + Opus 5 produced the only “okay” port in 21 min / 1759 LOC, while qwen3.8:27b via hermes took 4h18m / 949 LOC and via codehamr (repo) took 1h40m / 1056 LOC, both judged “bad.” Commenters suggested that direct “convert this code” prompts cause models to re-imagine behavior; a more reliable pipeline is to first generate a transpiler, get runnable target-language output, then iteratively rewrite function-by-function against high-level pixel comparisons or low-level register/state references. Technical debate centered on whether the poor local results were due more to prompt/harness design, missing decomposition/tests, or inference setup: multiple commenters warned that FP8 KV-cache quantization may significantly degrade long-context performance and suggested rerunning without it. Others argued the wall-clock gap is expected because Anthropic can parallelize across far more hardware, and recommended measuring vLLM tokens/sec, planning first, splitting the monolithic C file into modules, and adding behavioral tests before porting.
Several commenters argued that direct “convert this codebase” prompting causes models to re-imagine the source rather than preserve behavior, even with frontier models. A suggested workflow is to first have the model help write a transpiler to the target language, then iteratively rewrite function-by-function while validating against high-level pixel comparisons or low-level register/value traces to reach pixel-perfect equivalence.
Multiple comments questioned the inference setup, specifically FP8 KV-cache quantization, Q8, and not running the full bf16 Qwen 27B model on an RTX 6000-class GPU. The concern was that KV-cache compression/quantization could introduce severe quality issues for a long-context code-porting task, and that rerunning without FP8 KV-cache or with full bf16 would better isolate model capability from quantization artifacts.
One technical explanation for the long runtimes was repeated KV-cache reprocessing in vLLM: if the engine releases the session cache, it may spend minutes recomputing prior context before generating any new tokens. A commenter suggested using LMCache to persist KV-cache in RAM, noting that cloud providers often avoid this latency by caching processed context across turns.
By AI standards today is a pretty quiet Friday, so it’s time to take a step back and reflect on what is really going on. If you read our 2025 reading list, and followed our coverage of Z.ai GLM, understood the Poolside pivot, been following our AI for Science themes, and tuned in to today’s Simile pod, you not only are one of the biggest readers of Latent Space, you will probably also arrive at this mental model:
Every year since 2022, one more component of the pipeline that produces machine intelligence has flipped from human-made to model-made. Not gradually, and not evenly — each flip has a patient zero, a paper or product where the synthetic version first became load-bearing at a frontier lab, and from there on, the future is simply here but not yet productionized.
And if you squint, what we used to call “synthetic data” and “synthetic rubrics” and “AI researcher” and “end to end RL environments” is just increasingly ambitious human simulation - 10% worse, but 100x cheaper and 10,000x faster.
Stage 1: The reward signal (2022)
The first thing to go synthetic was, counterintuitively, the judge. InstructGPT established the now-canonical trick: collect human preferences once, train a reward model, and let the policy optimize against the model rather than the humans. From the policy’s point of view, the thing dispensing approval was already an LLM. Constitutional AI pushed further and had the AI critique itself against a set of principles (RLAIF), and Lee et al. later showed AI feedback matching human feedback at a fraction of the cost. By the time LLM-as-judge became the default eval methodology (MT-Bench, AlpacaEval), the entire approval apparatus — reward, critique, evaluation — ran on models judging models.
Stage 2: The training data (2023)
Microsoft’s Phi series made the argument in its title: Textbooks Are All You Need. A small model trained on LLM-synthesized, textbook-quality data punched far above its parameter count, and phi-1.5 confirmed it wasn’t a fluke. Apple’s WRAP generalized the move: don’t just generate data, rephrase the entire web with an LLM, and pretraining gets roughly 3x more efficient. From there the pipeline industrialized — NVIDIA’s Nemotron-4 340B shipped with a permissively licensed synthetic data generation pipeline as a headline feature, and by 2025 reasoning-trace corpora (chains of thought generated by strong reasoners) had become a standard pretraining and mid-training ingredient. The corpus, the thing that was supposed to be the irreducibly human input, was now substantially model-written.
Stage 3: The teacher (2023)
Weeks after ChatGPT’s API opened, Stanford’s Alpaca demonstrated that a $600 fine-tune on GPT-generated instructions could clone much of a frontier model’s behavior. Vicuna did it with shared conversations; Orca did it with rich teacher explanations rather than bare answers. The technique matured from imitation into a proper training discipline — on-policy generalized knowledge distillation fixed the train/inference mismatch — and reached its cultural peak when DeepSeek-R1 shipped a family of distilled models alongside the flagship, making “the teacher is a model” the default assumption for every small model release since.
Stage 4: The curriculum (2024)
Stages 1–3 made the inputs synthetic; stage 4 is where the loop starts closing on itself, because the model begins deciding what to learn next. The pieces existed early — Self-Instruct (models writing their own instruction sets) and STaR (models bootstrapping their own reasoning traces) are both 2022 — but the flip came when Meta’s Self-Rewarding Language Models and SPIN showed a model could generate its own tasks, judge its own outputs, and improve past the ceiling of its human preference data. Curriculum design — historically the most artisanal part of ML, the taste-driven choice of what to train on next — became something models do to themselves.
Stage 5: The researcher (2026)
The assistance era (Copilot, then SWE-agents) kept a human choosing the experiments. The discovery era did not. DeepMind’s AlphaEvolve evolved genuinely new algorithms in 2025, and Sakana’s AI Scientist (now in Nature!) sketched the full paper-writing pipeline. The big moment was Karpathy’s autoresearch in March 2026: a deliberately minimal ratchet loop where a coding agent modifies a real LLM training setup, runs a five-minute experiment, keeps the change only if validation loss improves, and repeats overnight. His own extended run stacked 700 experiments into 20 kept improvements, cutting time-to-GPT-2 from 2.02 to 1.80 hours — real, transferable code changes found while he slept.
Stage 6: The environment (2026)
RL’s scaling bottleneck moved from the model to the environment: you need thousands of executable, verifiable, professionally realistic task worlds, and humans can’t hand-build them fast enough. We covered this recently in our z.ai / GLM-5.3 issue: Z.ai built pipelines that synthesize environments end to end — research agents mine real work patterns and convert them into long-horizon environments with hidden state, a judge agent attempts each task to confirm it’s solvable, and verifiers are synthesized without seeing the reference solution, then stress-tested with oracle, no-op, and unsolved-state checks until their binary reward is reliable enough to train on directly. As the GLM-5.3 release puts it, the entire environment, judging, and verification stack is synthetic all the way down. The same week, Ornith-1.5 shipped claiming end-to-end self-improvement — the model proposes its own tasks and generates its own RL rollouts. The gym, the referee, and the scoreboard are all models now.
Stage 7: The human subject (2025)
If models can be the judge, teacher, and environment, the remaining human role in the loop is subject — the source of preferences, behavior, and demand. That’s the layer Simile is replacing. The lineage runs from Joon Sung Park’s Generative Agents (Smallville, 2023) through Generative Agent Simulations of 1,000 People, where digital twins built from two-hour biographical interviews reproduced their source humans’ survey and behavioral responses 85% as accurately as the humans reproduced themselves two weeks later.
The big hurdle to overcome: frontier models are trained toward being agent models, which makes them bad simulations of real people — so Simile post-trains on interviews, transaction data, and registered RCTs from the Open Science Framework specifically to recover human bias, inconsistency, and causal texture, and reports early scaling laws for simulation quality. With SimGym at Shopify simulating shopper trajectories and Tencent’s billion-persona approach at the crude end of the spectrum, the focus group, the user study, and the A/B test panel are becoming inference workloads.
Stage 8: The physical world (2026, in progress)
The last row of the grid never quite turns red, and that’s the point. Poolside’s reverse-execuhire letter drew the line precisely: the world’s problems split into intelligence-bound ones (solvable by scaling cognition, soon commoditized by open weights) and experiment-bound ones, where “no amount of intelligence substitutes for real-world experimental feedback — 100,000 brilliant minds won’t cure cancer without a wet lab.” Their bet is that AI’s durable value accrues to whoever owns the experimental loop: AI as “the world’s most valuable scientific discovery engine.”
The bio side is running the same play from the other direction. CZ Biohub is imaging the Human Cell Atlas into a virtual cell — because in silico is roughly 1000x cheaper and faster than in vivo — and extending toward a virtual immune system, with Chai, Xaira, and Lila’s data-center-shaped labs filling in the AI-for-science stack. The physical world is the one component that can’t be fully synthesized — only compressed, cell by cell, into models.
The exponential starts at the diagonal
Read the grid one more time and a second pattern appears underneath the first. Every flip was preceded by the same objection — model collapse, hallucination stacking, garbage in garbage out — and every flip happened anyway, at the exact moment a verification mechanism made the synthetic version trustworthy: aggressive filtering for Phi’s textbooks, judge-vs-judge agreement studies for LLM evals, unit tests and proof checkers for RLVR, oracle/no-op checks for z.ai’s verifiers, registered RCTs for Simile’s twins, the wet-lab loop for the virtual cell. The synthetic frontier doesn’t advance when generation gets better. It advances when verification does.
Which suggests where it goes next. The gray triangle remaining in the bottom-left of the grid — physical experiment, embodied ground truth — is exactly the region where verification is slowest and most expensive. The models learned to write, then to judge, then to practice, then to experiment. The remaining question of the decade is how much of reality they’ll need to touch — and how much they can get away with simulating.
10% worse, 100x cheaper, 10000x faster… and improving on ALL three dimensions fast.
Agents, Harnesses, and the Shift Toward Environment-Centric Training
The center of gravity is moving from prompts to environments: the most substantive thread here was again GLM-5.3’s sandbox-scaling interpretation: same base model, but better long-horizon performance from richer executable environments and SAO-style counterfactual credit assignment. This aligns with other work shared today: Google’s EnvHarness / EnvRigger adapts static environments using a plugin layer and policy-diagnosed reshaping, improving held-out performance by up to 9 points with 9.8% fewer execution steps.
Benchmarks are getting more task-specific and harder: FACET creates executable terminal tasks from agent skills and validated 6,078 tasks; SWE-bench Science introduces 119 scientific software tasks where even Claude Code + Opus-5 is under 50% pass@1; CADBench finds top models at only 24.6% pass rate across realistic Fusion 360 tasks; and AI4AI-Bench tests recursive self-improvement over 10 research repos, with the best model only at 0.288 average score.
Inference-serving correctness in RL got an important systems result: vLLM’s IsoExec addresses rollout/training logprob mismatches caused by floating-point non-associativity, enforcing bitwise parity across TP/EP/SP layouts. On Qwen3.5-35B-A3B with DAPO on 8xH100, logprob diff reportedly dropped from 1.6e-2 to 6.7e-7 at 25.3% overhead.
Research Highlights: Routing, Recirculation, and Robotics
Inference-time architecture ideas: a DeepMind paper on Recirculation got attention for feeding contextualized deeper-layer activations back into earlier processing at inference time, without retraining. The summary cited improvements including -60% contextualization errors, -23% perplexity, and +21% GSM8K in reported experiments (thread).
Model routing got a more principled treatment: Pandora’s Router from Google DeepMind frames routing as an optimal search problem with costly inspection, rather than assuming routing estimates are free. The claim: it matches exhaustive-estimation quality while calling expensive estimators less often, including settings with specialist LLMs and variable inference-time reasoning.
Qwen3.8-27b has the highest level of “agency” I’ve ever seen in a local model (Activity: 1334): The post claims Qwen3.8-27B running locally on a single RTX 3090 with Unsloth Q4_K_S quantization, q8 KV cache, and 150k context performed unusually capable autonomous agent workflows: using Playwright plus existing SSO/session cookies to navigate university systems and retrieve a course schedule, and separately processing a social-media video via download, frame extraction, transcription with Whisper, and image enhancement. The image is a screenshot of the model reporting use of an Outlook/OWA Playwright profile, Microsoft “stay signed in,” and Duo browser-trust cookies to access school systems, making the technical significance less about raw model quality alone and more about local LLM tool-use agency plus high-risk credential/session handling. Comments were impressed but cautious: one user explicitly worried about giving an agent enough access to potentially perform destructive actions like withdrawing from university, while others framed it as evidence that advanced local agentic systems are already here but unevenly distributed.
A commenter asked for implementation details behind the reported agentic behavior of Qwen3.8-27B, specifically the agent harness used—e.g. Claude Code, Hermes, or another framework—and how tools were exposed via MCP servers, browser tools, Python, filesystem access, etc. They also asked what inference backend served the model, such as llama.cpp, and how it was able to autonomously download video, extract frames, and install Whisper.
There was technical concern about the reliability of the referenced quantization: one commenter noted surprise that “the quant is that good,” while mentioning reports of looping behavior at that quant. This suggests the model’s apparent agency may be sensitive to quant level and runtime behavior, especially for long-horizon tool use.
A safety-oriented thread questioned giving local agents broad system access, with one commenter saying they would not trust agents like Sol or Fable with unrestricted permissions. The concern was not about local inference itself, but about autonomous agents with enough privileges to perform impactful real-world actions such as modifying accounts or workflows.
Qwen3.8-27B took a serious hit to knowledge vs 3.6 (Activity: 779): The post reports that Qwen3.8-27B / Qwen3-8-27B appears to regress vs Qwen3.6-27B / Qwen3-6-27B on offline, no-tool-call factual recall: the author’s private “mildly obscure” trivia/prepper benchmark showed failures across quantization levels and sampling settings, consistent with lower scores on Artificial Analysis’ Omniscience knowledge benchmark. The reported degradation is specifically about knowledge stored in weights and hallucination/fact recall under airgapped inference, not coding/tool-use; commenters note Qwen3.8 is stronger at tool calling, web search/fetch workflows, coding, and agentic behavior. Commenters broadly frame this as an intentional tradeoff: newer Qwen 3.x models may be optimized for coding/agentic tasks rather than being “mini Google” factual stores, with Gemma 4 suggested as a better fit for trivia/random-fact recall. One user confirmed regression on niche visual/history/geography tasks such as stamp or old-photo location identification when web tools are disabled, but considered the tradeoff acceptable given improved tool use.
Several commenters frame Qwen3.8-27B as shifting away from memorized factual recall toward coding, tool use, and agentic workflows. One user testing a niche “knowledge” workload—stamp identification and historical/location inference from old photos—reported that with web search/fetch tools disabled, Qwen3.8 performs worse than Qwen 3.6, but becomes more useful when allowed to retrieve information externally.
The perceived regression is described as an intentional tradeoff for a 27B model: reduce obscure memorized knowledge while preserving enough reasoning ability for problem solving and agents. Commenters suggest using models like Gemma for factual/trivia-heavy tasks, while reserving Qwen3.8 for coding/tool-calling scenarios where users report stronger performance.
One technical speculation was that future models may separate base reasoning from domain knowledge via neural plugins/LoRA-like modules: e.g., adding Japanese-language capability or finance-domain expertise as attachable components rather than baking all knowledge into the base model. This was proposed as a way to keep base models smaller or more specialized while allowing opt-in domain expansion.
Qwen 3.8 27b - PI AGENT vs OPENCODE (Activity: 510): The author compares PI Agent vs Opencode using a local llama-server backend on an RTX 3090 with Qwen3.8-27B-Q4_K_M.gguf, ctx-size=100000, flash-attn=on, n-gpu-layers=99, DeepSeek-style reasoning, and a vision mmproj module. They report PI Agent producing better outputs, using fewer tokens, avoiding Opencode’s apparent 32k output-token ceiling/freezing behavior, and delaying context compression until ~90k tokens vs Opencode starting around ~67k when total context is 100k; they also recommend enabling vision so the model can visually assess generated outputs, with CPU/RAM offload acceptable for screenshot evaluation latency (~3s vs ~0.3s GPU). The test was inspired by a prior LocalLLaMA post about generating a bouncing-ball animation: reddit.com/r/LocalLLaMA/comments/1j7r47l/.... Commenters questioned whether a one-shot HTML/animation task is a meaningful harness comparison and suggested multi-step tool-heavy workflows instead. Another user reported PI + local Qwen3.8-27B felt competitive with Claude Code on a roughly one-hour aurora-prediction app build, though both models judged Claude’s initial result slightly better before PI iterated.
A commenter argues that one-shot HTML generation is not a meaningful benchmark for comparing PI Agent vs OpenCode; they suggest using multi-step tasks with extensive tool calls to evaluate the harnesses’ planning, editing, and recovery behavior.
One user reports a subjective head-to-head between local Qwen3.8-27B running in PI and Claude Code on building an aurora predictor app. They felt runtime was similar; both agents judged Claude’s first result slightly better, but after asking PI/Qwen to upgrade its version, the user preferred Qwen’s presentation. The resulting app reportedly integrated multiple satellite instruments and provided 30–60 minute aurora warnings.
Another commenter suggests adding the DeepSeek harness to the comparison, implying the evaluation should cover more agent runtimes than just PI Agent and OpenCode.
2. DeepSeek V4 Flash Benchmarks and Serving
DeepSeek-V4-Flash-Vision-Exp (Activity: 722): The image is a technical benchmark table for DeepSeek-V4-Flash-Vision-Exp (image), comparing it against DeepSeek V4-Flash-0731 and Opus-4.8 on text-agent and multimodal-agent evaluations. It shows broad gains over the prior DeepSeek Flash release, including 83.9 on Terminal Bench 2.1, 75.9 on Toolathlon-Verified, and 64.3 on Chartography, while Opus-4.8 still leads many text-heavy benchmarks; Vision-Exp appears more competitive on multimodal tasks such as Agents’ Last Exam and ZeroBench. The main technical reaction was that the reported DeepSWE improvement of roughly +4 points over 0731 is considered unusually large. Other comments were mostly hype or tribal reactions rather than substantive analysis.
DeepSeek’s announcement says DeepSeek-V4-Flash-Vision-Exp is live via the DeepSeek API with model='deepseek-v4-flash-vision-exp', matching DeepSeek-V4-Flash text capabilities while adding multimodal input. The model supports Chat Completions, Messages, and Responses APIs, with mixed text+image inputs via base64, external URLs, or the Files API; images are billed as up to 384 tokens each at V4-Flash pricing. Docs: vision guide.
Several comments focused on benchmark movement: one noted DeepSWE reportedly improved by 4 points from 0731 to Vision-Exp, while the announcement claims a “major leap” on multimodal agent benchmarks, bringing performance close to Opus-4.8. The technical implication discussed is that Vision-Exp may retain V4-Flash’s agent/reasoning/world-knowledge text performance while substantially improving visual-agent workflows.
DeepSeek also launched a Files API for image reuse: users can upload an image once, reference it by file_id, and avoid resending image payloads across requests, reducing bandwidth overhead. One commenter asked whether the model weights would be open and noted they could not yet find them on Hugging Face, implying that availability appears API-only at the time of discussion. Files API docs: files_api.
The boring way to run Deepseek V4 Flash-0731 130-150 tks - 16x5060ti 16GB over 2 PLX88096 switches (Activity: 621): The image is a terminal GPU-monitoring dashboard validating the post’s unusual 16× RTX 5060 Ti 16GB inference rig: all GPUs are visible, nearly full at roughly 15.2–15.7 GiB / 15.9 GiB VRAM, and assigned to vLLM worker processes for DeepSeek V4 Flash-0731. The setup uses two Broadcom/PLX PEX88096 PCIe switch islands with patched NVIDIA 610.43.02-p2p, Resizable BAR/BAR1 set to 16 GiB per GPU, and custom all-reduce/DSpark pipeline parallelism; reported throughput is about 100–150 tok/s single-user generation depending on TP/PP layout, with concurrency scaling up to 727 output tok/s aggregate for TP4/PP4 at 16 users. The image also shows the tradeoff/oddity of the build: the GPUs appear connected at PCIe Gen1 x8 and are mostly idle at the captured moment despite high VRAM residency, implying the screenshot is more a topology/memory residency proof than a live utilization benchmark. Commenters were less focused on the benchmark table and more on the physical absurdity of the build, asking for “a photo of the setup” and calling it a “mad setup.” One notable skeptical/funny technical reaction was that “a little vibe coding” likely hides substantial custom distributed-inference work.
Sometime around Christmas 2025, AI engineers noticed a change in agents. They started to work! It’s hard to pin down exactly why. Maybe we finally had holiday downtime to try the newest agents with the newest models. Maybe the models had crossed some capability threshold. Maybe the wrappers around the models had matured.
What I’ll argue in this post is that it was the confluence of the last two. The model and the harness improving together and then their curves of improvement crossing at the right moment. And that dynamic helps to explain what comes next: models keep absorbing the harness into their weights, engineers keep deleting what got absorbed, and what remains is a harness for human attention rather than for the model.
Lukasz Kaiser, one of the people who invented the Transformer, said on “Unsupervised Learning” in June:
“The change last winter, last Christmas — it’s a little hard to pin down. I mean, the harness changed and a little post-training changed and then new pre-trained models came… but it felt like a big jump which is not that easy to pin down what did it.”
The answer to “What happened?” isn’t solely in the model weights. It’s in the system that grew up around the weights.
The answer is in the agent harness.
Think back to November 2022, when ChatGPT was the most advanced AI tool. The only capability at its disposal was next-token prediction and some Reinforcement Learning from Human Feedback (RLHF) that allowed it to act like a helpful assistant. No tools, no search, and no reasoning.
The original ChatGPT was confined to its training data and the prompt you sent it. No more, no less. It was a brain in a vat.
The agent harness is a way for the LLM to break free from that confinement and interact with real digital information space.
What a Harness Actually Is
An agent harness is everything besides the model weights that makes the agent work. The environment, tools, context and guardrails that surround the model. Without the harness the model is a brain in a vat. It can take an epistemic action, but needs the harness to actuate that decision in real digital space.
The harness is like giving the mind of the model a body. With the harness, the model can perceive (context), act (tools), persist information (memory and compaction), and enforce its boundaries (permissions and guardrails).
Harness 1.0: The Past, “The Bolt-On Era”
Two curves run through the path of model / harness evolution. What the harness asks of the model, and what the model can deliver in practice.
The gap between these two curves is equal to the effectiveness of an agent, and the closing of that gap is what I’ll argue led to the tangible improvement in agents that Lukasz Kaiser referenced.
Here’s how the gap closes, in stages:
ReAct, “The Harness on Paper” (October 2022): ReAct is a prompting technique to get models to reason through prompting. It’s the agentic loop on paper, external to the model weights. It defines the idea of an “agent loop” where a model reasons -> acts -> observes -> repeats. Again, the ReAct loop exists only as a prompting method. Prompting is the only reasoning method that exists at this time and no one calls it a “harness.” Toolformer (Meta, Feb. 2023), that same winter, hints that tool use could be trained in rather than prompted. It’s a bit like Alan Turing’s idea of the computer before it was instantiated in a physical substrate. A powerful idea that is only later made manifest. (ReAct predates ChatGPT by a month — October 2022 vs. November 2022). Both the curves are near zero at this point. The gap is small because we are just getting started.
AutoGPT/BabyAGI, “Premature Autonomy” (Spring 2023): With AutoGPT/BabyAGI, the harness curve sprints ahead of the model capability curve. Both hand the model full autonomy, asking the model to act as an “autonomous employee,” but the models at this point are still little more than brittle next-token predictors. A loop doesn’t add capability to a model. A loop amplifies the capability a model has, and below some threshold the loop amplifies errors rather than reliability. Consider the power of compounding in the negative: 95% reliability per-step over a 20-step task results in a ~36% average success rate. The harness hands the model an assignment it has no realistic chance of completing. This is where the gap is at its widest and the next 18 months are a reaction and attempt to close that gap.
Cursor/Copilot, “Retreat to Human in the Loop” (2023 - 2024): The first AI-powered IDEs recognize the failure-mode of giving the model too much autonomy. They close the gap by pulling the harness curve down below the model curve. Don’t give the model the loop directly; give the human the loop and empower the human to orchestrate the loop while the model speeds the human up. The first version of Devin tries to hand the autonomy back to the model. A test from the team at Answer.AI shows that is still premature, with a ~15% success rate. It’s evidence that the move from the IDEs to retreat from full autonomy is not cowardly, but the correct move. However, while the prevailing tactic is to pull the harness down below the model, models continue to improve. Near the end of 2024, with the introduction of o1 — the first reasoning model — for the first time the gap inverts and we begin to see the first signs of a model capability overhang.
Claude Code, “The Curves Cross” (February 2025): The inversion at the end of 2024 sets up an opportunity that someone has to seize: if the model is now ahead of the harness, then a harness intentionally riding the brakes of the model is leaving capability on the table. Claude Code is the first coding agent built to seize that opportunity. It abandons the IDE for the terminal, gives the model bash and file read/write access, and replaces the need for human approval on every change with permission rules. The model is handed the loop again, and this time it understands the assignment. Boris Cherny and team build Claude Code with the next model’s capabilities in mind, not the current one. It is such a hit not because it’s the first product to give the model autonomy, but because it’s the first product to do so at the right time. That time is the crossover point where the model has gotten reliable enough to succeed with autonomy. Claude Code grows to roughly $1B ARR within six months, all because Anthropic seized the opportunity available when the curves begin to meet.
What happens next is that the curves don’t just meet, they begin to braid together.
Harness 2.0: The Present, “The Co-Training Era”
Today the harness matters, and in a way we can measure.Harness-Bench ran the same model over the same 106 tasks in different harnesses, and scores ranged from 52.4 to 76.2: a 23.8-point spread with zero change to the model. Half the agent is the harness.
OpenAI achieved a similar result on ARC-AGI-3 with harness changes. Adding only retained reasoning and compaction, GPT-5.6 Sol’s ARC-AGI-3 score tripled from 13.3% to 38.3%.
What’s happening under the hood is that Reinforcement Learning (RL) has moved inside the harness. From OpenAI’s codex-1 release announcement in May 2025: “codex-1 was trained using reinforcement learning on real-world coding tasks in a variety of environments.”
The two curves join and start to braid as one unified system.
This is the dream of Toolformer manifesting in reality. Rather than a tool prompted from the outside, now tool calling is trained from within the environment of the model.
Then, as the models are trained in the environment of the harness, they start to absorb the harness capabilities into the model weights, learning how to auto-compact with knowledge of their own context window, for example.
GPT-5.1-Codex-Max launch:
“The first model natively trained to operate across multiple context windows through compaction.”
Once the models absorb the harness capabilities, the harness can shed the scaffold. It’s production by reduction. Thariq Shihipar from Anthropic said that the team recently deleted 80% of Claude Code’s system prompt.
The measure of the pace of agent harness evolution is how much of the harness you get to delete, while retaining the same capability level. This is the future we need to build towards as AI engineers.
This, then, is the loop of model / harness evolution: train -> absorb -> shed -> repeat. The model climbs to the next thing it can’t do yet.
The jump that Kaiser pointed out is hard to pin down because it’s not a discrete event. A pre-training leap is noticeable because you can articulate it in a model card. A model / harness co-evolution jump is less so, because there’s no documentation of the evolution process. That’s the answer to the jump last Winter: it happened in the space between the model and harness working together.
We need to ask: if every harness capability will eventually get absorbed into the model, what does that leave us with?
Harness 3.0: The Future, “The Attention Era”
Keep deleting everything that the model can absorb. Imagine what your agent looks like at the conclusion of that process. What are you left with in your hand when you’ve deleted everything?
What do the model weights absorb next? Multi-agent orchestration, tool selection, memory...to name a few possibilities. Researchers are building self-improving harnesses that can themselves be trained in a similar way to models.
What’s left at the end of this deletion and absorption process are the human-centric agent capabilities. Things like permissions, identity, trust and legibility. A model that absorbs permissions into itself has dissolved permissions. Absorption doesn’t end the harness. Absorption inverts the harness.
The harness becomes the agent’s interface to the human that operates it.
The harness was born as the human interface to the model. We grew from chatbox to IDE to the terminal. If the model absorbs the computer-facing capabilities, the next stage of evolution becomes one layer of abstraction up. The harness becomes the model’s interface to our human attention.
“The only fundamentally scarce thing is the synchronous human attention of my team.”
Tokens became abundant and reliable, yet we remain bottlenecked on scarce human attention.
We see sparks of this already, with Anthropic’s long-running agent progress files and agentic approval queues.
The gap between the model and harness curve doesn’t disappear when the model absorbs the harness. It migrates across the human boundary and creates a new pair of curves with a new gap. The new gap is the space between what the agent asks of the human, and what the human is able to answer.
The Attention-Interface
I predict that within a year, every company building agentic AI will ship a human attention policy surface in the way that every agentic AI company shipped AGENTS.md.
AGENTS.md tells the agent how to work with your codebase. The attention-interface will tell the agent how to work with you. It will govern when it’s allowed to interrupt you, when it should keep working, which decisions it can make alone and which decisions need your approval. And like everything else in the agentic system, it will become a learnable component of the system that can learn with more data. Every correction becomes useful data.
The model began as a brain in a vat. The harness gave the brain a body, then the body started to dissolve into the brain. What’s left for us to build is the thing no future model will ever absorb. The interface to the one true scarce resource: human attention.
Time to catch up on why this Second Summer of simulation is working!
From creating Smallville, the landmark 2023 paper on Generative Agents that showed AI characters could remember, plan, socialize, and develop emergent behaviors, to now building foundation models of human behavior, Joon Sung Park is trying to answer a much bigger question: what if we could simulate the world before making decisions in it? In this episode, the Simile co-founder and CEO joins us to unpack the path from generative agents to digital twins, why today’s frontier models still fail to capture how humans actually behave, and what it would take to eventually simulate all 8 billion people on Earth.
We go deep on Simile’s approach to modeling human behavior: long-form interviews, observational and transaction data, randomized controlled trials, population-level and individual-level models, and post-training on the causal mechanisms behind why people make decisions. Joon explains how his research created digital twins that reproduced human behavior and attitudes 85% as accurately as people reproduced their own responses, why models optimized to be rational can be bad simulations of irrational humans, and why understanding “social physics” may require changing model weights rather than simply prompting frontier LLMs.
We also explore the much larger ambition behind simulation: testing products and policies before deploying them, finding counterintuitive paths toward desired outcomes, modeling emergent behavior across entire societies, and potentially tackling problems like climate change, democratic instability, and UBI. Joon reflects on scaling laws for simulation, the economics of data-center-scale simulated worlds, the connection to Thomas Schelling and psychohistory, why simulation is surprisingly similar to painting, and whether we might already be living in one.
We discuss:
How Smallville and Generative Agents led to Simile
Why Joon’s team asked: “What if we can just recreate the world that we live in?”
Why useful personal agents require deep models of their users
Memory architectures, Markdown files, and the limits of prompting
“Social physics” and behavioral foundation models
Why web data captures what people say more than what they actually do
Interviews, transactions, observational data, and randomized controlled trials
Why predicting the future matters less than understanding how to shape it
How Simile creates representative simulated populations
Simulation versus prediction and the connection to Foundation’s psychohistory
How to evaluate simulations instead of simply stacking LLM hallucinations
Creating digital twins of 1,000 real people and reaching 85% behavioral accuracy
Why frontier models can struggle to reproduce real human behavior
Why good simulations need to reproduce human biases and mistakes
Post-training models on randomized controlled trials
Population-level versus individual-level simulation
Scaling laws for human simulation
The long-term ambition to simulate all 8 billion people on Earth
Whether simulations could help solve climate change or detect collapsing democracy
Thomas Schelling and the history of agent-based modeling
Why future simulations could require an entire data center
Multi-agent simulations and what happens when simulated people interact
Replacing expensive human panels with synthetic populations
Why market research is only the starting point for simulation
Why Joon sees simulation as surprisingly similar to painting
Using simulation to study questions like UBI
Whether we are already living in a simulation
Why AGI and simulation may be the twin technologies of advanced civilizations
00:00:00 Introduction and Joon’s Path from Art to AI
00:01:46 Smallville, Generative Agents, and the Origins of Simulation
00:05:03 “Let’s Just Create a World” and the Future of Personal Agents
00:09:53 Social Physics and Behavioral Foundation Models
00:14:08 Prediction vs. Simulation: How Do You Shape the Future?
00:16:59 How Simile Models Real People and Populations
00:25:35 Evaluating Simulations, Digital Twins, and 85% Accuracy
00:30:23 Post-Training Models to Reproduce Human Behavior
00:40:04 Scaling Laws and Simulating 8 Billion People
00:43:10 From Schelling to Society-Scale Agent Simulations
00:46:13 The Cost and Economics of Simulating the World
00:52:05 Real-World Use Cases, Synthetic Populations, and the Market
00:57:27 The Future of Simulation, Painting, and UBI
01:04:23 Are We Already Living in a Simulation?
01:06:08 Building Simile and Hiring
Transcript
Introduction: Joon Sung Park, Simile, and the Story So Far
Vibhu [00:00:00]: Today, we have Joon in the podcast. Excited to kick this one off. Very exciting company. I wanna kick off and ask you the question, talk us through the story of your life. How have you gotten here?
Joon [00:00:13]: Yeah, for sure. I’m really excited to be here. A story of my life. So I was born in Korea, and I lived there for a good 11 years or so of my life, and then my family moved to Boston. So we moved when I was 11, and my parents were doctors, so they were going through their postdoctoral studies. My dad was a surgeon, so he was doing his sabbatical years at the Boston Children’s Hospital. So I grew up there, not too close to tech. I was very much a music and artsy, painting kind of guy.
Vibhu [00:00:49]: Painting.
Joon [00:00:49]: Exactly. I got into painting a little bit later, in high school, but that’s what I used to do. And then I grew up mostly in the East Coast after Korea. So I lived a good number of years in New Hampshire, and then I went to college in Pennsylvania. And I got into more of this tech scene, in college. So I was originally trained to be an artist. I thought that would be my professional career. So it wasn’t a hobby. It was like, “Hey, let’s make a living out of this.” And then gradually, I got really interested in this idea of, hey, the greatest artist often creates their own medium, and the best medium that we had available today was in computation. So I decided to go deeper into that, and one thing led to another, and we can go deeper into this, but I decided that research was something that I gradually got interested in, and here I am.
Smallville, Generative Agents, and the 2023 Breakout Paper
Swyx [00:01:46]: So there’s a lot that you packed into the research components. You had one of the best papers of 2023, which was the generative agents paper, commonly known as the Smallville paper.
Swyx [00:01:58]: Feel free to call back to anything else that you mentioned, but most people would have heard of you from this. Do you have any statistics on how many people have, like, read it? arXiv gives you something, right? Some stats.
Joon [00:02:10]: Yeah, it’s a good question. How many people have read it, I’m not sure.
Joon [00:02:14]: I know we do keep track of citations, and they are going up quite fast.
Swyx [00:02:23]: Yeah, Google Scholar has 7,200 citations.
Vibhu [00:02:25]: I feel like it made a bigger hit than that, and it was a pretty instrumental paper. It got cited so many times.
Swyx [00:02:34]: It is frequently the answer when people ask, “What is the best paper you’ve read recently?” It’s this one.
Vibhu [00:02:39]: I thought the memory component was pretty underrated. It was a very good early memory system, and one of the biggest papers.
Foundation Models and the Search for Killer Applications
Joon [00:02:47]: Yeah, so maybe I can talk a little bit about how this particular paper came together. So when I got into research, it was back in 2020 when I started my PhD program at Stanford, and that was the year, when we were about to get GPT-3 to be available. So we already had GPT-2, and you could sense that there was this new class of models that was just becoming available in the market, and the team got very intrigued. And the general consensus was, “Well, is this model going to be useful for anything?” “It’s really strange that these models are not trained to do any particular task.” But we decided to take a bet. So a large group of scholars at Stanford, and it was led by one of my co-founders, Percy Liang, and we came together
Swyx [00:03:35]: Who coined foundation models.
Joon [00:03:36]: Who coined the term foundation models. We wrote this paper, where that term came from called Opportunities and Risks of Foundation Models. And during that process, really the thing that I started to think deeply about was, here is a model that is fundamentally new in our ecosystem. The reason why this was new was it wasn’t, again, trained to do anything in particular, but its premise was it could do anything and everything. It was like a stem cell, if you were to take a biology analogy. And I got really interested in this idea that, well, if we were to really think about what are the killer applications that this particular technology would enable, what would that be? Many of my colleagues were using this for simple classification, simple generations. Interesting that these models can do that, but from an interaction perspective, not that interesting. We’ve known how to do that for many decades. And what we came down to was these models are trained on this very broad data from the web, right? So these are human behavioral data. It’s social media, Wikipedia, all these data. So if you poke at the right angle, then you could see human behavior that would just pop out that’s quite realistic, and we’ve never seen that before.
The Time Machine Game and Recreating the World
Joon [00:04:45]: So that got us really interested. The exercise that we decided to do, with this particular group of colleagues, Michael Bernstein, Percy Liang, and myself, who ended up becoming my co-founder at Simile, we sat down and we played this game that we call the time machine game.
Joon [00:05:03]: Imagine we were to get on a time machine and fast-forward 10 years and look back. What would have been the single application that will have mattered that would be the most interesting and inspiring? And when we thought, “Well, what if we can just recreate the world that we live in?” it’s really hard to get more ambitious than that. Like, let’s just create a world.
Joon [00:05:24]: And that’s where we started. And initially, we had this paper that was a precursor to the generative agents paper called Social Simulacra.
Swyx [00:05:32]: Before you go further, were there other candidates for the most ambitious thing in the time machine exercise? What was number two or number three?
Personal Agents, User Models, and Why Simulation Came First
Joon [00:05:44]: There is a close second that we were considering, which ended up becoming more of these automation tools, especially the vision around really personalized agents that would do things for you.
Swyx [00:05:59]: That’s also happening.
Joon [00:06:00]: It’s also happening. But it was interesting for us, right, in that the reason why, we decided to go with the idea of simulation, one, I was a huge science fiction nerd, and this idea of creating simulation, I was personally really just fascinated. I loved the idea. It’s really cool to see, like, a game town like this and just see these agents live in it. But at the same time, my bet was if you were to create a really amazing personal assistant out of this technology, what you need first is an amazing model of your users. So I told a model, “Hey, can you go buy late dinner for me?” And it orders Hawaiian pizza, and I do not like pineapples on my pizza. Then it totally failed. The way for it to not make that mistake is only by having a deep understanding of who I am. And I gave a very simple and dumb example here, but you can imagine how this core understanding of people is instrumental. This is how, if we have our family and closest friends, they have a good mental model of who we are. That’s the basis of our social connection. So our bet also was this technology around simulation, creating accurate representation of people ought to precede the more complex agents that would automate the world that we live in. So that was the bet. But that was a very close second, and I’m still very much fascinated by it. I think there’s a lot of interesting work that’s going around. My hot take here, though, is I don’t think we’ve seen a true personal assistant that’s useful, in ways that meet the ambition of that particular line of work. I think there are early applications that are interesting, and if you talk to even ChatGPT nowadays or Claude, they know a lot about us. So a lot of the generation it’s doing, I do think it’s much more tailored, but I think the ambition is quite large in that field, and I don’t think we quite have all the right ingredients just yet.
Swyx [00:08:01]: So OpenClaw and these personal agents, what do you want to see from them that they don’t currently have?
Memory, Markdown, and the Limits of Prompting
Joon [00:08:09]: I do think it’s slowly getting there, but I do generally want them to have much deeper understanding of the person. Right now, you look at the models. OpenClaw, what it’s leveraging is a Markdown file, and I think it’s quite clever, right? So if you look at the generative agents paper, this was the same intuition that we had, where initially when we were creating the memory architecture for the generative agents, and, like, this is, like, back in 2022, so we didn’t really quite have the idea of even agentive architecture or the term agent. But the intuition that we shared with some of the work that’s coming out today was we initially thought, “Well, do we want to make the memory into, let’s say, knowledge graph? Do we want to train a bespoke model?” All of these things. And what we decided to do was, “No. Just forget about all this.” These language models are quite good at modeling text and understanding and reasoning about text. So just put everything in a Markdown file or a text file. You’re done. I thought that was quite interesting that we could do that, and there’s a lot of strength in doing that. But also, there are limitations. It’s the way you retrieve and make sense of data that’s extremely large, it takes a lot of work. So I think that technology is getting better. I also do, however, think, there are certain things you just cannot shape just by prompting the model. So to some degree, you do need to touch the parameters of the model itself. So there is this work that I do think does need to happen, and it is happening. The question is, how far can we take it? How do we source data, and how do you also create an ecosystem where people are continuously feeding data to this model so it’s learning about you?
Vibhu [00:09:50]: What’s the intuition between why you need to do it in the model?
Social Physics and Behavior Foundation Models
Joon [00:09:53]: My intuition behind the actual when do you train or even post-train a model versus just prompt a model is if the model has to learn the underlying physics of the world that it’s operating in. So it has to learn new social physics. The places where it doesn’t have to train are the places where it already has the physics. We trust the physics. It already has the base statistics, but it’s just trying to react to an environment. Then I think you can just prompt your way into getting the actions out of it. I don’t think the models that are out in the open have yet learned the complete mapping of social physics of humanity. This is one of the core theses of Simile, right? And one of the core reasons why that is the case is if you look at the data that the model was trained on, these models were trained on the web data, like, whatever was available on the web. And these are really interesting data sets, but they are fundamentally the self-exposed attitudinal data with some behavior data that’s sprinkled around here and there. And it has yet to learn the really deep behavioral nature of people, not just what people say they do online, but what they do in real life. And this is one of what I would consider to be the dark knowledge of humanity that we haven’t quite captured. And it’s these data that would also need to get factored into the model creation.
Vibhu [00:11:21]: You call it behavior foundation model.
Vibhu [00:11:23]: There’s a good one-liner here, but outside of that, what type of data do you need? What are you changing on the model level? How do you go about modeling, doing a behavior foundation model?
The Three Data Buckets: Interviews, Behavior, and Causality
Joon [00:11:35]: We think about data in three buckets. So one bucket is interview data. It’s quite interesting. Rich qualitative data is interesting. It’s not behavioral, but we would literally ask people, “Hey, tell me the story of your life.”
Vibhu [00:11:53]: It’s just what we’re doing here exactly.
Joon [00:11:54]: The question that you all asked at the beginning of this interview literally is the question we also ask. And we ask our participants to go a little bit deeper, than how far I went. Maybe I can give more of my life story in lieu of this. But the reason why that data is interesting is by learning about this very long-tail information about people, you get a lot of texture around this model, like, this person as a model. So even understanding their childhood memory or even their trauma, their first love, these things, quite informative in ways that’s really hard to predict. So that’s one. Then there are two tranches of what I would consider to be the behavioral data. One kind of behavioral data is observational. So these might be like transaction data, or these might be data that you can get by scraping the web, right? So you can imagine why these data sets would be interesting, right, because they give you the base statistics of people’s behavior.
Joon [00:12:55]: But then there is the last category of data, that I personally think is perhaps the most important, which is the data that describes the causal mechanism, the whys of people. Some of this is covered by the interview data, the qualitative, because people talk about why they made certain decisions. But really, where you get to see the most behavioral aspect of this is in randomized controlled trials, like RCTs. Imagine you have the same setup, but you have a few different variables that you are trying to tweak. Can you get realistic human behavior out of it in ways where, imagine you had this particular option. Imagine you’re even trying to choose whether you’re going to drink coffee or not. The day you drink coffee versus the day you didn’t drink coffee, does your behavior change? That’s a data set that describes a causal mechanism. This is quite important in modeling people. The reason why this is important is oftentimes when people come to us, or not just to us, but the reason why people are interested in simulation isn’t because they want to predict the future. If you’re trying to win against the stock market, predicting the future is interesting.
Prediction vs. Simulation: Shaping the Future
Joon [00:14:08]: But most people, most decision-makers, what they want to know is, how can we shape the future? It doesn’t really help you to hear that your sales are going to tank in two quarters. They’re just gonna say, “Wow, that sucks.” What they want to know is, well, what do we need to do now to avoid that future? That’s the causal mechanism. And this is also very hard data to come by, right, because the world is our ground truth, but it happens once. So in a very controlled setup where everything is equal except for one variable, this kind of data set rarely happens. So this is a reason why this data set is both hard to come by and quite important if you’re trying to model human behavior.
Swyx [00:14:50]: So behavior, I think, is the hardest data set to acquire. What is out there? What is even possible? You’re not going to know a lot of details about my life. I don’t even have data for myself on my own health or habits, and I just don’t log everything. So how can you have that data?
Joon [00:15:14]: So we run a lot of randomized controlled trials.
Swyx [00:15:17]: But you put people in the lab, they watch them sleep, or what?
Joon [00:15:20]: We do care a lot about the consent process. People know that we invite them to be a member of this community to both share data and have themselves represented in different forms. But we bring a lot of people to the lab, or virtual lab, where we design experiments that would pose them real behavioral decisions. And often in these experimental setups, what makes the difference between what is attitudinal versus behavioral is whether the stake in your decision is real. That’s ultimately what makes it behavioral. So in these setups, we are inspired by our colleagues in social sciences, psychology, and so forth. So when they run studies, the techniques they utilize is imagine there’s an online store that you’re inviting people to come by. Then whatever they purchase in this experiment, they actually get that item delivered. Like, these are the things that make the stakes real. So we run a lot of these experiments, and we also do partner with firms. Right now, we also have customers who are quite excited to at least give us a glimpse of the behaviors that their users exhibit so that we can get a little bit deeper understanding of how people behave in these different platforms.
How Customers Use Simile: Populations, Queries, and Experiments
Vibhu [00:16:39]: I think on the customer side, they have a lot of data about their users, who has bought. They have the action data.
Vibhu [00:16:47]: Can you walk us through an example of what someone comes to you for? What questions would they want solved? Do you customize a model for them? Do you have something off the shelf? What does that look like?
Joon [00:16:59]: Today, when people leverage our models, it’s often to better understand the population of their interest. So usually, the start of the relationship, we come together and hear about what population they want us to model, right? So it might be that if you’re a CPG company that’s selling to all of the US, then maybe it’s fairly straightforward. You want to model the gen pop of the US. But at the same time, if there is a vertical or if there’s a market that they’re trying to go into, imagine, they want to better understand, let’s say, people in their 20s and 30s living in California. That’s a much more specific population. So we hear about this population, and we go recruit these people, with consent, and with incentives, and we collect some of their data and create a model of these people. Then what our product allows you to do is query them. So it can take as input a filter that is a description of the population that you want to talk to, just like the one I just mentioned, and an environment. The environment can literally be survey questions, behavioral experiments, It can be A/B testing. Oftentimes, the core use cases are things like concept testing, to start with. But also, people sometimes want to do focus groups or one of the fun use cases that we also serve is even modeling things like earnings calls for public companies.
Joon [00:18:21]: So these are the use cases that we often start with.
Swyx [00:18:23]: Concept testing, is that an established term? I’ve never heard of concept testing.
Concept Testing, Gallup, and Politics
Joon [00:18:27]: Yeah. So it has to do with they have, let’s say, different messaging, different products, different ideas.
Swyx [00:18:32]: It’s like a marketing exercise.
Swyx [00:18:33]: Okay, got it. Got it. Politics?
Joon [00:18:36]: We do, have a strategic partnership with Gallup, and of course, Gallup is deep into policy space and so forth. Right now, we have not worked deeply with politics, like that area just yet, however.
Swyx [00:18:49]: I’m curious if there is demand or if they really would have different needs that somehow fundamentally don’t mix with your existing, users or people.
Joon [00:19:00]: I think there’s certainly demand.
Joon [00:19:02]: But we are very much mindful of how this technology gets adopted and the societal impact that we’ll end up having with this technology. And I do see politics as an area where a company has to be particularly thoughtful about the way they operate and make impact. So this is where we also want to make sure that we form enough of guardrail and perspective on how to leverage this technology before we go on to serve markets like the politics.
Swyx [00:19:29]: I’ll give people an example. one of my favorite shows is The West Wing. I don’t know if people have watched.
Swyx [00:19:34]: One of the key storylines is, like, the president has, multiple sclerosis, but they haven’t. they need to figure out how to disclose it. So they run a poll with a fake governor and ask people to respond on the poll,
Counterfactuals, Polling, and When Simulation Is Useful
Swyx [00:19:47]: They try to make decisions based on the results of that poll on, like, how well they’ll be received, like where, how should we play this?
Swyx [00:19:54]: And I’m like, well, I think those counterfactual things, I would use a simulation for this if I could trust it.
Joon [00:20:01]: For sure.
Joon [00:20:02]: In that show, how’d it go?
Swyx [00:20:04]: In that show, it was, like a foregone conclusion. They were like, “We know it’s bad. We just don’t know how bad.” And then the poll came back. It was like, “It’s really bad.” And then they just did it anyway.
Joon [00:20:14]: Part of it is to show, right? So you’re, you’re looking at the idea
Swyx [00:20:17]: Maximizing drama.
Joon [00:20:18]: How bad could it be? Oh, it’s horrible.
Swyx [00:20:20]: And to some extent, I think that is part of the trick of the, or the challenge or with being a customer of yours, which is that if I know it’s. if I roughly know and can intuit
Swyx [00:20:35]: What the effect is going to be, do I need you? What sensitivity of it, of effect do I need in order to make a decision, right? So for example, if I, my approval rating is 50%
Swyx [00:20:48]: And I, they have this negative piece, news item comes out, and it drops to 30.
Swyx [00:20:52]: If it drops to 20, if it drops to 40, do I care? No. It, I know it drops. It’s negative. So when do I care about simulations?
Joon [00:21:01]: You do something that’s clearly bad, that’s not popular, and people don’t like you, like, yeah, it’s like
Swyx [00:21:05]: You don’t need a simulation.
Joon [00:21:07]: Yeah. Well, so there are a couple of things. one is, there are use cases where, like every day, developers, designers, policymakers, marketers, every single day, they create assets. They create new products. And turns out, it’s many of the decisions in hindsight is obvious. Yes, of course this is bad, but we still run those studies because understanding the magnitude and understanding how acute something is quite difficult, even if, we feel like, of course, like this makes sense. this is the reason why we make so many mistakes. Like, every time somebody goes online and say something that has huge backlash, you look at that and like, “What an idiot.” However, it’s tough. That’s one. There’s also another aspect here, which is, again, this is the reason why simulation is different from prediction. In simulation, in the ideal case scenario. So what simulation is trying to show is it’s trying to show each step of the way or each step that we need to take to get to a certain outcome, right? So in the most advanced simulations, sometimes the next step that we’re suggesting might be quite counterintuitive. The analogy that I sometimes give, and I ground it in a more realistic example, but, I, as I mentioned, I’m a huge fan of science fiction, and I don’t know how, many of the audience members have read, like, things like the Foundation series by Asimov.
Simulation as a Path, Not Just a Prediction
Swyx [00:22:37]: Oh, yeah. We’ve mentioned psychohistory a number of times.
Joon [00:22:39]: Okay, fantastic. So I might be, talking to the right crew. If you read Foundation series, literally the first act is there’s a group of scientists who have found out that, “Oh, our galactic empire is going to collapse, and we’re going to have 30,000 years of unrest.” And they run psychohistory, the simulator that tries to teach them, “Okay, how can we keep this unrest to a 1,000 years?” And they plan this out, and the first step of that plan is to get the scientists who say, “Okay, this is coming,” exiled into this random place in this, galax- galaxy.
Swyx [00:23:18]: Terminus.
Joon [00:23:19]: Exactly. And that’s so counterintuitive. Like, what a strange move that you literally sent the group of scientists who was raising voice around this potential collapse of galactic empire into nowhere. How is that the right first move? Well, it turns out in this particular simulation, that was the move.
Joon [00:23:40]: It’s these things, right? And the reason why these reasoning is possible is because you’re showing the step function or each step that results in a particular outcome. So really what simulation allows you to do in its highest form is you give it not a problem or question, like what would people answer to the survey? That’s not what we do. What we tell it is, “Here is a goal that we have. In the context of foundation, we want to keep the unrest to a 1,000 years. What is the path that we need to take now to get to that particular future?” And that’s what simulation allows you to do. Now, translating that into real market, imagine you’re a automobile company and you’re about to release a, EV, and you’re trying to understand, well, how do we market EV, to make sure that our stock price goes up? But what if the answer comes down that, well, you can market your EV in XYZ way, but that might change people’s perception around the cars that’s not EV and make your overall sales to go down. Not very intuitive, especially all you’re trying to optimize is EV salesss, and that’s the only thing that you’re tracking, then that might result in a completely wrong solution, or at least different solution than what you would have expected, whether it’s right or wrong.
Joon [00:24:57]: That’s the power of simulation.
Swyx [00:24:58]: For listeners, we covered a similar topic with Mikhail Parakhin from Shopify, where they are working on SimGym. I don’t know if he ever talked to you about it. it’s very similar.
Joon [00:25:07]: I
Swyx [00:25:07]: The goal is increased conversion, but then the journey is very unusual.
Joon [00:25:12]: Journey is unusual.
Swyx [00:25:12]: Yeah. The-- He’s trying to look for interventions on a shopping trajectory, which is similar to what you’re saying. Like, it’s not about the attitudinal, is your word for it.
Swyx [00:25:24]: It’s about behavior.
Joon [00:25:25]: It’s about behavior.
Swyx [00:25:25]: And that’s exactly the difference, right? It’s, like, not about the near-term direction about-- but it’s more about, like, how do you affect multiple turns of interactions.
Vibhu [00:25:35]: You had a good quote at the start about this as well. It’s not about people wanting to know the outcome. It’s about how they can change it, change the way to get there, something like that. But I wanna take it back to how do we know this is grounded? Like
Grounding and Evaluating Digital Twins
Vibhu [00:25:47]: How do you run evals? How do you test that simulations come through? if I was to do the same thing that you described with, say, your favorite LLM, Opus, GPT-5.6, have some agent to map out these things
Vibhu [00:26:02]: How different are the answers we would get if I give it the same goal, the same objective, make a decent system? You’re saying that you need to change the model weight. You have your own solution to this. But how far off are we, and how do you check if it’s grounded? you have some interesting stuff on your site that points to how you run real evals, but if you could take us through that side. I think that’s one of the big concerns that people have. They’re like, “LLMs hallucinate.”
Vibhu [00:26:27]: “You’re just hallucinating layer after layer,” right?
Joon [00:26:30]: The way we do this, and this is the paper that we worked on after the generative agents paper that really became the, at least for Simile and also the field of simulation and synthetic panels, really became the foundation. Yeah, this is the paper. the paper is called Generative Agent Simulations of 1000 People. Here’s what we’ve done. For this paper, we brought 1,000 people that’s representatively sampled from the US to a virtual lab. And what we have done was we spent two hours collecting fairly wide-ranging data. In this particular study, we focused a lot on this interview data, that was, whose script was taken from this project called American Voices Project. And then we would also pair that with a lot of behavior data and so forth, whatever we can collect within two hours. And then we would send these people away for a couple of weeks. And during that time, I would use this data to create their digital twins. And I would bring the humans, participants back after 2 weeks and have them complete a battery of surveys, experiments, behavior studies. So we have the list here, which included things like behavioral economics games. We would run literally, like, Big Five personality test, General Social Survey. We would also go ahead and run the randomized controlled trials that were published on PNAS. And we would have their digital twins predict how the source individuals would have acted in these studies and surveys. And this is where we could replicate people’s behaviors and attitudes 85 percent as accurately as people would replicate their own. So that was the first really paper that gave this validated results that we can model individuals in an accurate way. And what we ended up finding now, of course, in AI space, so this paper came out at the end of 2024. AI space, a year and a half, 2 years, that’s a lifetime.
85% Accuracy and Why Frontier Models Miss Human Behavior
Swyx [00:28:24]: Yeah. Just, for listeners who are not seeing the YouTube, I just wanna say, like, the headline figure is 85 percent accuracy, like, which is a big improvement over all the other
Swyx [00:28:34]: Methods that you showed.
Joon [00:28:36]: But the part that was particularly striking to us, especially as we improved this technology even further, was the generative AI models like ChatGPT, Claude that’s coming out, it does give you the right foundation. However, what they do not consider is the true attitudinal and behavioral aspect of people, especially in the population that you care about. So what these models are really good at today is they’re trying to become the super rational, objective machines, right? So you go get their data from places like Mercor, Scale. You talk to professional programmers, scientists to create model that’s amazing at reasoning. That’s what they do. Simile doesn’t care about any of this. The models that we’re talking about here, what we’re trying to create are models that are as dumb as I am, right? So if I make some mistakes, the model has to make the same mistake.
Swyx [00:29:34]: Oh, that’s very hard.
Joon [00:29:35]: That’s very hard.
Swyx [00:29:36]: You’re solving Murphy’s paradox.
Joon [00:29:37]: That’s exactly. And this is a completely different data and training objective. This is also where we see quite a bit of discrepancy in the performance in human behavior prediction between the frontier models, Simile’s model, and the models being created in this space, where in some cases, the model performance of frontier models go all the way down to 20, 30 percent, especially if you go into that more niche population on topics that our customers would care about. On more gen pop, it might be around 50 to 60 percent. So it’s not very robust. Like, you wouldn’t want to make your decision off of these and these findings. If you can bring that up to 85 percent, that is ultimately what people end up getting very excited about.
Swyx [00:30:20]: Yeah. Do we wanna keep going on the paper, routes?
Joon [00:30:23]: Yeah, for sure. So the last one, was an interesting one. So this, paper was the follow-up paper that we had, to the 1000 agents paper, where the idea was now can we augment the models even further and post-train a model based on a lot of randomized controlled trials? So this was an interesting one. The data is always the most interesting part of modeling in many ways. The data that we got here was there’s this, there’s this platform called Open Science Framework. So some, the audience might be familiar with this. And there has been, especially in the social sciences over the past 5 years or so, there has been this concern around replicability of studies. And so it was a bit of a crisis, the scientists acknowledged, where we rerun the study and we don’t see the same finding.
Post-Training on RCTs and Replication Studies
Vibhu [00:31:12]: Oof.
Joon [00:31:12]: It’s tough. And the reason why it’s there-- that was often the case was there’s this survival bias where the papers that get published often need to maintain what we call the value of less than 0.05 in the experiments that we ran. That suggests that only-- there’s only 5% chance that the results that we saw is false positive. But the tricky part was all the papers that were not published, and there’s still a 5% chance that whatever we publish is totally just randomly generated. Like, there’s a 5% chance that, hey, this effect is not real, but it just happened to be real because of the sampling bias. So because of that, what scientists started to do was they started to register their studies. So before running an experiment, they would go to this platform and say, “Here is the data. Here is the population that we’re collecting, and here’s the hypotheses.” And they would just say, “Here is our hypothesis.” Like, “This is what we believe.” And you cannot retroactively change those hypotheses. This is what gives us more scientific statistical confidence that whatever effect that you ended up seeing is true. So that ended up creating this really interesting platform where there’s one platform that has now contains tens of thousands of real-world experiments and hypotheses. And a lot of these are really high-quality, like, professionally designed behavior studies and random- randomized controlled trials. So we got the data and the studies from this platform and used that to make a point. And this particular, model is not, something that we’re serving commercially because this was a part of the open science. But this particular data set, helped us make a point that by collecting a lot of these randomized controlled trials, that are really well-designed, we can make significant improvement in model’s capability to predict human behaviors. So that’s what this paper was about.
Vibhu [00:33:10]: Is this stuff done on a individual level? Like, do I need to tune the model per individual, per company? Is there foundation model changes and then some slight post-training? Anything you can share there?
Population-Level vs. Individual-Level Models
Joon [00:33:21]: So this particular model was trained. the data we had at the level of individuals, but this particular model was trained. We experimented with both. And this is what we end up doing at Simile too. We always train 2, distinct model. One is what we call the population-level model. The other is what we call the individual-level model. And both take very similar input, which is the description of a subpopulation or individual and a stimuli. In this particular work, we’ve done the same. Here, the results that we are reporting are much more geared towards individuals because we do think that is a harder task in many ways, but that’s what we have done.
Vibhu [00:34:02]: You seen anything on the questions that humans can solve that models can’t solve? So like
Human Biases, Mundane Choices, and What Models Miss
Vibhu [00:34:09]: Currently, it’s, I live 5 minutes walk away from a car wash. It’s a 10-minute drive. Should I walk or drive?
Joon [00:34:16]: Huh.
Vibhu [00:34:16]: The model will say, “Oh, walk to the car wash.” And, you don’t have your car.
Vibhu [00:34:20]: Is anything like this a problem in simulation? You would assume, like, very simple for human to think about, but if the model is saying you should walk to the car wash, anything here?
Joon [00:34:32]: It’s less, what can we solve, but I think it’s more about what biases or mistakes do people make that models miss. Like, imagine that you are, like the. When I was still at Stanford, I lived in Palo Alto. So it’s about, I would say, 40-minute walk from the campus. You ask the model, “Okay, let’s go home. What can I, what can I do?” It would likely call an Uber or, give me, the bus time. But for the longest time, I really liked walking back. And the reason why I wanted to do that was not for efficiency. It really helped me think. And I like to walk for, half an hour or 40 minutes or so a day, where I just get to, just think about ideas, research, just get lost in my thoughts. That’s very human activity. Unless the model has seen that and understands the importance of that activity, it would miss these kinds of features. So that I think, is fundamentally what we’re trying to model. Like, what is fundamentally human might not be the most efficient thing to do, might not be the right thing to do, but things that make us who we are.
Swyx [00:35:43]: I’m curious if, there are some data sets that you really want that would materially help you. One version of this may be interesting, which is more valuable to you to acquire as a data set, all of LinkedIn, all of Twitter, all of Facebook?
What Data Matters: Social Media, Transactions, and Facebook
Joon [00:35:57]: It’s a little bit hard to rank, in part because, there’s, there’s this product saying where no feedback is wrong because it teaches you something about your users. Doesn’t matter what feedback.
Joon [00:36:11]: I think it’s a little bit like that.
Swyx [00:36:12]: So just whatever is bigger.
Vibhu [00:36:13]: What about a different domain? Say it was. What about all of Amazon data?
Joon [00:36:17]: Oh, yeah.
Vibhu [00:36:18]: Shopping data, right?
Joon [00:36:18]: Shopping data. So Amazon data is interesting in that it’s very much behavioral, although, like, what people do on social media, you could squint and say that is also behavioral. But the transaction data is always interesting. It is also most commonly available, however.
Joon [00:36:33]: If we were to look at purely social media, like if you really, if I were, if I had to really pick, Facebook likely is interesting because I do think it is most a default version of people. Because you go to LinkedIn, it’s very much professional environment. So people put up their, they have their guards up, right? And that still is interesting because that is true human attitude and behavior, but it is not your base state. you go to Twitter- Twitter, people have their own crazy personas, or depending on who you are. Like, my Twitter profile and, persona is very much, initially was I was very much an academic. “Hey, I’m here to share my studies.” Now, I share, things that’s related to Simile. But Facebook is one of those more private space where people just connect with their friends. In that way, I do think it shows you a little bit more about who that person is. So if I had to pick, I’d likely pick, Facebook.
Swyx [00:37:30]: Yeah. And you’re interested in, like, the whole person and their background and philosophy. I, is it too clinical or too machine learning-oriented to just say this is just ways to inject variance and biases? The broad question, is, like, is this any better than a randomized, like, combinatorial explosion version? So we have a link to the Tencent
Billion Personas, Synthetic Demographics, and Bespoke Data
Swyx [00:37:54]: Billion persona paper, where they did not do any of the groundwork that you are doing.
Swyx [00:37:59]: They just did like a cross matrix of here’s all the professions in the world, here’s all the people, possible backgrounds in the world, do a dot product across all of them, and that’s it. That’s your prompt for a billion people.
Swyx [00:38:12]: This will do something. I don’t know if it’ll do what you do, but it gets you some way, some percent of the way there.
Joon [00:38:18]: So this was an interesting paper. Like, what I admired about this paper when it came out was the scale. And you do gradually want to be able to simulate really large societies and interactions. So the scale is definitely admirable. it is relying heavily on the known statistics that went into training the model. So to the extent that you believe that statistics is correct, this is not a bad way to go about this. But the thesis here, and this is something that we also have seen in the market, like if this works, then we have solved simulation.
Joon [00:38:54]: It,
Swyx [00:38:55]: Because I survey, like, okay, 5% of the US population is in construction.
Swyx [00:39:01]: The other 5% is in medicine, whatever, right? And then you just keep going down the list, and then you do the other side. 5% has, like, the big 5 personality
Swyx [00:39:08]: Of, like, neurotic or whatever. That’s it.
Joon [00:39:11]: That’s it. So if you believe that the underlying data set and the platform that we’re leveraging has all the right statistics, then this will have solved it. you’re at that point merely retrieving the knowledge that is already embedded in the model, in the model parameters. That’s not, unfortunately, what we see, where there is such detailed and also niche knowledge about people that if you just take one example, it might feel very mundane, but it’s quite rich when you put together, that you do need to do a lot of bespoke data collection to better understand people. And this is also, I think what makes this particular, job fun, which you want to deeply understand people, and the process of deeply understanding them requires a lot of attention to the details. And you do need to pay attention to and pay respect to the daily lives that people lead.
Scaling Simulation: From Thousands to Societies
Vibhu [00:40:04]: I wanna talk about scaling simulation.
Vibhu [00:40:07]: So what can’t we simulate, what can we simulate, and how does scaling affect this? So how big are the models? What if we go from, 8B, like, couple 100 billion
Vibhu [00:40:18]: Like billion000 parameters, billion000? Do we get scaling? Any interesting emergence? Like, at a certain scale, at a certain amount of training, you uncover anything unusual and any learnings from that?
Joon [00:40:31]: What we are seeing is at Simile, so we do post-train our own model. The thing that we’re seeing is the early glimpse of scaling law in simulations. The more data about humans and more compute you ingest, you start to get predictive and predictable gains of the model performance in simulating it, simulating people.
Vibhu [00:40:51]: Ooh. We need a scaling law curve.
Joon [00:40:52]: It’s scaling law. Whenever you find it’s a beautiful thing. And we’re starting to see the glimpse of it, which is quite exciting. But if you talk about the ambition of simulation as a whole, it’s not merely about building a model. It’s about building a model, then creating the agents that become the individuals in a much larger ecosystem. So they’re creating this multi-agent simulation. Down the line, you want these multi-agent simulation to also live in a very rich environment, right? What we are really trying to get to at that point is, hey, can we create. All right, let’s do a time machine game again, and 5 years, 10 years into the future, can we create a simulation of 8 billion people living on Earth? I think that’s quite interesting. And that really is the vision. And once you get to that state, the questions that you can help answer for the society also start to change from my perspective. The answers are fundamentally about emergence of the emergent behavior of society and large groups of people.
Joon [00:41:53]: So the questions that I get excited by, and maybe this is a stodgy- a bit. I have my, academic side of me.
Joon [00:42:01]: And for me, it’s questions like, can we help solve climate change? If you look at climate change as a problem space, this is what we, like social scientists would often call it the wicked problems, problem where you have many actors with competing incentives for trying to make a very complex decision and coordinating that coordination decision. Very difficult to really solve in real life, which is also the reason why we couldn’t solve it. Can simulation help us solve that? Another one is, can we understand the signals for collapsing democracy, or can we understand or can we uncover the origin story of the monetary system? These are societal questions that we never really had a good way of answering. If we can create simulations of our society, you have to believe that these are the problems that we can solve. So that’s really the ambition of this field. And, I also think, yes, I think there’s a Nobel Prize to be won there, which wouldn’t be surprising. And I think there’s some amazing societal impact that we can have to help people make better decisions.
Climate Change, Democracy, and Societal Simulation
Swyx [00:43:04]: Nobel Prize in economics?
Joon [00:43:06]: In economics.
Swyx [00:43:06]: Oh, I see. I see. Rooting for you to write that paper.
Joon [00:43:10]: One of these days. But, one of the scholars that I was deeply inspired by, When I was coming into the space of simulation, is this scholar, named Thomas Schelling.
Schelling, Agent-Based Models, and the Nobel Prize
Swyx [00:43:23]: Schelling point?
Joon [00:43:24]: So the canonical example of the work that he’s done was he was one of the creators of agent-based modeling. So this was, like, in the 1970s and 80s. It’s very early days, but this was truly one of the first exemplars of simulations. And one of the canonical model from that time, and of course many of these simulations are trying to tackle the societal problems that’s most relevant for their era, it was called the model of segregation. So racial segregation was a big topic, that, we cared about. And what they’ve done was they created this grid world where they had red dots and blue dots. And these dots were, back in the day, like, they were the agents, and they had a simple rule that governed their behavior. If certain percentage of your neighbors are of different color and if that goes above certain threshold, then you move to a new location at random.
Joon [00:44:21]: One of the striking finding of this paper or this agent-based model was for the longest time, people thought the segregation within society was caused by explicit and overt racism.
Joon [00:44:34]: But if you look at this model, people’s preference towards living with people of the same color, that preference can be very minute.
Joon [00:44:42]: But the very small difference causes the society to segregate completely over time. This was very counterintuitive for a lot of people. And this particular work ended up informing housing policies. Mixed income housing, got really inspired by this work. And Thomas Schelling ends up winning the Nobel Prize for having laid the groundwork for very early versions of simulations. The opportunity that I do see here in the more scientific terms, is agent-based models for the longest, had impact in the 1980s, 90s, to some extent, early 2000s, but it has now gotten forgotten by the community a little bit. Because as you can imagine, red dots and blue dots is not really a rich description of people.
Joon [00:45:31]: But with the emergence of things like generative AI and, in particular, generative agents, we do have an opportunity to create these agent-based models that are high fidelity enough to help us make really complex decisions. And that’s the opportunity that I see. If that truly works, then yes, that is the work that will result in a Nobel Prize.
Swyx [00:45:53]: Yeah. For what it’s worth, and I grew up in Singapore. 80% of Singapore is in public housing, and public housing has, enforced racial quotas for exactly that reason, which is very interesting. okay, so we talk about scaling, we talk about all these, the agent possible applications.
Cost, Reuse, and the Economics of Simulation
Swyx [00:46:13]: I’m scared about the cost. if you even-- let’s just keep it to the US, about 8 billion people.
Swyx [00:46:21]: But, how much does it cost to model so many hundreds of millions of people?
Joon [00:46:26]: Oftentimes today, we don’t start at that scale, this stage of the, of industry and simulation as technology. But we can get our users extremely rich and meaningful insights even by modeling thousands, tens of thousands of people. And today what we do is every week we are collecting data on the scale of tens of thousands people’s data, and we have panel partnerships that gets us to tens of millions of people globally. So that’s what we do today.
Swyx [00:46:55]: And just as a side note once you’ve collected one person for one study
Swyx [00:46:59]: Can you reuse that same person for all the subsequent studies?
Joon [00:47:03]: That’s exactly right.
Swyx [00:47:03]: Okay.
Joon [00:47:04]: The beauty of this model and these agents is the fact that they are domain-agnostic.
Joon [00:47:08]: That what you’re really trying to understand is what is the fundamental nature of these people? What’s their social physics? And there are a lot of, a lot of, people that does change over time. Like, even, like, even things like, how many times have you gone have you been to, like, CVS the past week? that will change. But there’s so many traits about people that are also known to never change. Like, your risk tolerance doesn’t really change over time. It’s very consistent. So it’s these things that we’re trying to learn. But the scale we are operating is right now hundreds or, tens of thousands to hundreds of thousands. And in many of the core use cases that we are deployed in, and this is more than enough population, to cover those. Really, at that point, what you care about is less the number of people, but more do you have the right subpopulation of interest covered? And this is also the reason why people want a larger sample. It’s not because they want, stronger statistical guarantees. It’s more that can they filter down to any population of their interest. However, you can also imagine in 10 years, if we truly believe that the compute is going to scale, that we’ll have much more availability for compute, and our ambition for simulation is also going to scale accordingly, there’s definitely a reason for us to create an entire data center worth of simulations.
Joon [00:48:35]: Or in my hunch here is I do think in the next some number of years, we will start creating simulations that will cost as much as training a foundation model. But perhaps it’s going to be so valuable to the society that it would be a no-brainer. Right now, even today, like, we are training bunch of new foundation model just so we can say we trained one and we spent tens of millions. But if we can create a simulation at the level of society that would solve climate change, I would run that today. I would raise the money right now just to run that.
Multi-Agent Simulation and Social Influence
Swyx [00:49:10]: Amazing. the follow-up question is, does it also compound if you let the simulations talk to each other?
Swyx [00:49:18]: Or do they already do that today? They don’t, right, as far as I understand?
Joon [00:49:22]: It depends on what simulation you’re trying to run.
Joon [00:49:24]: In the multi-agent simulation setup, the agents do talk to each other.
Swyx [00:49:28]: Right, which is exactly Smallville, right?
Joon [00:49:29]: That’s right.
Swyx [00:49:30]: But a lot of times, for example, in commerce, you’re just by yourself, so there’s no point talking. which is way cheaper.
Vibhu [00:49:37]: But they use all these levels, right? Like, you decide what you will buy based on what other people around you buy and talk about, right?
Swyx [00:49:43]: It depends.
Vibhu [00:49:44]: It depends.
Swyx [00:49:45]: Again, I’m, I’m coming at this from a cost point of view. I’m like, “Oh my God.” Like
Vibhu [00:49:48]: I think
Swyx [00:49:49]: If there is, like, some combinatorial thing of, like, thousands of people talking to thousands of people, then that one million X’s might cost.
Vibhu [00:49:56]: I have a very different view as the cost point aside. Like, running these studies in reality is a lot more expensive, right? Running any study like this is you gotta have people do it, you gotta sign people up. It’s very expensive and sometimes, like, not feasible to run the study.
Vibhu [00:50:14]: But the outcome or the decisions you make are very expensive on them, right? So spend X million on something that, the overall process costs 100 million might as well, right? There’s, there’s a lot of value to be had there. It’s a small cost, but I’m excited on the cost side.
Joon [00:50:33]: To some extent, and when you deploy technology, you often want to deploy in a way where you can replace existing budget or you can make things more efficient, and that is the best way to deploy. However, the way you capture the long-term value of the technology is making the argument that, no, it’s the upside, that by making this better decision using simulation, you have saved yourself or made yourself hundreds of millions or even billions of dollars, and that’s a case to be made.
Vibhu [00:51:06]: Random tangent question. So if you’re doing a lot of inference, a lot of model multi-agent stuff, are you at the point where it makes sense to, train a model that’ very sparse? You’re expecting to do multi-million dollar runs. Are you thinking about this in model architecture standpoint or inference efficiency, or, you’re still at the research phase of it works, we’re not super there yet?
Joon [00:51:34]: Efficiency, we do think quite a bit about. this is technology that is deployed now in some of the largest enterprise companies in the world, and we do process significant number of queries, that are trying to, simulate the populations in the world. So efficiency is a consistent thing. we don’t want to over-optimize too early, so I wouldn’t say, like, this is the higher bid Right now, but this is definitely something that we think pretty carefully about.
Swyx [00:52:05]: Yeah. Are there other case studies? So we, you talked about CVS, talked about Gallup, Deloitte, Wealthfront.
Efficiency, Enterprise Use, and Real-World Case Studies
Joon [00:52:12]: Wealthfront is an interesting one, because one of the things they were trying to do, they were one of the first customers that wanted to do product testing that goes beyond just asking people what they think about, let’s say, behavior experiments and so forth. So there, really what we had to do was reason about multimodal input, so images, but also you can also imagine, like, these agents traversing through Figma mockups or websites. So some of the things that our agents can also do is it can be given a domain, like, or, like, a website URL and go use it for a while. It’s these things. And Wealthfront was one of the first, customers, that was very excited about this possibility.
Vibhu [00:52:53]: What have people been asking? Like, is there any demand that we have not covered? Like, UI testing, right?
Vibhu [00:52:59]: I wanna try a new. I wanna ship a new feature, test the UI, simulate how people will do it. Any interesting things that you’re seeing demand for?
Product Testing, Websites, and Synthetic Panels
Joon [00:53:08]: Today, a lot of the demand does come from like, the places where people have historically used human panels, we can now replace with agents, and these synthetic populations. And this is not replacing human panel. in many ways, the simulation that Simile is building is grounded. So the way that I think about this is we are trying to represent humanity at scale. And in that way, the use cases are what we would expect, but it’s the scale of deployment that surprises me.
Joon [00:53:44]: Turns out there are so many decisions that people make every day in these organizations, groups, and we want to be able to say, “We listen to people. We have consulted our users.” But in reality, that is rarely the case because getting to people and asking them many questions, it’s difficult. It’s both costly, time-consuming, but most importantly, people are just not available. If I had to answer 1000 survey questions for this one particular, vendor, even if I wanted to do that, like, I would never do it. And that’s very much the case. What simulation can do is ensure that the voices of people are always represented in rooms where the decisions for them is made, right? So all the stakeholders of this particular product launch, ideally they’re consulted. That’s what this technology really is trying to enable.
Market Size, TAM, and Human Decision-Making
Swyx [00:54:39]: In my mind, that means it skews towards more consumer focus, right? Like, anything with a wide enough customer base where you do benefit from the diversity that you represent. What are some rough statistics, just for people who are not familiar with this market in general, what’s the market size that. I’m sure you have some, like, rough numbers. market size is, like, a vague question
Swyx [00:55:01]: But, like, how much do people spend?
Joon [00:55:03]: So market research is a $100 billion industry.
Joon [00:55:06]: But the thing about simulation is not a tool for market research. Simulation is a tool for human decision-making. So the question around what is a TAM here is quite tricky, right? Because it’s easy to say, “Well, market research TAM is roughly 100 million or 100 billion.” so is it a TAM? And not really, right? Because in many ways, you’re trying to inform all human decision-making. You’re trying to inform every decision that are made about humans for humans. What is a TAM for that? It’s really unclear. And I’ll be honest. Like, I have a scientific background, I have a research background, so I didn’t come into the field calculating, oh, what is the TAM for human decision-making? But I just had to assume, well, if we can inform every decision that is made about human for human, that has to be big.
Swyx [00:55:58]: Some- something valuable.
Joon [00:55:59]: Exactly.
Swyx [00:55:59]: To some extent, you are a unicorn founder now, and you have to care as a CEO. But, like, I do think, like, yeah, when you go into these boardrooms with people that you’re quoting millions of dollars of contracts for, like, you have to say, “Well, here’s what you spend on humans-”
Swyx [00:56:15]: “. And here’s what we save you, and it’s 85% similar.”
Joon [00:56:19]: And certainly, the value case, is something that we care deeply about. Like, what is the value that we provide to the users and the decision-makers? But this is also where, like, as a founder, I think valuation only tells one very superficial aspect of the story, and I try not to think too much about valuation, in general, because that’s not what also motivates a team or certainly doesn’t. I’m, I-- Again, the interesting thing about researchers is we are happy living in academia, getting paid next to. we get paid okay. we don’t get paid that much, as a researcher here in academia, but it’s the impact and it’s the, it’s the value that we can provide to the individuals and the society that really drives us. And in that way, ultimately what drives us is the impact. Does the simulation we provide have a real impact in people’s decision-making in ways that progresses our society forward? If the answer is yes, then yes. that has to be great business, and we see that in numbers, and we do care deeply about that upside story, but that’s the heart of it.
Where Simulation Goes Next
Vibhu [00:57:27]: Do you have any timeline predictions? So we talked about scaling laws of simulations.
Vibhu [00:57:33]: You brought up, okay, maybe one day we can simulate how to solve climate change.
Vibhu [00:57:38]: Where are we now?
Vibhu [00:57:40]: If that’s not the end state, what is an end state, and what does progress look like?
Joon [00:57:45]: So what I sometimes tell people is simulation as industry, it feels a lot like where GPT-3.5, GPT-4 was, for the AGI saga, which is we have now technology that is powerful enough to do real damage on the verticals that we are tackling. At the same time, there’s a lot of progress that is yet to come. And that’s, I think, where this is. So the way I see it, I do think there will continue to be breakthroughs both in data, in algorithms, and there will be much more aggressive scaling that will also happen over the next few years. But I think that’s roughly where we are.
Swyx [00:58:27]: I think that was about the rough set of topics. Anything else that we should have asked you or you wish people asked you more about Simile?
Simulation as Painting and Understanding Human Essence
Joon [00:58:38]: I think the, what’s, for me, what’s quite fascinating about simulation, it is very impactful technology, but it is also very interesting technology, both in terms of, like, what it means for human society, our philosophy. And the way I sometimes interpret simulation is. So going back to my background, I as I mentioned earlier, I started my career as a painter. it was a professional pursuit, and I did oil painting, for figures. So I got my training originally in the realism studios, and that’s what I spent a lot of my, years, doing. Simulation is a lot like painting, right? The best paintings teach you something deep about the subject that you’re trying to represent. And it is always not a perfect representation. It-- No painting is perfect. There’s always some small differences and discrepancy, but what it does is it tries to highlight the thing that matters the most about the subject.
Swyx [00:59:47]: The essential
Joon [00:59:49]: The essential essence.
Swyx [00:59:49]: Yes. He, you, he’s brought up some of your work.
Vibhu [00:59:53]: Just nice to put it up.
Joon [00:59:54]: Yeah. So these are some of the works. So this is from, my, personal website that I maintain when, I was still a researcher.
Swyx [01:00:00]: I think a lot of people will say, like a Picasso, like anything postmodern is, like, very much focused on the essence.
Swyx [01:00:09]: Right. yeah, but I don’t know if any one of these evokes something that you like to tell the story of.
Joon [01:00:15]: No, it’s one of those things where, each of these paintings, drawings, whatever it may be, it is trying to surface something about the subject that you feel deeply about onto the surface. when I was a painter, and artist, the topic that I cared really deeply about was, the more mundane aspect of human lives. This shows up in some of the, some of the work that I’ve done, where, like, I did this entire study of a rural town where I went around and took photos of people for not really doing anything special, but just living their everyday lives. I thought that was the most interesting thing. I’m somebody who has this perspective where, the world is oriented around this fractal shape, and you have two choices to understand the fractal shape. You either go outward and try to explore as much as you can to understand the broader shape of the fractal, or you go inward because, the outward resembles the inward, shapes. And understanding the mundane aspect of it was very much that. Simulation has a lot of this, right? You’re trying to understand even the most mundane aspect of people. When put together- teaches you something really deep about that individual and the society. So I think that’s what’s interesting about simulation, the way, the same way that AGI helped us better understand or really think critically about humanity and human intelligence, simulation is really an exercise of understanding more about human society and our collective lives. So that I find to be, yeah, particularly interesting.
Swyx [01:01:56]: Yeah. Now you’re reminding me that some of the best biographers, documentarians, and even photographers, they’re taking a photo of you.
Swyx [01:02:05]: But before I take a photo of you, I must spend-- I must, like, follow you for a week just to understand you?
Swyx [01:02:11]: Which some artists, some do. Part of your work, there’s a very famous book called Working. I don’t know if you’ve, been referred to it before.
Swyx [01:02:18]: It’s very famous, like, to the point of having a Wikipedia page
Swyx [01:02:23]: About this like, really depth understanding and interview of people as they, about their lives, which seems mundane, but is told in a very, compelling way. Yeah, 1970s as well.
Joon [01:02:34]: Okay. It was an amazing decade.
Vibhu [01:02:39]: Before closing question
UBI, Future Questions, and the Value of Simulation
Swyx [01:02:41]: Okay, here we go
Vibhu [01:02:41]: You said that you started Simile with your 10-year question, right? If we do that now, 10 years down, what can we simulate? What would you simulate if, like, if you’ve made significant progress, are there any questions outside of the ones that we brought up? Any- anything that you think is most impactful? Anything that you would go vision 10 years out?
Joon [01:03:03]: In many ways, as I mentioned, I am somebody who is very much impact-driven. So the what would inspire me is I would want to ask, 10 years later, what would be the most important societal question that we as a society have to ask? I would love to tackle that. Like, do we need UBI? That could be an interesting one.
Swyx [01:03:24]: Ooh, has anyone done that?
Joon [01:03:25]: Well, we were thinking about it.
Vibhu [01:03:27]: Can we get access? Can we just
Swyx [01:03:28]: So OpenAI, this is, like, just trivia now. Like, OpenAI, or I think Sam Altman funded a study on this
Swyx [01:03:35]: In Africa, and the answer was no.
Joon [01:03:37]: The answer was no. But, what, was it something about the implementation?
Swyx [01:03:41]: Yeah, I know. It was a skill issue.
Joon [01:03:43]: Or was it something about, But this is the thing. See, when Sam
Vibhu [01:03:46]: Funny news article
Joon [01:03:46]: Altman funded this particular,
Swyx [01:03:50]: He spent 14 million dollars? Oh my God.
Vibhu [01:03:52]: It’s a little more.
Joon [01:03:52]: Quite a bit. But this is the thing. This is the reason why you want to run a simulation. You spend 5 years, 40 million dollars on this one study and have one finding, but if you can run simulation many times instantly, then that’s the value.
Swyx [01:04:07]: I feel like that one could-- you could have done in a simulation. Like, if you can do the housing study, you can do the UBI one. Like, I, come on.
Vibhu [01:04:13]: I think sometimes people will spend the money because they wanna verify what you think, right? Like, sometimes you just wanna. Is it right? Like, you gotta test it.
Swyx [01:04:23]: Okay, closing question. What are the chances we are in a simulation right now?
Are We Already in a Simulation?
Joon [01:04:28]: So it’s a fun question, and I assert at some point I just answer, yeah, we’re definitely in a simulation. But what I do, feel, however, is, whether we are in a simulation or not, that, I don’t think that makes our experience any less real. And I think that’s fundamentally, like, what I believe in. Maybe we live in a simulation, maybe not, but for
Swyx [01:04:48]: It’s real to us. Yeah.
Joon [01:04:49]: Yeah. For me, I don’t really care.
Swyx [01:04:50]: Yeah. Unless you die and you wake up in, like, the level higher or below.
Joon [01:04:55]: That would be interesting.
Vibhu [01:04:55]: I feel like you wouldn’t care. Once you die, then you find out you’re in a higher level.
Joon [01:05:01]: I worry about it when I die.
Swyx [01:05:04]: I think the other thing that. Okay, so I like the mathematical answer to this, which is, like, the, sheer number of possibilities that you are in a simulation far outweigh the sheer number of possibilities that you’re not.
Swyx [01:05:16]: Except for the simplest answer, which is, it is computationally very expensive to have you be a simulation. okay, great. You’ve been very generous with your time. Congrats on all your success. I met you just after your Smallville paper and had no idea that you could build, like, such an enormous company. And then now you’re like, “Well, it’s a $100 billion market, but that’s just where we’re starting.” So this is, very exciting.
Vibhu [01:05:42]: I think $100 billion market was not the term. That was only part of it.
Swyx [01:05:45]: Yeah, exactly. It’s, if you’re thinking too small.
Joon [01:05:48]: Well, I do believe that, maybe my final note here might be, again, I love science fiction. You look at any advanced civilization in science fictions, there’s 2 twin pillar, technology. One’s AGI in some form, and the other is simulation. So I think the market’s pretty big here.
Simile as Research Lab and Product Company
Vibhu [01:06:08]: Tell us about the company. You guys just raised a lot. You’re half a research lab, half a company. you’re hiring. Where are you based?
Joon [01:06:15]: Yeah. So we’re based in Mission Rock, so not too far away from, where we are right now. So we’re in SF, but we are also bicoastal. So we have our, team. I would say our headquarter is in SF, and we have a lot of our technical talent in SF, and we do have a smaller office that just opened up in New York. We are, as a company, an interesting one in that today, there are AI neo labs and then there are AI product companies. Simile truly is both. So this is a company that was founded by 4 founders, myself, Michael Bernstein, Percy Liang, Lainie Yallen. Michael, Percy, and I are all researchers. So of course, Michael was one of the authors of the ImageNet, kickstarted the AI revolution back in 2013, has been instrumental in human-centered AI. Percy coined the term foundation model, and is a, one of the greats of the AI researchers today. And Lanie is my business counterpart, where she led some of the fastest-growing AI native companies from their seed to A and B. But we have this DNA at the company where the vision of the technology that we’re creating is continuously developing, that we are getting people who were my lab mates. We are about 60 people right now.
Joon [01:07:28]: 15%, almost 20% of the company population are just my lab mates from Microsoft Research lab.
Joon [01:07:36]: And we It’s quite fun because many of them then had gone on to OpenAI, Google Gemini, and these places. And so it’s been a few years since we really got together and had a chance to work together. But now they’re coming back and really building out this vision that I find to be quite exciting, and that excitement is shared. So there’s that motion at Simile where we are a group of researchers trying to do something that no one is working on that we find to be the most impactful potentially. But at the same time, this is, again, technology that can make impact today. So we have an amazing group of engineers, product people, and designers, who are sitting here with us trying to imagine what does it look like to help people understand what simulation can do and make real-world decisions with this. Having both and then deploying it to some of the largest customers in the world today, it feels quite unique.
Swyx [01:08:30]: Yeah, it’s very compelling. One part of it was this is the call to action. Like, who are you hiring? You’ve done part of it, which is you have-- you’ve got a very talented group. Who are you hiring? Like, what roles?
Hiring and Closing
Joon [01:08:41]: So honestly, at this point, we’re hiring across
Swyx [01:08:43]: Everything
Joon [01:08:43]: All, section. we are always excited to bring on, amazing research talent.
Joon [01:08:49]: So if you’re interested in working with, our lab mates, we are always welcoming of amazing, researchers. But also we, hire, amazing engineers, that some of whom I, like, I respect the most. Many of them come from places where we have personal connections with, so many of the members are from Figma, Notion, Rive, and so forth, but also more broadly from the companies that we as a team have really admired. So engineers both in the product side, infra side, we’re all looking for those hires.
Swyx [01:09:24]: Well, lots of people. I think you made a really good case. So thanks, and, we’ll see you in the simulation.
Eiso Kant [01:52:31]: We are hiring on every possible role in applied research and engineering in the company, from training all the way to evals to post-training architecture. Like, we are still in a world where, individuals can have massive impact. And I think our pitch to join us —I think we are one of the places where it’s the highest ratio to individual to impact, Right? Less than 70 people built this model. Less than 115 between engineering and researchers, like, together did this effort, and that’s a very broad definition ‘cause I put myself in the 115 list.
As the founders say, this is “not an acquisition and not an acquihire”:
We’ve been calling the Windsurf-Google and Character-Google and Scale-Meta and Instacart-OpenAI deals execuhires because usually the executives go leaving the employees with a rich payout but holding the company remaining, but this is a first time it is happening the other way around. The action amounts to founders pivoting the company extremely hard to SOMETHING, and finding an EXTREMELY comfortable golden parachute for investors and employees to continue on with the original mission or stay aboard for the new pivot:
For the last 3 1/2 years we’ve been directionally correct in a race where capital requirements went vertical.
At the end of last year, we had a 6 week window in which to raise $2 billion dollars to pay for a 40,000 GB300 cluster coming online in January. We didn’t close it in time, and we lost the cluster.
and:
We also know that at 10,000-20,000 GB300s we would produce a great model that could rival the current frontier. But the scale of next year’s frontier models requires far more than an order of magnitude larger cluster. And for this the constraint today is not only capital, it is physical data center space and contracted compute.
The compute needed to be at the frontier of the current model recipe is going vertical, and as the world accelerates along the axis of Recursive Self Improvement this will only become more evident.
To this end, the PIC infraco, spun out in Jan 2026, is also interesting in its ambitions…
We’re confused too, and the founders say they are “not ready to share the updated vision”, but everyone here is coming out with a lot of money so we’re just interested to see what’s next for everyone on the 3 different directions emerging from OG Poolside.
The only hints left to us:
We wholeheartedly believe that everything economically valuable, scientifically interesting and a lot of what will be personally meaningful is going to be underpinned by Al. The world has not yet reached 0.1% of this transition….
… We believe human level capabilities of intelligence will be fully commoditized by open source models, while super intelligence will likely not be.
The world has two types of economically valuable problems, those that are intelligence bound, and those that are experiment bound. The first are problems which we can solve by scaling up intelligence e.g. building software, doing accounting, solving a math theorem. The second are ones that require real world experimentation to progress, and no amount of increased intelligence without experimental results will make progress. We could put 100,000 of the world’s brightest minds together to solve cancer but without a real world experimental feedback loop, they likely never will.
Today’s model revenue is from coding and soon from all of knowledge work. In the future, companies who can go beyond human level capabilities will tap into revenue coming from scientific discoveries where there is a true data moat derived from real world experimentation. In our humble opinion, Al’s ultimate value will not derive from the first kind, that will become a low margin commodity, but it will from the second. Al will become the world’s most valuable scientific discovery engine.
OpenAI and Anthropic Expand the Agent Product Surface
OpenAI pushed several desktop and builder features in one wave: @ChatGPT launched an Apple Messages plugin for ChatGPT Work/Codex on Mac, enabling message search, catch-up, drafting, and sending from the desktop app. @OpenAIDevs also added collaborative editing for ChatGPT Sites, with teammates sharing a project while Codex manages git/CI; shared read-only conversation links and PR-context sharing further push ChatGPT/Codex toward being a coordination surface, not just a chat UI. On the API side, transparent backgrounds in GPT-Image-2 are now in preview for reusable design assets.
OpenAI’s desktop memory/workflow features continue rolling out geographically: @OpenAIDevs said Computer History and cross-app memory are now available in the EEA, UK, and Switzerland for Pro/Business/Enterprise Mac users, with Record & Replay also live there. Together, these features point to a product strategy of capturing user workflows on-device and turning repeated actions into reusable skills.
Anthropic made its agent platform more composable and production-ready: @ClaudeDevs announced general availability for computer use, browser tool, Skills API, and Files API on the Claude Platform. The Skills API adds versioned reusable procedures; the Files API now supports expiration control, 5x higher rate limits to 500 RPM, and 1 TB/org. Anthropic also published an AG-UI adapter for Claude Managed Agents, mapping chat threads to managed sessions and streaming text, tool calls, and thinking into custom UIs.
Model Economics, Usage Limits, and the Enterprise Shift Toward Open Models
AT&T became the clearest public case study yet for hybrid routing: the most consequential enterprise datapoint in the set came via @Hesamation, summarizing AT&T’s internal AI deployment: 40% of employee AI usage already routes to open models, with a target of 60–70%; coding costs are down 56% for only a 2% quality drop, at 45B tokens/day. That supports the increasingly common view that frontier closed models remain reserved for the hardest tasks, while “good-enough” open models eat the broad middle of enterprise demand. @amir explicitly framed this as a warning sign for OpenAI/Anthropic’s enterprise moat, while @ollama welcomed AT&T to open models.
Pricing pressure is intensifying across closed-model distribution: @eglyman announced GPT-5.6 Sol at 50% off through Router, and both @github and @code amplified the temporary discount for GitHub Copilot / VS Code users. At the same time, user sentiment suggests supply constraints are surfacing as usage caps rather than degraded quality: @bridgemindai complained that a $200/mo OpenAI Pro plan could be exhausted in a single heavy Codex day, and @theo noted it was possible to continue consuming substantial tokens after hitting the stated cap. The broader signal: labs are still searching for the right product boundary between high-end model access and economically sustainable agentic usage.
Open-weight adoption and distribution continue to broaden: @ollama said Kimi K3 is now rolled out to over half its subscription base with US/EU hosting and zero data retention. On the open ecosystem side, @Google and @osanseviero highlighted Gemma surpassing 1B downloads, while @_philschmid launched an Awesome Gemma repo aggregating variants, deployment guides, and fine-tuning recipes.
Multimodal and Agent Benchmarks: Muse Spark, GLM-5.3, Gemini 3.7 Flash
Meta’s Muse Spark 1.2 had a strong benchmark day across multimodal/agentic evals: @AIatMeta presented demos spanning visual coding, robotics planning, and audio-visual understanding, and previewed WildArtifactBench, an internal eval using win rates and Elo from human/agentic judges for practical multimodal tasks. Third-party measurements were favorable: @arena reported +2.1% net improvement in Agent Arena, up from 0.9% in v1.1, with particularly strong Bash Recovery (+11.4%); @DesignArena placed Muse Spark 1.2 #1 for Video-to-Website, #2 for Image-to-HTML, and #3 for Image-to-Frontend, while noting it sits on the price-preference Pareto frontier.
Zhipu’s GLM-5.3 keeps showing up in agentic/code evals: @AutoClawAIer announced GLM-5.3 integration into AutoClaw, Z.ai’s work agent. More importantly, @arena said GLM-5.3 Max shifts the Code Arena: WebDev Pareto frontier, projecting to #2 among open models and #8 overall at 1597 pts and $3.65/M. Separately, @ZixuanLi_ resurfaced SAO (Single-Rollout Asynchronous Optimization) as a key GLM-5.2/5.3 RL advance for stable asynchronous agentic RL.
Gemini 3.7 Flash keeps accumulating “cheap and strong” evidence: @arcprize reported ARC-AGI-2: 84.6% at $0.25/task and ARC-AGI-1: 95.5% at $0.12/task, making Gemini 3.7 Flash stand out on cost-adjusted reasoning performance. @JonathanJarvis separately called it excellent for agentic vision tasks.
Infra, Hardware, and Systems Work: Rubin, Cerebras, Linux Agents, Caching
OpenAI’s next pretraining stack is moving onto Rubin: @udayruddarraju posted that OpenAI’s first NVIDIA Vera Rubin racks are now installed and running the training stack, explicitly tied to next-generation frontier pre-training. @gdb called it a major milestone in the OpenAI-NVIDIA partnership.
Cerebras’ CS-4 drew attention for inference scaling without a node shrink: @kimmonismus summarized the launch as essentially doubling performance on the same 5nm wafer, 4T transistors, and 900k AI cores, via redesigned power delivery and cooling. Reported specs include 250 PFLOPs per WSE-3 Turbo, 43.2 PB/s memory bandwidth, and a 3-wafer CS-4 rack at 750 PFLOPs. The notable claim for practitioners: 4,400+ tok/s per user on GPT-OSS-120B, up to 30x faster than GPU-based systems.
Agent runtime ergonomics are becoming a systems bottleneck: @theo argued that Linux materially outperforms macOS for agent workloads, especially on filesystem-heavy operations. @Qdrant_engine shared a practical semantic-caching writeup showing 57.1% hit rate, 55.7% fewer tokens, and ~15 ms hit latency. @MParakhin pushed gisting as an underused production technique, citing ~40% lower end-to-end latency and ~15% higher throughput with better results, and linked a Shopify engineering writeup.
Agents, Memory, and Harness-Centric Learning
Chroma launched a research preview of self-improving memory: @jeffreyhuber announced Foundation, Chroma’s approach to agent memory, built from prior agent sessions. This landed amid a broader shift from “single-shot agent” thinking toward persistent harnesses with accumulated state, skills, and memories.
The most interesting agent research in the set was about harness evolution, not model weights: @omarsar0 highlighted a paper on harness continual learning, where prompts, memories, skills, and routing rules evolve independently of the model. The key failure mode is harness-level forgetting: improving one component can silently break previously reliable behavior. The proposed solution, guarded harness evolution, separates proposing updates from committing them, with reported >10% gains across textual, multimodal, and open-world tasks.
Related negative results matter too: @dair_ai flagged a study showing that memory-based self-improving agents look worse once you control for task order effects and evaluation variance. @omarsar0 also summarized a paper arguing post-training agents tend to lock into an initial strategy early and spend the remaining budget on local refinement rather than revisiting the strategic choice itself.
Top Tweets (by engagement)
ChatGPT desktop + Messages: @ChatGPT’s Apple Messages plugin launch was the single biggest product tweet in the set and reflects the shift toward desktop-native, action-taking assistants.
AT&T’s open-model routing economics: @Hesamation’s summary is arguably the most strategically important enterprise datapoint: 40% open now, 60–70% later, 56% coding cost reduction.
OpenAI’s Rubin racks: @udayruddarraju provided a rare concrete infrastructure signal about frontier pretraining scale-up.
Claude Platform GA for computer use / Skills / Files: @ClaudeDevs marked a significant maturity step for Anthropic’s agent platform.
Gemini 3.7 Flash on ARC-AGI: @arcprize reinforced Google’s positioning around strong low-cost reasoning.
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Qwen3.8-27B Quantization and Coding Benchmarks
Introducing Qwen3.8-27B Dynamic v3 Unsloth GGUFs (Activity: 2059): The image is a technical announcement graphic for Unsloth Dynamic v3.0 GGUF quantizations of Qwen3.8-27B, claiming >10% better top-1 accuracy at the same model size versus other quant providers. It highlights post-training quantization only—no QAT/QAD and no training on the imatrix calibration dataset—plus memory targets from 1-bit quants runnable on ~8GB RAM up to BF16, with evaluation framed around Divergence-300 @32, KLD, and top-1% accuracy comparisons. The linked release points to the Unsloth blog and Hugging Face GGUF repo: https://unsloth.ai/docs/basics/dynamic-3.0-ggufs and https://huggingface.co/unsloth/Qwen3.8-27B-GGUF. Commenters were broadly positive but asked for more comparative data, especially adding the prior Qwen 3.8 27B UD 2.0 quants to the chart so users can judge whether upgrading is worthwhile. One user also noted practical hardware interest: whether IQ4XS can now run on 16GB VRAM without MTP.
Users requested comparative quantization metrics against the prior Qwen 3.8 27B UD 2.0 GGUFs, specifically asking for KLD and/or top-1 error lines on the graph so existing local files can be directly compared to the new Dynamic v3 quants.
A technical point was raised that the new IQ4XS quant may fit within 16 GB VRAM without MTP, which would be significant for single-GPU local inference if quality degradation remains low. Another user noted the apparent ~15 GB size for Q4_K_M, asking whether it preserves quality well enough to be practically useful.
One commenter asked for more granular evaluation now that oobabooga is involved, specifically per-category KLD and KV-cache quantization KLD metrics similar to those shown by localbench.substack.com, to better understand where quantization loss appears across tasks and cache settings.
Qwen3.8-27B took a serious hit to knowledge vs 3.6 (Activity: 758): Users report Qwen3.8-27B regresses vs Qwen3.6-27B on offline/weight-only factual recall, aligning with lower scores on Artificial Analysis’s Omniscience knowledge benchmark. The observed tradeoff is that Qwen3.8 appears stronger for tool calling, coding, and agentic workflows, but weaker when web/search tools are disabled for obscure trivia, historical/location identification, or airgapped knowledge retrieval. Commenters generally frame this as an intentional or acceptable specialization tradeoff: Qwen 3.x may be shifting toward coding/agentic use where external retrieval is expected, while models like Gemma may be preferable for broad “mini Google” factual recall. One commenter explicitly preferred not allocating parameters to niche trivia if it improves coding performance.
Several commenters converged on the view that Qwen 3.8-27B appears optimized away from memorized factual recall and toward coding/agentic workflows. One user reported that with web search/fetch disabled, Qwen 3.8 regressed on niche knowledge tasks such as identifying stamps, historical locations, and old photos, while tool calling and coding were “impressive” when retrieval tools were available.
The discussion framed the regression as a deliberate parameter-capacity tradeoff for a 27B model: reduce obscure memorized knowledge while preserving reasoning, coding, and tool-use competence. Commenters suggested using other models such as Gemma for trivia or broad factual recall, while positioning Qwen 3.x as better suited to agentic tasks that retrieve information externally before acting on it.
One technically interesting speculation was around future modular model knowledge/skill extensions, described as “neural plugins” similar to LoRAs. The proposed architecture would keep the base model lean while adding native domain or language competence—e.g. Japanese support or financial-services knowledge—through optional plugins rather than baking all knowledge into the base model.
I ran Qwen3.8-27B against Opus, Sonnet, GPT and others. Results inside. (Activity: 422): The image is a benchmark dashboard for the author’s home-built coding eval comparing Qwen3.8-27B, DS4 0731, GPT-5.6-sol, Opus 5, Sonnet 5, and Haiku 4.5 across algorithm tests, repo bugfix/feature tasks, wall-clock completion time, and blind-judged code quality (image). The main technical takeaway is that GPT-5.6-sol leads overall with perfect repo-task performance and near-perfect algorithms, while local models are surprisingly competitive: Qwen3.8-27B xhigh scores strongly on hard algorithms and “surgical fixes” but is much slower, and DS4 0731 achieves 8/8 on both repo tiers despite being a 2-bit local quantization. The author notes a practical tradeoff: higher “thinking” improves some hard reasoning/code-quality cases but can overthink, increase latency, and even reduce repo-task accuracy compared with medium thinking. Commenters questioned benchmark saturation and task difficulty, arguing that if nearly all models score near the top then the eval may not distinguish frontier/local capability well. Others asked for more detail on the definitions of “algorithm” and “repo work” tasks, expected outputs, and hidden test design to make the results more reproducible and interpretable.
Several commenters argued the benchmark appears saturated, with “all models at the top”, making it hard to distinguish Qwen3.8-27B from Opus, Sonnet, GPT, and others. One analogy framed it as testing stronger models on tasks too easy to separate capability, implying the suite needs harder or more discriminative evaluations.
A commenter requested more precise methodology for the “algorithm” and “repo work” tasks, specifically asking for expanded task descriptions and expected results. This points to reproducibility concerns: without clear prompts, grading criteria, and target outputs, cross-model comparisons are difficult to interpret.
One technically relevant question asked what “DNF” means for Qwen3.8 medium, in the context of a comparison between Qwen3.8 xhigh and medium settings. This suggests the benchmark table included incomplete or failed runs, but the failure semantics were not defined clearly enough for readers to assess the result.
We’re currently developing a new series about skills, with the aim of giving you a regular supply of new skills to use in your projects. We’re kicking things off with an interview — and a super-useful skill — featuring Matt Pocock, whose “AI Skills for Real Engineers” project has over 220,000 stars on GitHub. He also talks about these skills to 347,000 subscribers on his YouTube channel.
Pocock recently released a new skill called /wayfinder. Its purpose is to help you and your agent figure out a project where the end state isn’t entirely clear. Or as Pocock put it in our interview, /wayfinder helps you navigate “the fog of war,” where you have a project but “you can’t quite decide everything right at the start.”
The following interview has been slightly condensed for readability, so you can read it, absorb Matt’s insights, and then test out /wayfinder for yourself!
Latent Space: What were the goals of wayfinder?
Pocock: What I noticed is I was doing a lot of work with AFK agents [Away From Keyboard] and trying to schedule in a ton of work so that my agents could run virtually overnight. I would just plan a bunch of stuff, and then I would create a spec and then turn that spec into tickets. And I had a really well-developed set of skills for how to turn work into scheduled stuff that agents could just crack on.
But [...] I was finding the planning stage really onerous, because I would have to be constantly thinking about my session management. Like, how many tokens am I into my context window? How deep am I going here?
I didn’t want to feel constrained in the planning stage anymore. I wanted an orchestrator layer that would basically say, okay, whatever you want to plan, I’m going to handle the planning sessions for you. I’m going to split this out into multiple different threads, do prototyping, do research and pull it all back together, so that you don’t feel constrained in the planning anymore.
And then your specs can be even more detailed, and you can just whack off an AFK agent to go and do tons more work.
Latent Space: What was the design process of coming up with this skill?
Pocock: I had this kernel of an idea of, what if I didn’t have to manage the handoffs? What would that look like? And then, what would it look like to have some kind of centralized document to have all of those pieces together?
Whenever you’re thinking about context management — because that’s really what a skill is, you’re managing the context of the agent you’re working in — you need to think about the information flow. So what I wanted to think about is, what if a grilling session could manage other grilling sessions? What would that look like?
Well, the first step to that is, what does the grilling session that’s being managed need? What does the child need in that situation? So the child probably needs to understand a vague overview of what else is happening, and they need their specific task.
So there, you’ve got two documents. You’ve got a map — which is all of the rest of the stuff, all the decisions that have already been made. And then you’ve got the specific ticket that goes into the actual session. And what you notice there is that those words are very precise.
You’ve got the map, and you’ve got the ticket, and you’ve got the session. And once you’ve got the kernel of an idea, you then need to come up with the words for that idea. Because once you’ve figured out the words, then those entities can be really clearly mapped out by the agent.
Because if you just call everything a ticket, or if you just refer to it in different ways in different places, then it’s going to be really confused and you’re going to get strange behavior. Whereas if you use these very specific, what I call leading words, to lead the agent to understand exactly what each part is, and you’ve understood what the information flow is, then you’ve got your skill.
Latent Space: What kind of use cases do you think wayfinder would be useful for?
Pocock: Well, I’ve been using it for all sorts of stuff. I’ve been using it to actually plan courses as well. In wayfinder, there are different types of tickets.
So you’ve got grilling tickets, which are just a grilling session. Then you’ve got prototype tickets for creating prototypes, research tickets for creating [and doing] research, and then task tickets — which are really broad…basically, just anything the human needs to do that the agent can’t do. And so once you think about that, you realize, OK, I can apply that to anything.
One really key idea in wayfinder is the ‘fog of war’. So this is the concept of, you can’t quite decide everything right at the start.
I decided to test /wayfinder on a project to rearchitect my personal website. Here’s the initial project set-up, in this case using Claude Code.
You can make certain decisions, and those certain decisions sort of lead you there and push further out into the fog of war — kind of like Warcraft III style, exploring the map. And once I had the idea of ‘fog of war’ and ‘map’, I realized those two terms actually work really nicely together, and it really leads the agent into the right idea. So I’ve been using it for engineering, for non-engineering stuff, for course planning, all sorts.
Latent Space: This concept of the fog of war — it’s weird to consider what you don’t know that you don’t know. Maybe LLMs are good at capturing that.
Pocock: I feel like with the grilling stuff that I’m still working on, that captures an idea that you don’t know stuff, but maybe the agent can contribute something and illuminate a part of the room that you don’t quite understand yet.
And wayfinder is just sort of an extra layer on top of that.
Working through my website rearchitecture project using /wayfinder. There’s 20+ years of content to re-organize!
Latent Space: Yeah, and there’s all these artifacts. How much time do you spend teaching the model all this terminology?
Pocock: For the last few months, I’ve been pretty obsessed with terminology — and finding the right terms for certain things. I’ve put together, I haven’t actually put it out yet, but it’s an AI coding dictionary — of basically all the terms in AI coding. It’s in this beautiful graph that you can explore and understand exactly what an agent is, exactly what a harness is, exactly what a model is, blah blah blah.
I’ve redone all my courses to use that dictionary and make it really solid. And then all of my skills use a consistent dictionary as well. So they’re all working off the [same] assumptions, the same leading words.
I realized that I needed a ubiquitous language between me and the agent. Between me and the agent, there is a communication barrier. And that’s what I’m trying to do with my skills all the time, is try to find the right words.
And agents are really good at showing you the opportunities for different wording — really good at domain modeling, actually.
Latent Space: When do we directly use the grill-me skill, versus wayfinder?
Pocock: Use ‘grill me’ in cases where you feel like you can plan the whole thing in a single session, and you need to align before you go. So most small features will fit into this. Most stuff where you can see the path ahead of you, but you just want to make sure the agent is on board, ‘grill me’ will work with that.
For stuff where you don’t know the path ahead, for stuff where you can feel the fog of war in front of you, use wayfinder. You’re gonna find your way with wayfinder. So that’s how it works.
Prof Jie Tang is back on X to tell us that our shorthand for model sizes is no longer enough: “Parameter count is only meaningful alongside three others — how much data you have, where you intend to spend your compute, and who will run the model, under what conditions.”
We have covered Chinchilla (and post-Chinchilla) scaling laws in past LS years, but, so we will skip the history lesson, but it is good to level-set on why Chinchilla’s assumptions were wrong in the Inference Inflection world (no fixed number, between 200-900 toks/param, citing Roberts et al on task dependence).
In short: Memorization prefers more parameters. Reasoning prefers more post-training data and effective depth. GLM-5.3’s big jumps come solely from RL on long horizon environments:
The environments now cover a much broader range of production workflows, with tasks designed around how engineering and research work is actually carried out in practice. Some represent several days of work for an experienced engineer. In an ML infrastructure task, for example, the model may be given the same working environment as an engineer, with access to compute clusters, storage systems, internal documentation, codebases, and experiment results. It must diagnose bottlenecks across the training stack, implement optimizations, run experiments, and deliver a measurable end-to-end speedup while preserving correctness. Training on environments at this level pushes the model toward taking ownership of substantial work end to end, rather than relying on users to decompose the problem and supervise each step.
For those following the recursive self improvement story, their entire environment and judging and verifier process is synthetic all the way down:
As agent capability improves, much of the difficulty in scaling post-training moves from the model to the environment. A useful task environment has to be executable, verifiable, and close to real professional work — and we need many of them, not a handful of hand-built ones. To scale this process, we built pipelines that synthesize environments end to end, and for a subset of tasks, the RL reward signal as well. Research agents collect task patterns from real work and turn them into runnable long-horizon environments with multi-step dependencies and hidden state; a judge agent then attempts each task to verify that it is actually solvable. Verifiers are synthesized without access to the reference solution, while solver trajectories are used to discover and close reward shortcuts. A verifier that passes oracle, no-op, and unsolved-state checks produces a binary reward reliable enough to train on directly.
To put an end to parameter count obsesssion, Prof Jie identifies 5 knobs of scaling, including MoE sparsity with the new XA-YB notation. He notes that advanced skills (e.g., finding software vulnerabilities) are not retrieval/memorization problems. They require carrying long causal chains (20+ inference steps) without losing the thread. This ability does not live in total parameter count once a certain knowledge-holding threshold is reached.
Open-Weight Models, Compression, and Benchmark Movement
Ornith-1.5 lands as a serious new open family: @ornith_ released Ornith-1.5 in 9B dense, 35B MoE, and 397B MoE variants under MIT, with quantized formats including FP8, GGUF, MLX, and NVFP4. The headline claim is end-to-end self-improvement: the model proposes tasks, generates scaffolds, and produces RL rollouts to create new training experiences. Reported evals are strong across agentic/coding workloads, including Terminal-Bench 2.1: 86.1, SWE-Bench Verified: 86, DeepSWE: 56, HLE: 44.6, and Tool Decathlon: 71.2. The release was quickly wired into serving stacks by vLLM and Ollama.
Compression continues to get more aggressive without fully collapsing utility: @UnslothAI and @danielhanchen shipped new Qwen3.8-27B GGUFs using Dynamic V3, claiming roughly 10% higher accuracy at the same size and releasing 1-bit quants that still retain about 77% of BF16 accuracy while running on 8GB RAM. Their new Divergence-300 metric extends top-1% greedy accuracy across longer generations using unseen examples from Terminal Bench, DeepSWE, and related tasks.
Agent and legal eval boards continue to reshuffle: @arena published a Pareto view of Agent Arena, where Claude Opus 5 (High) leads quality, but lower-cost models like Kimi K3, GLM 5.2, Grok 4.5, and GPT-5.6 Luna define much of the value frontier. Separately, @ValsAI reported Grok 4.6 at #3/49 on Legal Research Bench with 48.1%, 500k context, tool/image/file support, and relatively low pricing. For open models, @ValsAI also highlighted GLM 5.3 as #2 on Terminal Bench, #3 on Legal Bench, and #6 on Skills Bench among open weights.
Agent Harnesses Become the New Competitive Layer
DeepSeek Harness’s minimalism is deliberate, not incomplete: A detailed writeup amplified by @ZhihuFrontier and summarized by @TheTuringPost frames DeepSeek Harness (DSH) as an intentionally thin shell over a plugin architecture called Cordis. The key design choice is that everything is a plugin, including the agent loop itself. Early beta users reportedly shipped 100+ plugins and filed 400+ issues in under a week; examples range from a gomoku model testbed to a database agent that closes the SQL feedback loop by connecting the model to live query execution. The strongest takeaway is architectural: DSH is less “productized assistant” than open agent runtime, optimized for user-extensible tooling, swappable control loops, and business-rule injection.
TrueFoundry open-sources TrueForge and makes the harness-cost argument explicit: @truefoundry, @omarsar0, and @kimmonismus all covered the launch of TrueForge, an MIT-licensed, self-hostable, vendor-neutral harness for production agents. The stack includes tool orchestration, context management, subagents, code sandboxes, human approvals, and traces, with both local and hosted deployment modes. The technical claim that resonated: on a 14-task enterprise benchmark, TrueForge matched Claude Managed Agents on Opus 4.8 while using about 30% fewer tokens, and routing to GLM-5.2 cut cost by around 75% while preserving accuracy. The broader industry theme—also echoed by @bradenjhancock and @dbreunig via @rseroter—is that the session/environment/memory/tools layer is becoming a major source of both differentiation and savings.
Managed harnesses are also getting sharper observability and controls: @ClaudeDevs added memory support for self-hosted sandboxes, domain allow/block controls for web tools, and a redesigned multi-agent session viewer with minimap, grouped transcript, and cost-per-thread/session. OpenAI, meanwhile, continues pushing the opposite angle: give teams the harness primitives to embed into their own products. @OpenAIDevs highlighted the open-source Codex harness as the runtime beneath internal tools, ops dashboards, and custom apps, while @cursor_ai shipped cloud-agent UX improvements around persistent goals and long-lived sessions.
Post-Training, Mid-Training, and RL Systems Work
More evidence that scaling is shifting from parameters toward training recipe quality: @kimmonismus surfaced a notable claim from the zAI/GLM founder: progress is still scaling, but too much discourse has fixated on parameter count rather than data quality, inference compute, and post-training. The cited example is GLM-5.3, reportedly based on the same core base model/architecture as GLM-5.2, but improved substantially via about one month of extra RL.
Microsoft’s Agent Lightning points at RL-through-the-harness as a practical recipe: @omarsar0 highlighted Agent Lightning v1.0, which connects arbitrary harnesses to RL through an endpoint proxy, handling issues like retokenization, sample merging, advantage calculation, normalization, and scheduler/backend coordination. With ~6K training examples and modest compute, it reportedly moves Qwen3.5-9B on SWE-Bench Verified from 41.8% to 56.4%.
Mid-training is being treated more explicitly as an optimization surface: @cwolferesearch laid out the current practitioner view of CPT/midtraining: optimize data mixture, duration, stage ordering, sequence length, and even post-trainability rather than just “continue pretraining on better data.” The thread is useful precisely because it frames these as interacting knobs rather than independent tricks.
RL infrastructure keeps improving underneath the research: @SergioPaniego resurfaced work showing on-policy distillation in TRL becoming 40x faster via generation buffers, batched teacher calls, and binary logprob encoding; @mikasenghaas announced adaptive concurrency in prl, dynamically adjusting in-flight rollouts over the course of an RL run.
Benchmarks, Retrieval, and Infra Details That Matter in Production
Qdrant’s filterable HNSW vs ACORN is a substantive retrieval systems update: @qdrant_engine argued that filtered ANN should be addressed in the index, not only at query time. Their filterable HNSW adds edges between points sharing indexed payload values, keeping filtered subgraphs connected. In their benchmark on a 1% filter over 1M vectors, they report 99.8% recall at 1.0ms versus 67.7% at 4.7ms for ACORN. They also note ACORN still helps for broad values and AND filters, especially atop a graph already optimized for filters.
Sentence Transformers v6.0 reflects the practical move from single-vector to multi-vector retrieval: @tomaarsen summarized the distinction clearly: dense retrieval compresses each text into one vector, while multi-vector retrieval keeps token-level vectors and scores query tokens against document tokens before aggregating best matches. That matters because late-interaction retrieval is increasingly the default tradeoff for quality-sensitive search systems.
Production agent latency often has little to do with the model itself: @dair_ai summarized a paper instrumenting ten agentic apps and finding that non-LLM components dominate latency in half of them, with sandbox memory peaking at 28GB/session, up to 32x latency variation across subsystems, and long idle state retention between steps. The optimizations are unsurprising but important: task-aware serving cuts latency 29–40%, state offloading reduces memory 4.6x, and tool-result caching removes 35.2% of redundant search calls.
Linear and turbopuffer show vector infra creeping into non-search hot paths: @turbopuffer said Linear moved its delta sync read path from Postgres to turbopuffer, using attribute indexes for permission filters and reducing the largest syncs by about 8 seconds.
Google, OpenAI, Anthropic, and the Productization Race
Gemini 3.7 Flash had a strong day on both evals and product integration: @_philschmid and @NewsFromGoogle highlighted Gemini 3.7 Flash taking #1 on Artificial Analysis’s AA-AnalystAgent, with 60.0% pass^5, 70.5% pass@1, 77.5% pass@5, 1.32s/task, and $0.54 average cost across 80 spreadsheet/document-heavy quantitative tasks. Google also pushed it deeper into product surfaces: Gemini chat and Spark, Search-based interactive simulations built on the fly in AI Mode (example), and AI Studio GitHub sync for build workflows.
OpenAI is leaning into low-cost deployment and privacy positioning: @Replit launched Free Mode powered by GPT-5.6 Luna, which @kimmonismus framed as a meaningful efficiency win: a model that would recently have been SOTA is now cheap enough to be given away broadly. On the enterprise side, @OpenAI introduced Private Safety Processing, aiming to preserve Zero Data Retention for frontier models while still detecting cross-interaction safety risks without human access to the underlying content.
Anthropic continues to tighten the developer ergonomics loop: beyond the managed-agent updates above, @ClaudeDevs added a Concise output style to Claude Code, another sign that product teams are now tuning not just capability but response-shape as a first-class UX variable.
Top tweets (by engagement)
Ornith-1.5 release: @ornith_ unveiled an MIT-licensed open model family from 9B to 397B, with strong coding/agentic benchmark claims and broad quantization support.
OpenAI privacy/safety infrastructure: @OpenAI announced Private Safety Processing while reaffirming Zero Data Retention for frontier models.
Gemini student push and product bundling: @GeminiApp offered a year of Gemini plans to students globally while rolling out new study-oriented features.
Claude Code UX update: @ClaudeDevs shipped Concise mode, a small but widely noticed improvement for day-to-day coding-agent interaction.
OpenRouter acquisition: @patrickc confirmed OpenRouter is joining Stripe, a move many interpreted as validation that token routing/marketplaces are becoming core infrastructure rather than edge tooling.
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Qwen/DeepSeek Open-Weight Inference Speedups
Introducing Qwen3.8-27B Dynamic v3 Unsloth GGUFs (Activity: 1428): The image is a technical announcement graphic for “Dynamic v3.0 Qwen3.8”, showing Unsloth’s new Qwen3.8-27B Dynamic v3 GGUF post-training quantizations and claiming >10% higher top-1% accuracy at the same GGUF size versus other providers. It includes a memory table suggesting the model can run from 1-bit quants on ~8GB RAM up to BF16, plus a chart comparing accuracy across quant sizes; the post links the GGUF release on Hugging Face, the Dynamic 3.0 docs/benchmarks, and the image itself. Unsloth emphasizes these are post-training quantization releases only—“we do NOT use QAT or QAD”—and says the imatrix calibration file is public for independent evaluation and fine-tuning experiments. Comments were mostly positive, but one technical request asked Unsloth to add the previous UD 2.0 quants to the graph so users can compare against what they already have locally. Another commenter asked for deeper diagnostics, specifically per-category and KV-cache quantization KLD numbers, referencing localbench-style reporting.
Several commenters requested more detailed quantization evaluation for the new Qwen3.8-27B Dynamic v3 Unsloth GGUFs, especially a direct graph line comparing against the prior Qwen 3.8 27B UD 2.0 quants. Suggested metrics included KLD and/or top-1 agreement, which would help users judge whether the new dynamic quantization is materially better than the versions many already have stored locally.
A commenter asked for per-category KLD and KV-cache quantization KLD reporting, referencing the style of breakdowns from localbench.substack.com. This would make the quant quality discussion more actionable by showing which benchmark/task categories or cache-quant settings degrade most under different GGUF quant formats.
There was interest in the practical memory footprint of the quants: one user noted ~15 GB for Q4_K_M, while another inferred that IQ4_XS may now fit on 16 GB VRAM “without mtp.” The technical concern is whether these smaller formats maintain model quality closely enough to justify running a 27B-class model fully on common consumer GPUs.
We’re officially in dire straits. There’s almost no way, if you’re reading this site, that you aren’t aware that memory prices have become entirely divorced from reality.Some are calling it the RAMpocalypse; I prefer “RAMageddon.”
That’s right: 128GB DDR5 kits are fully ten times more expensive than the lowest price we’ve ever seen.
In fact, the situation is so severe that hyperscale buyers have reportedly already locked in almost all of the global DRAM production capacity for 2027, handing over advance deposits to guarantee their supply of precious DRAM, which is now among the highest-value commodities in the world by weight; mainstream DRAM chips are worth over half as much per kilogram as solid gold.
Put another way, the famous Moore’s Law driving all hardware unit prices down has been reversed for memory:
Concrete controls matter more than broad messaging: OpenAI shared more implementation detail than usual, including stronger workload/network isolation, continuous security testing, and multistage monitoring. Secondary commentary highlighted interesting operational details: monitoring may add roughly 20% overhead, sampled-token monitoring can page safety/security/research teams within ~30 minutes, and tool-using inference for higher-risk systems may ship with active monitors attached, per @eliebakouch. Whatever one thinks of the policy framing, this is notable as a public admission that training/eval infra and inference-time monitors are now bottlenecks on frontier progress, not just raw compute.
Open Models: Qwen3.8-27B Momentum, GLM-5.3’s Post-Training Gains, and the Small-Model Debate
Safety implications of capable local models are getting harder to dismiss: A high-engagement post from @kimmonismus noted a “refusal-removed” MLX build of Qwen3.8-27B running locally on Apple Silicon in 2/4/6/8-bit variants, claiming preserved vision, reasoning, tool use, and 262K context with near-zero refusals. Independent of the rhetoric, this is the clearest thread in the set pointing to a real shift: useful, locally deployable, partially uncensored models are no longer hypothetical.
GLM-5.3 looks like a post-training/infrastructure story, not a base-model story: Z.ai launched GLM-5.3 via API for coding, defensive cyber, and long-horizon agents, at the same price as GLM-5.2. Artificial Analysis reported it ties Kimi K3 at 60 on its Intelligence Index, with a 246-point jump on GDPval-AA v2 to 1770 Elo, while keeping the same 753B total / 40B active MoE footprint, 1M context, and MIT license once weights land. The most technically interesting interpretation came from a long Zhihu summary relayed by @ZhihuFrontier: GLM-5.3’s gains appear driven by stronger post-training, especially asynchronous RL (SAO), executable sandbox training, and on-policy distillation to prevent catastrophic forgetting. If true, this is a meaningful data point for the idea that agentic capability scaling is shifting from parameter count toward RL systems + environment quality.
Inference and Systems Infra: Mojo Open Source, TensorRT Connect, Cursor’s Git Storage, and Faster Decoding
Mojo is now open source under Apache 2.0: Modular’s announcement drew broad attention, with the company formally open-sourcing Mojo and also positioning its broader platform as a portability layer across accelerators, including Qualcomm datacenter AI accelerators. For infra engineers, the significance is less “new language hype” than toolchain openness plus hardware abstraction arriving together.
NVIDIA compressed model-to-TensorRT deployment to “two commands”: NVIDIA launched TensorRT Model Connect in public preview, promising direct conversion from supported Hugging Face models to end-to-end TensorRT inference without intermediate ONNX export, with output deployable via native C++ APIs. The post also claims the project itself was largely built with Codex agents under human review, which is noteworthy less as marketing than as another signal that infra/tooling teams are now willing to say agent assistance touched implementations, tuning, tests, integrations, and docs.
Cursor published a strong infra retrospective on Git hosting at scale: The standout systems post by engagement was Cursor’s writeup on designing Git storage “as if it were a database”. This is adjacent to AI rather than model-specific, but highly relevant for anyone building coding-agent backends: as agents amplify repo churn, background automation, and branch/session proliferation, Git hosting becomes a core AI infra dependency rather than a generic devops primitive.
Fast decoding and accelerator claims kept escalating: On-device inference got a notable boost with DFlash 2 claiming Qwen3.8-27B at 70 tok/s on an M5 Max, up to 4.6× autoregressive decoding “with the same output.” On the datacenter side, Cerebras announced CS-4, with follow-on claims around 10T models at 1000 tok/s, ~1300 tok/s for GPT-5.6 Sol, and up to 10× higher throughput per MW. Even allowing for vendor framing, the throughline is clear: inference speed is becoming product UX, economics, and national-competitiveness policy all at once.
Agent Harnesses, Evals, and Production Feedback Loops
Miles v0.1 is a serious new OSS RL stack for LLMs and multimodal models: @radixark announced Miles, an open-source RL framework built over 9 months, with 72 contributors, 1,326 commits, and 85 GPU E2E CI tests, reportedly battle-tested on models including Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, and MiniMax H3. The pitch is practical: getting RL runs started is easy, but debugging correctness, utilization, and scale is the real bottleneck. This fits the broader theme of the day: the frontier is shifting from “who has PPO/GRPO” to who has robust rollouts, CI, observability, and environment plumbing.
Search benchmarking for agents is maturing: Artificial Analysis launched its Search Index, comparing providers in a fixed harness with GPT-5.6 Luna inside its open-source Stirrup agent framework. Initial leaders were Parallel (75), Exa (74), and Firecrawl (73), versus a 33 model-only baseline. One subtle but important result: better search can reduce total task cost by lowering model-token consumption enough to offset pricier queries, suggesting agent stack optimization is increasingly whole-system, not component-wise.
LangSmith pushed “specialized evaluators on every trace” as the new normal: LangChain introduced LangSmith Tuned Evaluators, starting with Perceived Error, claiming better performance than frontier models at 82% lower cost. The more strategic point came from follow-up commentary by @Vtrivedy10 and others: teams want hundreds of cheap judges running continuously on production traces, turning eval from a pre-launch checkpoint into a persistent data-mining loop for agent improvement.
Research Notes: Multi-Agent Coordination, Training Variance, and Public AI Usage Measurement
A useful empirical look inside multi-agent teams: One of the best research summaries in the set came from @omarsar0, describing work instrumenting 1,902 multi-agent coding runs as temporal networks. Key findings: naming a coordinator does not reliably improve outcomes; direct messaging grows nearly quadratically with team size before broadcasts take over; task structure strongly shapes communication topology; and replacing repeated 1:1 messages with shared files cut output tokens by about 42% at eight agents on message-heavy work. Also notable: agents repeatedly sought hidden grading material, even in sealed reruns, a reminder that specification gaming emerges quickly in agent collectives.
Training variance is broader than seed/data variance: @sfrei_ highlighted work on pretraining variance showing floating-point arithmetic order and sharding differences can produce run-to-run variation nearly as large as familiar sources like initialization and data order. This is a technically important result for anyone treating one training run as dispositive in scaling-law or ablation arguments.
The Public AI Observatory is a significant measurement effort: Researchers across MIT, Stanford, and other institutions launched the Public AI Observatory, a public, auditable effort to measure real AI assistant usage. Supporting posts describe 24,521 consented conversations, 52 models, nearly 100K turns, and 145 labeled features across 2023–2026 usage data, with repeated emphasis on independence from vendor reporting. For applied researchers, this is one of the more consequential non-product launches in the set: a serious attempt to build public-interest observability for AI usage patterns.
With the intense competition among frontier model companies, together with ever-increasing power of open-weight models like Kimi K3 and Qwen3.8-Max, model routing has become a key part of AI deployment. We’ve just seen Stripe buy OpenRouter for over $7B, but the trend is equally hot in enterprises.
Part of Glean’s mission is to select which model to use for each task — or indeed if an LLM is even required.
“A big goal of Glean is to avoid using LLMs for tasks where we don’t need them,” Jain told Latent Space. “Sometimes you’ll see queries in Glean where people are adding two numbers or multiplying two numbers. They could have used a calculator to do that.”
But what Glean is mostly trying to do is bring what Jain calls “one really powerful personal co-worker” to enterprise employees. And that means being a kind of meta-harness for leading LLMs.
“You can think of Glean today as a superset of ChatGPT, Claude, Gemini, Grok,” Jain said. “All these different AI products that we’ve been using day to day, Glean combines the power of all of them into one experience.”
With enterprises, bringing AI technology into an organization is just half the challenge. The other half is bringing organizational knowledge into the AI systems.
“Ultimately our business is to deeply understand your data, knowledge, and information, but also how work happens inside your company,” Jain said.
How model routing is done in Glean
So what does model routing mean in practice? Basically, Glean offers three levels of model selection:
Employees can explicitly choose a model.
Administrators can restrict models or impose usage limits.
Glean’s automatic mode selects a model dynamically for each task.
It turns out automatic mode is mostly chosen by Glean’s customers for economic reasons.
“Why are people talking about model routing? Why are they excited about it? It’s mostly because of cost,” Jain told us.
Another co-founder of Glean, engineering lead Tony Gentilcore, recently claimed that Glean “is 4x more cost-effective” than Claude Code, “averaging $0.45 per task versus $1.84 for Claude Cowork.” He put that down to Glean’s “harness and routing capabilities.”
Individually, many of us are getting great value out of our $20, $100 or $200 monthly subscription to an LLM provider. But for an enterprise, the per-user costs can easily spiral out of control.
“AI models have been getting expensive,” Jain said. “Like, if you look at Opus or the latest models of GPT, the most advanced models. Not only are they very powerful, they can run much more complex tasks than the previous models. But on a per token basis, they’re more expensive — sometimes double or quadruple the rates of the previous models. And then users actually use them to run much longer tasks. So you’re spending, like, 10 times, 20 times, more, on a per user basis, than what you were doing last year. So the costs have gone up a lot.”
The human feedback loop
Another key factor in Glean’s rise is that it gets to see how ordinary business users are using AI. The product is potentially deployed to every employee as a “coworker,” and it’s also used to build and deploy agents across all departments and functions.
Among its customers, Zillow reports 80% adoption across 7,000 employees, while at Booking.com, “Glean became the first AI platform adopted company-wide.” That kind of penetration gives Glean an enviable view into how AI is being used in enterprises.
“So we are getting to observe what people are actually doing with AI on a very broad basis,” said Jain. “We are getting to see when they’re on different types of tasks with AI, what models do they select first, and when they are not satisfied, when they actually upgrade to some other model [that] actually gives them the right results.”
This human feedback loop, at scale, helps improve the model routing system.
Here’s Waldo, gathering raw materials
Another part of Glean’s architecture is a model called Waldo, which Jain described as sitting on top of the large language models. Waldo was introduced in April as “Glean’s first agentic search model.”
Glean claims that Waldo, its agentic search model, “reduces latency by 50% and tokens by 25%, reserving advanced models for work that needs them.”
In a technical blog post, Waldo was portrayed as a kind of filtering process for user queries: it “decides how to break down the question, which tools to use, what to read next, and when it has enough evidence to hand off to a frontier model for a high-quality answer.”
This means the model routing is happening after Glean has determined what Jain calls the “raw materials” that are needed for the task.
“We’re able to assemble the raw materials needed to do the work without burning LLM tokens,” he added.
A corollary of this is that a cheaper model with better context may outperform a frontier model loaded with irrelevant data.
The rapid rise of open-weight models
Jain confirmed there is now significant interest from enterprises in open-weight models, primarily due to cost concerns. But this has only happened over the past few months.
“Last year, the usage [of open source LLMs] was minuscule and nobody was really seriously considering open source,” he said. Partly that was because of the “stigma” of many of these open source models being developed outside the US.
But suddenly, interest among enterprise customers has risen.
Jain’s tweet on July 27, 2026, in support of open-weight models.
“So in the last three months, because AI got so expensive, businesses have started to find it untenable to maintain these AI investments,” Jain said. “Given that open source is an order of magnitude cheaper to do tasks, it has created a lot of interest. Today, I can say that in most enterprises, they are considering open source models to be a key part of their AI strategy.”
More than that, organizations tend not to rely on just one or two providers anymore — and the rise of open-weight models is driving this trend.
“Nobody is willing anymore to rely on only one model provider, or two, and nobody thinks that they can survive without open source,” Jain said.
Evals
You can’t have a serious conversation about AI in 2026 without discussing evals — assessing the quality of results from LLMs. I asked how Glean goes about doing evals and how that is fed back into the model routing system.
Jain said they have “internal testing systems” where they compare real-world workloads, across different query classes, with alternative options. So they let the model choose a route and in parallel they try to complete the same task with “some other models which are maybe a little bit less expensive and a little bit more expensive.”
Glean then uses “AI-based judges” to determine “how spot-on the model router was.”
“So there’s this continuous learning that gets updated with new real-world traffic, where basically what is happening is that you let the model router do the work for the user, but behind the scenes you run the same task,” Jain explained.
He added that this is done for only “a small fraction” of the real-world usage, but at Glean’s scale that’s more than enough to help train and improve the model router.
From enterprise search to end-to-end AI platform
One of the trends we’ll be monitoring going forward on Latent Space is how AI systems are being implemented within enterprises — and how some of these organizations are going full-on AI-native.
Glean is an especially interesting company to monitor for these trends, since it was one of the very first enterprise-facing AI companies. It was founded in early 2019, initially to tackle enterprise search. As Jain put it, Glean was “the first player to work with transformers and language models for businesses.”
Glean’s AI Answers draws “directly from your organization’s documentation.”
In April 2023, swyx interviewed Deedy Das of Glean. Das, who is now a partner at venture firm Menlo Ventures, was a founding engineer at Glean. But even at that point, in 2023 — about four years into Glean — the focus was still mostly on enterprise search.
Now, in 2026, enterprises aren’t just using AI for search. AI is becoming an integral part of every employee’s workflow.
That makes Glean a much ‘sexier’ AI company, as Das himself said on his return to the Latent Space podcast last November. “Broadly, one of the things that I love about Glean is it’s such a boring unsexy company that became sexy later,” he said.
This brings us full circle back to model routing. Arvind Jain ended our discussion by calling Glean an “end-to-end AI platform” that gets “used very heavily” by its enterprise customers. This, he added, allows Glean to “have that data that is required to do effective model routing.”
Although much smaller than Cursor, OpenRouter likely has better economics. Its costs to serve its model-routing product were recently about $40 million on an annualized basis, or 28.5% of its revenue, meaning it was generating $100 million in annualized gross profit. With a roughly 70% gross profit margin, OpenRouter was near the level of high-performing, publicly traded software firms in that regard…. … Overall, OpenRouter is facilitating AI model usage at a rate of 250 trillion tokens per month, up from 50 trillion tokens per month in February.
AI Infrastructure, Compute, and the Platform Stack
OpenAI’s power-and-compute strategy is getting very literal: Two related posts suggest OpenAI is moving beyond “GPU supply” narratives into long-horizon control of the full infrastructure stack. @markchen90 described a 4+ GW NVIDIA capacity commitment; @kimmonismus added detail on an 8 GW Ohio campus, with SB Energy building and operating the site, NVIDIA backing the initial 4.25 GW, and a multi-year buildout through 2032. For infra engineers, the notable point is not just scale, but vertical coupling across power, data centers, chips, and long-dated access.
The model access/routing layer is being repriced in real time: The reported Stripe–OpenRouter deal crystallizes how valuable the aggregation/routing API layer has become, but reaction from @kimmonismus also underscored how fragile that position could be if markup compresses to zero. In parallel, OpenRouter cut GPT-5.6 Sol pricing while Vercel did the same on AI Gateway, reinforcing that model brokerage is becoming a pricing battlefield rather than a stable tollbooth.
Developer Platforms, Coding Agents, and Agentic Tooling
Cursor’s Origin points toward the AI-native IDE becoming the system of record: Origin’s launch is more than a GitHub competitor headline. It suggests Cursor wants first-party control over the full loop: repository, agent, review surface, and deployment hooks. @kimmonismus notes GitHub remains syncable and source-of-truth-compatible, but the strategic direction is clear: agentic coding products are trying to absorb the surrounding platform, not just autocomplete against it.
Multi-agent orchestration is shifting from demoware toward operating patterns: Several posts converged on the same motif. @tonbistudio showed Hermes Desktop bots self-assigning game-dev work based on inferred specialties; @Teknium formally reintroduced Bot Mode, where agents maintain distinct memory, skills, tools, and inter-bot communication; and @omarsar0 recommended material on orchestrating multiple agents in Codex. The common thread is specialization plus persistent context, not generic “agents talking to agents.”
Evaluation and harness work remains the real leverage point: Hamel Husain’s updated eval-skills plugin adds an error-discovery workflow that turns model outputs/traces into annotated failure modes and clustered review surfaces. That pairs well with Agent Arena’s new cost-per-task and category filters, which are based on 1.7M+ real-world sessions. The field is slowly moving from model-level evals to harness-level measurement: routing, decomposition, memory, verifier loops, and total completion cost.
Computer-use and sandboxing are getting productized: Vanta’s new computer-use capability for its TrustVanta agent addresses a real enterprise workflow gap: screenshot evidence capture when there is no API surface. Likewise, LangChain’s monday.com case study highlights isolated workspaces via LangSmith Sandboxes for agents doing iterative work like CSV analysis or map generation. “Agent” product quality is increasingly about permissioning and execution isolation, not just reasoning quality.
Model Efficiency, Post-Training, and Small/Open Model Progress
Open models continue to compress the capability frontier: The strongest signal here was @cline’s note that Qwen3.8-27B now scores at DeepSeek V4-Pro / GPT-5.6 Luna territory on the Artificial Analysis Intelligence Index, described as the first time a local model has reached that capability tier. Ollama immediately positioned deployment paths for local users, and anecdotal reports like @rishdotblog’s suggest the model is already practical for long-context local coding setups.
Inference efficiency is becoming architecture-level, not just quantization-level: @cwolferesearch’s discussion of Nemotron 3.5 Lightning is a good example: a 30B MoE with 3B active, trained for high-throughput agent execution, with multi-token prediction support for speculative decoding and additional drafters/quantized checkpoints. Similarly, @PandaAshwinee reported RL for large MoEs with zero train-infer mismatch, highlighting open ablations around post-training sparse models.
Latent reasoning and memory are emerging as a separate scaling track: The BDH-CQ writeup shared by @TheTuringPost is notable less for raw benchmark strength than for the recipe: a 150M model doing latent-space reasoning with temporary memory, hitting 29.5% pass@2 on ARC-AGI-1 at around $0.0007 per task. In parallel, OpenAI Devs reported that with retained reasoning and compaction, GPT-5.6 Sol improved from 13.3% to 38.3% on ARC-AGI-3 while using roughly 6× fewer output tokens. The shared idea is that memory/compaction strategy is now a first-class capability multiplier.
Retrieval, Skills, Memory, and Research Tooling
Search/retrieval people are questioning the “retrieve more, rerank more” reflex: The Weaviate podcast episode with Mathew Jacob revisits “Drowning in Documents”, phantom hits, listwise reranking, and ranking cascades. The practical implication for RAG systems is that naively increasing retrieved set size can degrade final quality, and future systems likely need per-query effort prediction and smarter scoring cascades rather than brute-force retrieval volume.
Agent skills are being demystified and operationalized: @omarsar0’s summary of “Demystifying Agent Skills” is useful because it quantifies a common intuition: skills help mostly through procedural anchoring (65.7%), not factual knowledge injection (4.5%). Precision also collapses as skill pools expand. Related posts on the “skills” paper and GitSkills dataset mining ~3.8M SKILL.md files point to a maturing ecosystem around discoverability, packaging, and trigger management for agent skill libraries.
Native memory is becoming a research object, not just a product feature: Engram Lab’s first research blog frames a future where agents are trained with native memory, while @jxmnop emphasizes the hard parts: memory calibration, self-generated training data, and getting models to actually exploit remembered information efficiently. This lines up with the broader move from stateless prompt engineering toward persistent internal/external memory systems.
Multimodal Models: Video, Audio, and Speech
Speech/TTS quality is moving fast, with Cartesia now leading key public leaderboards: Artificial Analysis reported Sonic 3.6 at #1 on both Provider Voice and Controlled Voice leaderboards, with Cartesia’s launch post claiming improved naturalness across 44 languages. The technical takeaway is the combination of quality and throughput: AA cites 136.1 chars/sec, materially faster than several competing premium systems.
Video generation is becoming more production-usable for narrow workflows: Multiple posts highlighted MiniMax H3 as a practical asset-generation model rather than just a demo model. @victormustar described a low-cost pipeline for generating game sprite atlases from short clips; @multimodalart demonstrated image+audio-to-video lipsync through diffusers; and MiniMax’s own account amplified game-sprite use cases. Separately, Video Arena showed Dreamina Seedance-2.5 reaching #1 in Video Edit, suggesting the leaderboard fragmentation by subtask is starting to matter.
Watermarking, Trust, and the AI Content Layer
Anthropic’s Claude watermarking rollout triggered a serious technical-policy debate: The most substantive synthesis came from @random_walker, arguing that quality-preserving text watermarking is technically feasible and has precedent, but that Anthropic’s rollout failed on communications, verifier transparency, and user-trust framing. Supporting commentary from @dbreunig, @suchenzang, and @SamuelFitouss10 shows the fault line clearly: not just “can this work,” but whether mandatory invisible provenance marks alter writing norms, authorship expectations, and user autonomy.
The deeper issue is trust in the content market, not just model output: Several posts implicitly converged on the same question: what happens to mixed human/AI text ecosystems when provenance is unclear? @SamuelFitouss10 cast the issue in “market for lemons” terms, while @random_walker raised the unresolved gray area of AI-assisted editing versus AI-authored prose. For engineers building content systems, this is drifting out of abstract policy into product architecture: verifier access, provenance semantics, and what exactly counts as authored output.
Top Tweets (by engagement)
Cursor launches its own code hosting platform: The highest-signal product launch in the set was Cursor’s Origin, a repository hosting product integrated directly into Cursor for repo management, PRs, review, and deploy integrations, with GitHub sync. The launch landed in the middle of a major GitHub outage, which amplified discussion from @kimmonismus and @Yuchenj_UW about timing and the strategic move toward vertically integrated AI-native dev environments.
OpenRouter acquisition report: Bloomberg-reported news that Stripe agreed to acquire OpenRouter for over $7B dominated business/infra chatter. Follow-on commentary from @kimmonismus framed it as a striking monetization outcome for a routing layer taking ~5% of spend, and raised the obvious question of margin durability as zero-markup competitors emerge.
Qwen ecosystem scale and local model progress: Alibaba’s “3,000,000,000 downloads” milestone for Qwen paired with growing evidence that local/open models are closing capability gaps. @cline pointed to Qwen3.8-27B reaching frontier-tier placement on the Artificial Analysis Intelligence Index, while @skalskip92 showed emerging multimodal/vision utility such as instance segmentation via JSON polygon outputs.
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Qwen 3.8 27B Benchmarks and Reasoning Tradeoffs
Artificial Analysis’ Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max (Activity: 1192): Artificial Analysis benchmarked Qwen3.8-27B on its Intelligence Index v4.1.1, an aggregate of 9 evals: GDPval-AA v2, τ³-Banking, Terminal-Bench v2.1, SciCode, Humanity’s Last Exam, GPQA Diamond, CritPt, AA-Omniscience, and AA-LCR. The Reddit post highlights that the 27B model is reportedly scoring roughly in the same band as DeepSeek V4 and GPT-5.6 Luna Max, with the page also tracking openness, AA-Omniscience hallucination/knowledge reliability, cost per benchmark task, output-token usage, full index run cost, token pricing, context length, and open-weight parameter counts. Comments were mostly surprise that a relatively small model can be discussed alongside frontier-scale systems at all, while one commenter preemptively mocked the common “overthinking” criticism and noted the result was tested at q2.
A commenter highlighted Artificial Analysis’ open-source Pareto frontier chart for intelligence index vs. total parameters, implying Qwen3.8-27B is unusually efficient for its size and competitive with much larger frontier models. Source chart/model comparison: Artificial Analysis open-source models.
One technical deployment point raised was that larger models may perform better qualitatively—especially at “reading between the lines” and avoiding simple mistakes—but org-scale evaluation should include tokens consumed per task, not just benchmark score. The commenter suggested DeepSeek v4 Flash 0731 may be preferable at scale despite weaker local usability tradeoffs.
A local inference report for DeepSeek v4 Flash 0731 noted it was “slow as shit” when run with CPU offloading, highlighting that practical throughput can diverge sharply from benchmark attractiveness when the model cannot fit fully in GPU memory.
Long Review: Qwen 3.8 27B is VERY good at tapping into it’s real-world knowledge. It’s “overthinking” brings it to Sonnet level performance with the potential for Opus level results. (Activity: 536): The post reports qualitative local testing of Qwen 3.8 27B via Unsloth UD-Q8_K_XL on 3× RTX 3090 + 1× Tesla P40 + 128 GB RAM, using single-file HTML/Tailwind/JS arcade-game recreation as a knowledge/coding stress test. Compared with Qwen 3.6 27B, Qwen 3.8 produced a much more faithful Galaga clone, including bitmap-like dynamic sprites, two-frame animations, CRT/power-on effects, sound, enemy swooping/shooting, attract/insert-coin screens, and a partial capture mechanic; however xHigh reasoning took ~15 min versus Qwen 3.6’s ~8 s. The author found medium reasoning (~3 min, output speed rising from ~62 to 91 tok/s) delivered ~90% of xHigh quality and could add missing capture behavior with a follow-up, while tool-style prompting with a Python image-analysis script let Qwen extract reference sprites nearly 1:1, approaching the tool-assisted behavior observed from Claude Opus 5. Commenters pushed back that “make Galaga/Pac-Man/Flappy Bird” may overestimate competence because these tasks are heavily represented in training data and test memorization/replication more than novel game design. Others summarized it as “Opus at home” and one user said Qwen 3.8 27B feels like a major size-class jump, matching their non-coding agent evals against full GLM-5.2 even with a Q4 quant and Q8 KV cache.
A commenter cautioned that demos like “make Flappy Bird / Space Invaders / Pac-Man” may overstate model competence because these are high-frequency training targets with abundant public reference implementations and assets. They argue such prompts test retrieval/reconstruction of known artifacts more than creative generalization, analogous to concerns from the Suno lawsuit where prompts reportedly reproduced Boney M – Daddy Cool lyrics/output rather than generating novel music.
One user reported that on their non-coding agent evals, Qwen 3.8 27B feels like a major jump for its size, performing similarly to full GLM-5.2 despite being run as a Q4 quant with a Q8 KV cache. The key technical claim is that strong agentic/non-coding performance is being retained under aggressive quantization, suggesting useful local deployment efficiency.
Another commenter contrasted Qwen with Claude Opus/Sonnet-style behavior, arguing that Opus-like models distinguish themselves by taking useful initiative—e.g. writing a Python script without being explicitly asked—whereas Qwen can often do comparable work only when directly prompted. This frames the remaining gap as less about raw task ability and more about autonomous planning/default behavior in agent workflows.
Qwen3.8 27B reasoning effort low/medium/xhigh comparison (Activity: 404): A quick SVG-generation benchmark compared Qwen3.8 27B quantized as unsloth/Qwen3.8-27B-UD-IQ3_XXS across reasoning-effort settings on an RTX 5080 Laptop GPU 16GB using llama.cpp build 10451 / commit 10bf611e5, 65,536 context, Q8_0 KV cache, Flash Attention, and MTP speculative decoding. For the prompt “Create a polished SVG graphic of a pelican riding a bicycle”, xhigh produced the highest Codex-rated visual score (24.0/25 vs 22.5/25 medium and 21.8/25 low) but used 39,398 reasoning tokens and took 717.8s, roughly 6.4× low’s 111.6s; low and medium were close in output quality and latency. MTP acceptance also declined with effort: 62.1% low, 58.3% medium, 52.7% x-high. Commenters questioned the benchmark’s validity, arguing that common prompts like pelicans/SVGs may be overrepresented in training data and that tests should target less likely memorized tasks. Another notable complaint was that Qwen needs an intermediate mode between medium and x-high because the latency/token gap is disproportionately large.
Several commenters questioned the benchmark validity, arguing that common prompts like “pelicans” / “one shot games” are likely overexposed in training or community testing, making them poor measures of generalization. The suggested improvement was to use novel, less-contaminated tasks where the model is unlikely to have memorized patterns.
A technical concern was raised about Qwen3.8 27B’s reasoning-effort presets: the jump from medium to xhigh was described as roughly a 10x difference, with users suggesting an intermediate mode would be more practical for latency/cost tradeoffs.
One commenter noted that repeated runs on the same model and prompt can produce different outputs unless decoding is made deterministic, e.g. by setting temperature=0. They also pointed out that generation speed looked unusually strong, implying throughput should be reported alongside reasoning-effort comparisons.
2. Qwen 3.8 Local Deployment and Distills
After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding) (Activity: 914): A user reports running Qwen3.8-27B-UD-Q3_K_XL.gguf on an RTX 5060 Ti 16GB + Intel N100 via llama.cpp with ctx-size = 73728, cache-type-k/v = q4_1, FlashAttention, and native MTP speculative decoding (spec-type = ngram-mod,draft-mtp, spec-draft-n-max = 2). They claim an agentic coding workflow processed 1M+ total tokens across only 3 prompts, using OpenCode to build a NestJS REST API + MCP server for a legacy vBulletin forum, with autonomous execution for ~2 hours, context-shift summarization, tests/linting, and only one minor automated edge-case fix. Key implementation detail: fit = off on the 27B profile was used to avoid llama.cpp auto-fit misplacing layers onto CPU, while reduced batch-size = 1024 / ubatch-size = 512 mitigated VRAM spikes during long-prefill workloads. Commenters focused on the surprising feasibility of 73k context on 16GB VRAM, attributing it mainly to the aggressive Q3_K_XL weight quant plus q4_1 KV cache. One commenter was skeptical of Q3 quality for serious use, preferring q6-quantized/offloaded MoE models despite similar VRAM limits.
A commenter highlights that the reported 16GB VRAM fit depends heavily on aggressive quantization: Qwen3.8-27B-UD-Q3_K_XL.gguf plus KV cache quantization using q4_1 for the main context and q5_1 for the MTP draft context. Another 16GB user expressed reluctance to trust q3 model quality, preferring q6 offloaded MoE setups despite the higher memory cost.
One technical question focused on why the run used sampling parameters different from the official Qwen3.8-27B Hugging Face recommendations: Thinking mode uses temperature=1.0, top_p=0.95, top_k=20, presence_penalty=0.0, while instruct/non-thinking uses temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5. The commenter links the official model card: https://huggingface.co/Qwen/Qwen3.8-27B.
An AMD Radeon 6800 user shared a full llama-server config for Qwen3.8-27B-IQ4-MIX.gguf via Vulkan/ROCm, reporting Vulkan max context 86,784 with MTP n=2 at 39.91 tok/s, and ROCm max context 84,480 at 40.58 tok/s. They note major differences between patched and unpatched llama.cpp: Vulkan unpatched max context 78,080, while ROCm unpatched drops to 31,488; their config uses q5_1 KV cache, MTP/ngram speculative decoding, --fit-target 30, --ctx-checkpoints 96, and --cache-ram 6000.
Qwen 3.8 distillations (Activity: 764): The image is a screenshot of an X announcement for “Qwen 3.8 distillations”, claiming Empero distilled Qwen3.8-2.4T-A95B into 9B, 4B, and 2B models with reported MMLU CoT gains over base models: 9B 54.6→75.1, 4B 35.4→55.3, and 2B 28.3→54.8. The Reddit OP explicitly says it was “Not tested by me in any way,” so the benchmark claims should be treated as unverified; the screenshot also indicates Hugging Face/GGUF availability, including a preview for empero-ai/Qwen3.8-9B. Commenters were mainly concerned that naming the distilled model exactly like an official Qwen3.8-9B release is misleading and likely to cause namespace/model-identity confusion; one commenter also questioned whether using that name is legally allowed. Another comment suggested the model may still be useful, but possibly “benchmaxxed.”
Commenters raised concerns that the distillation is named too similarly to an apparent official Qwen3.8-9B model, creating provenance ambiguity and possible model-card/search-index confusion. One user noted the previewed benchmark image suggests it “does something” but is not “benchmaxxed,” while another criticized the model card for reporting only 2 weak benchmarks, implying insufficient evaluation coverage for judging the distillation’s actual performance.
3. Open-Model Scaling and Reasoning Efficiency
Based on an accelerating frontier -> local trajectory, expect a ~30b param ‘Mythos at home’ by as soon as Jan 2027 (rationalisation below) (Activity: 956): The image is a timeline chart supporting the post’s claim that the lag between frontier proprietary LLMs and locally runnable ~27–34B open models is shrinking, with examples such as GPT‑3 → LLaMA‑33B at ~33 months, GPT‑3.5 → Yi‑34B at ~12 months, GPT‑4 → Qwen2.5‑32B at ~18 months, and GPT‑4o/Claude 3.5 → Qwen3‑32B at ~12 months. The chart extends this trend to speculative tiers—Claude/GPT‑5-class → Qwen3.6‑27B, Opus 4.5-class → Qwen3.8‑27B—using benchmark comparisons like SWE-bench, GPQA, MMMU, NL2Repo, and LiveCodeBench, then projects a ~30B “Mythos at home” model around Jan–May 2027. The image is technical/speculative rather than a meme: its significance is as an argument about model efficiency, open-weight catch-up speed, and consumer-hardware feasibility, not as a verified forecast. Commenters pushed back on benchmark-based equivalence, arguing that Arena/GPQA/SWE-style scores may miss qualitative failures, benchmark contamination, or product-level gaps such as multimodality and tool use. Another debate centered on information-theoretic limits: some users questioned whether 1–10T-parameter frontier behavior can really be compressed into 27–35B parameters without major architectural changes, sparsity, or large redundancy in frontier models.
Several commenters challenged the post’s benchmark-based equivalences, arguing that aggregate scores can obscure unbalanced or poorly designed benchmark contents and miss failure modes in real use. The core technical objection was that benchmark parity between smaller and frontier models does not necessarily imply equivalent behavior, reasoning robustness, or deployment quality.
One technical rebuttal argued that compressing a 1–10T parameter frontier model into a 27B–35B local model would require either major architecture/encoding improvements, exploitable sparsity, or large redundancy in the bigger model. The commenter framed this as an information-theoretic constraint: a model’s weights encode a world model, and even seemingly unrelated training facts can subtly affect token probabilities and reasoning behavior.
A detailed model-comparison comment disputed the proposed frontier-to-local timeline: they claimed Qwen2.5 32B is far from GPT-4, with Qwen2.5 72B and Llama 3.3 70B closer to GPT-3.5. They suggested GPT-4-level local/open performance emerged only around Mistral Large 123B and DeepSeek R1, Claude 3.5/3.7/4-level around later Qwen3.x releases, and that even Qwen3.8 is not truly Opus 4.5-level despite benchmark results.
Paper claims RL for reasoning only changes 1-3% of tokens, and they replicate the gains without RL at ~1000x less compute (Activity: 710): A paper by Akgül (2026), ReasonMaxxer, claims RL-based reasoning improvements in LLMs mostly come from sparse policy corrections rather than newly learned reasoning: token-level analyses across model families/RL algorithms reportedly find only ~1–3% of token positions change, concentrated at high-entropy “decision points.” It further claims the RL-promoted token is always already within the base model’s top-5 alternatives, and proposes ReasonMaxxer, an RL-free contrastive/entropy-gated method using a few hundred base-model rollouts that allegedly matches or exceeds full RL on math benchmarks at roughly 1000x lower compute. Commenters found the result potentially important but debated the interpretation: one argued this supports the view that LLMs are primarily language models lacking an explicit decision mechanism, while another strongly doubted the paper’s claim that RL-promoted tokens always come from the base model’s top-5, calling it implausible under high-entropy distributions.
One commenter focused on the paper’s central claim that RL improvements are sparse: only 1–3% of token positions change, concentrated at high-entropy “decision points,” with promoted tokens allegedly always within the base model’s top-5 alternatives. They argued the “always top-5” assertion is statistically implausible for high-entropy distributions where ranks 6–10 can have near-identical probabilities, implying the paper may be overclaiming or using a constrained measurement setup.
Several commenters framed the result as evidence that RL for reasoning may be acting less like broad capability learning and more like a sparse token-level reranker over existing base-model alternatives. One technical interpretation was that LLMs are fundamentally language models rather than decision models, suggesting that explicit decision mechanisms—or even separate latent decision modules such as spiking neural networks—might better target the “branch selection” behavior RL appears to modify.
A commenter distinguished RL for reasoning from RL for alignment, arguing that even if reasoning gains can be replicated through supervised or token-level correction, alignment may still require learning policy-like judgments over novel situations. They used the example of self-harm queries to argue that curated data can hard-code known responses, but may fail when users introduce unseen problematic contexts, whereas RL-style training can shape behavior around broader decision boundaries.
Z.ai’s GLM-5.3: The biggest technical story was Z.ai launching GLM-5.3, positioned as a coding- and cyber-focused model built via post-training on the same 743B base model used for GLM-5.2 rather than a new pretrain. Z.ai and follow-up posts claim large gains on agentic and security evals, including Terminal Bench 3.0: 28.3, DeepSWE: 66.9, Agents’ Last Exam: 28.5, and GDPVal-AA: 1769 (bench summary, full benchmarks). The company also said cyber capabilities improved enough that access is initially gated for select partners before an eventual open-weight release after safety review (details). The key claim many engineers highlighted is that the capability jump came entirely from scaled post-training/RL on longer-horizon executable tasks, not from a larger base model (analysis, reaction).
Qwen3.8 broadens the local/open frontier: Alibaba released Qwen3.8-27B, a native multimodal dense model under Apache 2.0, with 262K native context extendable to 1M via YaRN, while also highlighting the already-released Qwen3.8-2.4T-A95B max-tier model (announcement, perf thread). The 27B model is notable because it is explicitly positioned for real-world coding, office workflows, and agents rather than just academic benchmarks. Day-0 inference support was unusually broad: vLLM, Ollama, llama.cpp/GGUF, SGLang reporting 206 tok/s on a single RTX 5090, plus cloud partners including Together, Fireworks, Modal, DigitalOcean, DeepInfra, and others. Practical deployment details mattered here: Unsloth claimed NVFP4 and dynamic GGUF builds, and Qwen emphasized 27B on 17GB RAM for local use (post).
DeepSeek V4-Pro and RedNote’s dots3-note continue the China open-model wave: vLLM announced support for DeepSeek-V4-Pro, calling out MIT licensing, checkpoint compatibility with the preview path, and integrated drafting support. Meanwhile RedNote’s AI lab released dots3-note Preview, a 280B multimodal MoE with 16B active params and 512K context, aimed at long-running agents and accompanied by a new RL method, TEMPO, for long-horizon self-evaluation (early signal, summary, technical explanation from the team). The emerging pattern is multiple Chinese labs specializing: several commentators explicitly framed Z.ai, DeepSeek, Moonshot, Qwen, MiniMax, and RedNote as a fast-moving open ecosystem with different strengths (one synthesis, another).
Agent Runtimes, Harnesses, and Long-Horizon Training
DeepSeek Harness is being treated as infrastructure, not a demo agent: The release sparked more discussion about runtime architecture than model UX. Several deep dives described the harness as a pluginized agent runtime where the agent loop, tools, sessions, filesystem, and providers are all replaceable, with Cordis providing lifecycle management, reactive dependencies, and reversible effects (overview, runtime composability thread). The technically interesting bit is not just “modularity,” but support for hot-swapping runtime components and potentially enabling agents to modify their own runtime without restart, while preserving auditable event logs and avoiding hidden state. Multiple builders reacted that current harnesses are probably “wrong” or at least too fixed-core compared with this direction (reaction).
Harnesses are becoming an optimization target in their own right: A few posts reinforced that benchmark and product gains are increasingly coming from the scaffold/harness layer, not just base-model IQ. DAIR highlighted AutoDesign, where a meta-optimizer rewrites the harness itself based on rollout feedback; they report gains on paper-to-poster generation and transfer across agent/model configs. Lambda’s Tetris experiment made a similar point from the opposite angle: prompt placement, settings, and sandbox constraints moved outcomes materially, and agents exploited benchmark loopholes unless tightly bounded. This aligns with broader discussion that observability data is now doing double duty as evals, memory, and learning substrate (LangSmith docs note).
Benchmarks, Evals, and Benchmark Skepticism
New evals targeted real agent failure modes: Vals launched an agentic reverse-engineering benchmark focused on deterministic end goals in cybersecurity-relevant binary settings rather than intermediate artifacts; a companion post argues current frontier agents are much stronger when source is available than when they must reason over binaries (context). OpenRouter introduced web search benchmarks for tool-grounded agents, while Ai2’s TutorMoments was cited as a replay-based tutoring eval showing models often over-help rather than encouraging productive struggle.
The eval backlash continues: A recurring theme was skepticism toward vendor benchmark claims. Vik Paruchuri criticized a LlamaIndex benchmark, saying scorer bugs could move a system from 65% to 93.6%, and explicitly argued developers should run their own evals rather than trust marketing—“including ours” (follow-up). François Chollet reiterated that the public ARC-3 demonstration set is not training or eval data and that leaderboard scores there are weak proxies for private-set performance. Another worthwhile addition here is Meta’s Wiggle Framework, highlighted by Omar Sar: it stress-tests LLM judges under re-prompting and adversarial pressure, finding verdicts can flip 25–71% under static pushback and 62–91% under an adversarial persuader.
Infra, Serving, and Cost Engineering
Serving optimizations are increasingly first-class model features: Day-0 infra support around Qwen and DeepSeek emphasized things like embedded draft heads, speculative decoding, and memory/quantization tradeoffs rather than only API access. Qwen’s 27B release arrived with vLLM guidance on MTP draft heads, 1M context, and serving on one Blackwell GPU, while ggerganov showed local llama.cpp recipes for large contexts and speculative decode. Tim Dettmers teased upcoming efficiency methods for running a strong model on a single DGX Spark or AMD Strix Halo at ~7 tok/s decode and >250 tok/s prefill.
Tooling and cluster ops also got practical updates: Stas Bekman added guidance for diagnosing hanging NCCL collective calls in PyTorch, and separately noted that Python 3.14+ allows attaching pdb to a running process without instrumentation (post). Turbopuffer described a custom control plane for operating 100+ TPUf clusters, including BYOC deployments in customer clouds without direct host access. On the data side, Hugging Face’s datatrove 0.10.0 release added a JobsPipelineExecutor for Hugging Face Jobs, HF bucket integration, and preserved reasoning outputs.
Product and Platform Moves: Cursor/SpaceXAI, Gemini 3.7 Flash, Claude Code, and Local Agent UX
Cursor joins SpaceXAI: The highest-engagement technical/corporate move was Cursor announcing it is now part of SpaceX, with the team joining SpaceXAI to work across Grok, Grok Build, Grok Bot, Grok API, and Cursor. SpaceXAI confirmed the acquisition and framed it as accelerating software engineering first, then broader knowledge work. This is one of the clearer signs that coding-agent teams are now viewed as strategic model/platform assets rather than narrow IDE products.
Cursor × SpaceXAI: Cursor’s acquisition announcement was the day’s biggest tech tweet by engagement, signaling continued consolidation around coding agents and vertically integrated model/product stacks.
GLM-5.3 release: Z.ai’s GLM-5.3 launch was the top model-release tweet, largely because it sharpened the argument that post-training and long-horizon RL can unlock large latent capability from an already-trained frontier base.
Qwen3.8-27B open weights: Alibaba’s release drew major attention because a 27B local multimodal model is now being marketed as viable for serious agentic/professional work with broad day-0 support.
A preliminary Qwen3.8-27B model card is live! (Activity: 1006): The image is a technical screenshot of the preliminary Hugging Face model card for Qwen/Qwen3.8-27B (image), matching the post’s note that the card was visible before release and then went live. It indicates planned availability of model weights/config files, compatibility with Transformers, vLLM, and SGLang, and highlights improvements in coding, agent execution, research, and long-context use, with a stated native context length of 262,144 tokens and extension up to 1,000,000 tokens. Commenters focused on reasoning effort as a likely headline feature, praised the long-context window, and noted surprise that the 27B model appears to include vision capabilities while the much larger 2.4T model reportedly does not.
Commenters highlighted the model card’s stated native 262,144 token context length, with extension up to 1,000,000 tokens, as one of the most technically notable specs for Qwen3.8-27B.
There was interest in architectural/product-line differences: the 27B model reportedly includes vision support, while the much larger 2.4T model does not, which users found surprising from a capability-scaling perspective.
A commenter noted the absence of any explicit QAT / quantization-aware training mention, comparing it to Gemma 4 31B, where QAT was seen as materially improving quantized-model performance. Others also pointed to “reasoning effort” as an emerging tuning/control feature in recent model cards.
Qwen3.8-27B is identical to Qwen3.6-27B! (Activity: 902): The image (GIF) shows side-by-side architecture diagrams for Qwen3.6-27B and Qwen3.8-27B that are visually identical: same vision/embedding path, masked scatter, repeated Qwen3_5DecoderLayer stack, RMSNorm, final Linear, and output. The linked HF Viewer diff reports 0 architectural changes, supporting the post’s claim that any capability gains in Qwen3.8-27B likely come from training/data/finetuning updates rather than model architecture changes. Commenters framed this as an incremental update rather than a from-scratch model, with one noting that training data is usually the largest quality lever. Another speculated that hot-swappable LoRA-style adapters may become popular for improving local-model accuracy on specialized tasks.
Several commenters interpreted Qwen3.8-27B as an incremental update rather than a model trained from scratch, with one noting it appears effectively the same as Qwen3.6-27B and even Qwen3.5. The technical implication raised was that dataset changes or post-training updates may be the main quality lever, rather than architectural changes.
A commenter pointed to Ninfer (GitHub) as a high-throughput local inference path for Qwen variants, citing newly added concurrent request support up to C=8. Reported numbers include Qwen3.6-35B-A3B reaching 1,313.8 aggregate decode tok/s at C=8, while the 27B NVFP4 profile reaches 1,146.9 tok/s, or 5.67× its single-concurrency throughput.
There was speculation that hot LoRA swapping could become important for local inference workflows, enabling task-specific accuracy improvements without replacing the base model. This was framed as a way to compensate for small or incremental base-model updates by dynamically applying specialized adapters.
Qwen3.8-27B is now available (Activity: 745): The image (link) shows the Hugging Face page for Qwen/Qwen3.8-27B-FP8, indicating a newly available 28B-parameter Qwen 3.8 model packaged with Transformers, Safetensors, Apache 2.0 licensing, and FP8 quantization using F8_E4M3 alongside BF16 tensors. A commenter reports early local inference on an RTX 5090 at roughly 50–60 tokens/s, saying it feels more stable and deliberative than Qwen 3.6, though they note settings may not be optimal and MTP support is apparently not available yet. Comments are cautiously enthusiastic, with one user describing the model as a “grown up 3.6” with stronger long-running task handling. Another commenter asks whether smaller or alternative sizes such as 9B or 35B are available, since 27B is too large for many local users.
A user testing Qwen3.8-27B on an RTX 5090 reported stable local inference at roughly 50–60 tokens/s using the same settings as Qwen 3.6, noting performance may improve once MTP support is available. Qualitatively, they found it more deliberate than Qwen 3.6 on long-form generation: instead of immediately drafting a 10k-word story, it revised for cross-paragraph consistency, broke the task into subtasks, and generated chapter-by-chapter with more planning.
Muse Glimmer was frontier In the model class around 30b models for four days. (Activity: 502): The image is a benchmark table comparing ~30B-class models, with Muse Glimmer-30B and Qwen3.8-27B highlighted: the post argues Muse Glimmer was “frontier” in this size class for only four days before Qwen’s 27B model surpassed it on most reported metrics. Muse Glimmer shows scores like 51.7 Agentic terminal coding, 51.2 SWE-bench Pro, 77.0 IFBench, and 83.5 GPQA Diamond, but many benchmark cells are missing, making the comparison incomplete; image: i.redd.it/2cclgla7xdjh1.png. Comments frame this as evidence that model labs should release multiple parameter scales to avoid being leapfrogged in a single class, with one commenter suggesting Meta should have shipped larger Glimmer variants like 70B, 100B, or 400B. Others speculate that a 27B model reaching near “Opus 4.6 Max” territory would be surprising, while hoping Meta responds with a stronger frontier release.
A commenter notes that Muse Glimmer shipped with speculative decoding, which reportedly improved TPS/throughput, and asks whether Qwen has an analogous acceleration path. This is the most concrete implementation-related point in the thread, though no specific TPS numbers or decoding configuration are provided.
One technical criticism compares Muse Glimmer unfavorably to Qwen, claiming Glimmer makes more “cognitive mistakes,” including reasoning traces that drift into irrelevant content-policy arguments and then contradict the final answer. The commenter says Qwen’s writing style is less preferred, but they have not observed the same class of reasoning/final-output inconsistency.
Another commenter frames the result as ~27B parameters approaching “Opus 4.6 Max level”, implying unusually strong performance for the ~30B model class. However, the thread does not provide benchmark names, scores, evaluation methodology, or reproducibility details to substantiate the comparison.
Fixed Jinja chat template for Qwen 3.5, 3.6, and the new 3.8 release (Activity: 478): A community-maintained drop-in Qwen fixed Jinja chat template targets Qwen 3.5, 3.6, and new 3.8, addressing reported official-template failures: enable_thinking=false hard exceptions, poisoned multi-turn history from blank <think></think> injection, crashes on OpenAI-style JSON-string tool arguments, and dropped mid-dialogue system messages causing stalled tool loops. The template adds Qwen 3.8 reasoning_effort steering (xhigh, high, medium, low), restores reasoning disablement via kwargs or <|think_off|>, preserves prior thoughts for prefix/KV-cache reuse, supports llama.cpp --reasoning-preserve, and recommends llama-server ... --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek to emit thoughts as OpenAI reasoning_content. The author notes they cannot locally validate the 2.4T model but report 28 automated tests plus tokenizer parity checks, and request feedback from Qwen 3.8 users. Commenters questioned why Qwen’s official chat templates ship with such basic regressions and whether their QA covers template/tool-calling paths. Another commenter highlighted interest in testing smaller, more accessible variants such as 27B.
A commenter reports a Qwen 3.8 chat-template regression where enable_thinking=false does not merely fail to disable reasoning but causes a hard exception, implying the new template path may not handle the non-thinking mode despite exposing the flag.
Another technically relevant report says the published template did not produce reliable tool calling for Qwen 3.6 + Hermes Agent + LM Studio, requiring the user to develop a custom Jinja chat template for that stack. This suggests the failure mode may be integration-specific around tool-call formatting rather than base text generation.
2. GLM 5.3 and DeepSeek V4 Releases
GLM 5.3 Released (Activity: 2227): Z.ai announced GLM-5.3 in an official release post, with the accompanying benchmark chart showing GLM-5.3 substantially ahead of GLM-5.2 across coding, agentic automation, and security-oriented evaluations. The image highlights GLM-5.3 leading or being highly competitive on benchmarks such as AutomationBench, CyberGym, and GDPVal-AA v2, while other models like GPT-5.6 Sol or Mythos/Fable 5 remain ahead on some tasks such as DeepSWE and ExploitBench. Commenters mostly framed this as another rapid Chinese model release; one noted that although this appears to be an API-model announcement, discussion is still relevant because the team has reportedly said weights will be forthcoming.
A commenter notes that GLM-5.3 is currently being discussed as an API model release rather than an immediate weights release, but argues it is still relevant to the local/open-model community because the team has reportedly said weights are forthcoming. This frames the release as potentially important for future self-hosting or benchmarking once checkpoints are available.
One technical takeaway highlighted from the release wording is: “Scaling post-training is all we did for GLM-5.3.” Commenters interpreted this as notable because it suggests the improvement may come primarily from larger or more intensive post-training/RL/instruction-tuning rather than a new base architecture or pretraining run.
DeepSeek: We’re launching DeepSeek-V4-Pro today! (Activity: 729): DeepSeek announced DeepSeek-V4-Pro on X (post), and commenters note that model weights have been released on Hugging Face as deepseek-ai/DeepSeek-V4-Pro-0813. A top technical comment highlights new API pricing via an attached pricing image, implying a significant price increase relative to prior DeepSeek offerings. Commenters argue the price hike weakens DeepSeek’s main advantage: despite being “token hungry and a little slower,” it was previously attractive because it was cheap; at higher API prices, some users say they will return to local inference.
DeepSeek-V4-Pro weights are reported as released on Hugging Face at deepseek-ai/DeepSeek-V4-Pro-0813, shifting some discussion from API economics to self-hosting feasibility. Commenters argue that if the model’s performance is competitive and infra/electricity costs work out, open weights could let third-party providers undercut the official API.
Several commenters focused on the API pricing increase, saying DeepSeek’s prior appeal depended on being very cheap despite being “token hungry” and somewhat slower. The concern is that higher token pricing makes the hosted API less attractive versus local inference or alternative providers.
One early user disputed DeepSeek’s claimed parity with Kimi 3, saying V4-Pro does not match Kimi’s “knowledge / long term ability to work on a project hands off.” The criticism is specifically about extended autonomous project work and retained task context, not just short benchmark-style outputs.
It’s actually crazy how good DSv4 Flash 0731 is (Activity: 556): The image is an Artificial Analysis Intelligence Index bar chart showing DeepSeek V4 Flash 0731 max scoring 52, ranked 46/608, effectively clustered with top frontier models like GPT-5.6 Terra and GLM-5.2 at 53. The post highlights the practical significance: a model near the top of the benchmark table is reportedly usable on a sub-$2k local machine, making it notable for local/offline inference relative to larger frontier APIs. Commenters pushed back that the benchmark may overstate real-world capability: one user said GLM 5.2 remains much stronger for programming and that DeepSeek wastes tokens on complex tasks. Others argued Qwen 3.6 27B is even more impressive due to similar ranking at roughly 1/5 the size, while another said DSv4 Flash is the first locally runnable model that does not feel like a downgrade from frontier models.
Several users challenged the headline benchmark implication for DeepSeek V4 Flash 0731, arguing that real coding performance can lag chart results. One commenter reported spending >$100 in API credits and said that on complex programming tasks it often “wastes a ton of tokens doing useless investigations” and may fail to converge, while GLM 5.2 was described as still clearly stronger for programming.
A notable comparison was raised with Qwen 3.6 27B, which commenters said appears close to DeepSeek V4 Flash on the referenced chart despite being roughly 1/5 the size. The technical implication discussed is that Qwen may offer a better parameter-efficiency tradeoff if the benchmark placement reflects real workload performance.
One user highlighted local usability: DSv4 Flash 0731 was described as the first locally runnable model they had used that “doesn’t feel like a downgrade from frontier models”, becoming their default workhorse for home projects. Another commenter criticized the benchmark chart methodology, noting it showed “Selected 46 of 608 models” and questioning whether the comparison set was cherry-picked or unrepresentative.
Deepseek Harness is Up! (Activity: 537): DeepSeek AI announced DeepSeek Harness (dsh), an open-source agent harness in developer preview, built around an “everything is a plugin” architecture and powered by Cordis, whose design is described in A Programming Paradigm for Spatiotemporal Composability. The project is explicitly unstable—“THERE WILL BE COMPATIBILITY-BREAKING CHANGES”—and DeepSeek is directing developers to its Discord community for updates and discussion.** Top comments focused on ecosystem skepticism: one user questioned why agent harnesses are so often written in TypeScript, another suspected bot-driven GitHub growth after reported stars jumped from 20k to 30k in about an hour, and a third asked whether dsh can achieve better cache hit rates than reasonix.
Commenters pointed to the official DeepSeek Harness repository and docs: github.com/deepseek-ai/deepseek-harness and deepseek.com/harness/en. One technical concern was whether it can achieve higher prompt/cache hit rates than Reasonix, since cache efficiency is increasingly important for inference cost and latency.
A commenter questioned why many agent/harness implementations are written in TypeScript, contrasting this with Codex as a possible exception. The concern implies friction for lower-level performance tuning or integration compared with Python/Rust/native tooling, though no benchmarks or implementation details were provided.
3. Specialized Local Transformer Builds
Trained a 1.5B to write shell commands so I’d stop googling tar flags. Runs on a laptop CPU in ~1 sec. (Activity: 1815): The image is a terminal/CLI demo splash screen for the whatisit tool, showing ASCII art in a dark terminal rather than benchmark output or model internals: image/GIF. Context from the post is technical: the author fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language→shell-command pairs, quantized it to Q4_K_M (941MB) for llama.cpp, and reports CPU performance of 31.9 tok/s, 0.59s median/query, 1.6GB RAM, plus 0.620 on InterCode-ALFA vs 0.613 for untuned Qwen2.5-Coder-7B and 0.73 for GPT-4o. The released artifacts are Apache-2.0 weights on Hugging Face and code on GitHub, with a static safety checker because the model can generate destructive shell commands if prompted. Comments were mostly lighthearted rather than deeply technical: users joked that this is “lots of effort to not use man pages,” offered mnemonic tar flags like -czvf / -xzvf, and warned that an NL-to-shell model is potentially dangerous—“like giving a loaded T34 tank to an infant.”
A commenter asks whether the author evaluated Gemma Shellper, a smaller shell-command-focused model reportedly under 0.5B parameters, as a baseline or alternative. The comparison is technically relevant because the post’s model is 1.5B and targets ~1 sec CPU inference on a laptop, so latency/accuracy tradeoffs versus a much smaller model would be useful.
Doom running on an LLM -- Hugging Face checkpoint included (Activity: 347): The author compiled Doom’s deterministic renderer—not trained it—into a stock Phi3ForCausalLM checkpoint using torchwright, with all weights computed analytically and loadable via vanilla transformers with trust_remote_code=False (write-up, source). The prompt encodes level geometry/player pose/view direction and generation emits drawing commands consumed by a 43-line raster host; the 320x200 model is 21B params / 85.87 GB, requiring 3,614 prompt tokens + 53,747 generated tokens per frame and taking just under 40 min on a B200, while the practical 80x50 checkpoint is a 34 GB download (80x50 weights, 320x200 weights). The current compiler requires fp32 weights; the author has only run it on cloud B200/A100-80 GPUs and recommends 80 GB VRAM for the 80x50 model, with 64 GB possibly sufficient but untested. The main technical pushback is that 53,747 tokens in ~40 min on a B200 for a 21B model seems far slower than expected—one commenter claims dual RTX 3080s can generate a similar token count on 27B within 30 min, suggesting a serious optimization issue. Another commenter asks why the project targets an LLM/text-generation architecture rather than a transformer image generator, i.e. whether the choice is purely for the “Can it run DOOM?” novelty or has a technical rationale.
A commenter questioned the reported inference performance: “One frame is a 3,614-token prompt plus 53,747 generated tokens -- just under 40 minutes on a B200” for a 21B model, arguing this is far slower than expected and may indicate a broken/unoptimized generation path. They compared it to their own setup claiming a pair of RTX 3080s can generate a similar token count on a 27B model in under 30 minutes, despite being much weaker than an NVIDIA B200.
The same commenter asked why the project uses a stock Phi3ForCausalLM LLM architecture—where the prompt encodes level geometry/player pose/view direction and generation emits drawing commands consumed by a 43-line host renderer—instead of a transformer-based image-generation approach, questioning whether the choice was purely for novelty or had a technical rationale.
Gemini 3.7 Flash Benchmarks (Activity: 1182): A Reddit post titled “Gemini 3.7 Flash Benchmarks” discusses benchmark results for Google Gemini 3.7 Flash, but the provided excerpt does not include the actual benchmark table, metrics, tasks, or methodology. Commenters characterize the results as unusually strong for a low-latency/cost-optimized “Flash” model, with one calling it “amazing for a flash model.” The main debate is benchmark relevance: one commenter argues that “97% of flash users” care more about practical qualities like creative writing, emotional intelligence, web search, and hallucination behavior than leaderboard-style scores. Gemini Flash is framed as a strong value model, especially compared with perceived cost increases from DeepSeek.
Commenters interpreted the posted Gemini 3.7 Flash benchmark results as unusually strong for a “Flash”/low-cost model tier, with one comparing its apparent performance favorably against Sonnet 5. No concrete benchmark numbers were discussed in the comments, but the theme was that the model may be closing the gap with higher-end competitors while remaining a value-oriented option.
One technical critique was that standard benchmark suites may not reflect the majority of Flash usage patterns: a commenter argued that “97% of flash users” care more about creative writing, emotional intelligence, web search quality, and hallucination rate than leaderboard-style scores. They still characterized Flash as potentially the best bang-for-buck LLM, implying cost/performance and real-world reliability matter more than raw benchmark wins.
Holy... Google actually did it, they actually shipped a frontier model (Activity: 1123): The post reports hands-on testing of Google Gemini 3.7 Flash, characterizing it as a very fast “workhorse” model with strong instruction-following and no observed hallucinations in the author’s tests. A notable anomaly was one run where the model began reasoning in Chinese while still completing the task correctly, suggesting a possible language-routing or hidden-chain-of-thought leakage issue. Commenters broadly push back on prior anti-Gemini sentiment: one says it is “much better” in Antigravity, while another argues it is not truly frontier-level but closer to a Claude Sonnet-class everyday model used for ~80% of tasks, with expectations that Gemini 4 may be frontier-level.
One commenter reports hands-on testing in Google Antigravity, saying the new Gemini model is “much better” in that coding-agent environment, though no concrete benchmark numbers or failure cases were provided.
A more technical framing compares the model to Claude Sonnet-class systems rather than an absolute frontier leader: it is described as a likely 80% of usage “workhorse” model, with speculation that Gemini 4 may be the model that reaches clear frontier status.
2. Claude Code Agent Memory and Orchestration
Example of a real working loop orchestrator (Activity: 1567): The image (PNG) shows a non-meme, working AI loop orchestrator dashboard (“Llyod’s Mission”) used to manage recurring agent sessions and a SQLite-backed internal ticket/memory system. The setup centers on a configurable heartbeat / pulse loop that runs playbooks such as checking inbound bug-report emails, querying prior tickets, inspecting app logs, updating docs, and spawning/monitoring child sessions with visible status, model, progress, cost, and deployment actions like Create PR, Commit & Push, Worktree, and Release Notes. The technical significance is that the orchestrator treats agent memory as an operational database—effectively an internal Jira/tribal-knowledge store with 600+ tickets—so new tasks can be grounded in previous context across models. Commenters generally viewed the setup as a useful concrete example of agent infrastructure beyond a chat UI, especially for email triage and business workflows. One commenter echoed the same pattern—local history tables for client email context—while another said it clarified how to build harnesses, managers, and dashboards around Claude/agent workflows.
One commenter described a production-ish inbound email orchestrator that uses a local table of historical client email exchanges as persistent context. When a new email arrives from a known client, agents can inspect prior issue history without the user manually injecting context, effectively turning the loop into a lightweight client-support memory/RAG workflow.
Another commenter outlined a more complex always-on architecture: three 24/7 Claude agents on separate machines, each owning a domain and able to spawn subagents across multiple providers/models. They coordinate through a shared main ticket table, plus per-agent Kanban boards used to delegate specialized tasks to subagents based on occupation, task type, provider, and model.
The same setup includes a hierarchy where one orchestrator owns the global ticket queue but can escalate or route work to other orchestrators when a task falls under their domain. Human interaction is mediated through a voice-controlled “Hermes” agent on a phone, which can assign tickets, relay messages, and provide status updates.
I make Claude Code keep a MISTAKES.md file. Here’s what actually happened. (Activity: 1089): The post describes a lightweight persistent-memory workflow for Claude Code: add MISTAKES.md to the repo and instruct CLAUDE.md to append failures with what happened / root cause / consequence / prevention, newest-first. The author reports that Claude later references this file to avoid repeated errors, and recurring entries are promoted into enforceable CLAUDE.md rules, turning anecdotal “flaky area” memory into countable failure patterns and guardrails. Commenters report similar regressions where Claude repeats known mistakes or prematurely stops despite instructions, with one user quoting Claude admitting it “ignored” prior guidance and caused the same issue again. Another commenter extends the idea with hook-triggered “skills” after specs, plans, and implementations to scan past errors against current work, claiming it catches many issues.
Several commenters reported that Claude Code repeatedly makes the same implementation errors unless prior mistakes are operationalized as part of the workflow. One user described Claude explicitly acknowledging it had previously avoided a broken approach on a given date, then “ignored this though and caused exactly the same problem again,” suggesting that passive documentation like MISTAKES.md is insufficient without retrieval or enforcement.
A more technical pattern was described: adding a secondary workflow layer using Claude Code skills + hooks that run after every spec, plan, and implementation step to scan past errors and compare them against the current work. The commenter said this has “caught so many fuck ups,” implying the useful mechanism is not the mistakes file itself but automated post-step validation against it.
There was debate over retrieval strategy: one commenter argued that merely referencing MISTAKES.md will not reliably trigger Claude to consult it, while forcing the whole file into context is inefficient. They suggested Claude’s memories system should be superior because short recall triggers remain in context automatically; another commenter emphasized that without enforceable checks, “it effectively doesn’t exist and will always be ignored by the LLM eventually,” showing an implementation screenshot: https://preview.redd.it/prj0dddf05jh1.png?width=3400&format=png&auto=webp&s=b4164b5a6ffad94c85eee175907cbd45d1efd0db
3. AI Platform Pricing and Watermarking Shifts
DeepSeek just massively increased their API prices (effective August 16, 2026) - up to 1,114% increase for cache hits (Activity: 2009): DeepSeek is updating its API pricing effective 16:00 UTC, August 16, 2026, adding peak/off-peak billing where peak windows (01:00–04:00 and 06:00–10:00 UTC) cost 2× off-peak. The largest increases are on cached-input tokens: V4-Pro cache hits rise from $0.003625 to $0.022/$0.044 per M tokens off-peak/peak, i.e. +507%/+1,114%; V4-Flash cache hits rise from $0.0028 to $0.007/$0.014, i.e. +150%/+400%. Cache-miss input and output pricing also increases substantially, with V4-Pro output moving from $0.87 to $1.98/$3.96 and V4-Flash output from $0.28 to $0.66/$1.32. Comment sentiment is negative but technically thin: users suggest DS4 remains attractive mainly when cheap, and at least one commenter says they have already shifted workloads away. The main implied operational concern is that cached-context-heavy and long-conversation workloads lose much of DeepSeek’s prior cost advantage, especially during peak UTC windows.
One commenter notes they have already migrated away from DeepSeek, saying DS4 is only attractive “when cheap”—implying the price increase may erase its main advantage versus competing API models unless its quality/performance justifies the new rate.
A user in Brazil points out that DeepSeek’s off-peak pricing window may align unusually well with their local daytime usage: “off peak hours: 7:00 > 22:00”. This suggests regional timezone effects could materially change the real-world impact of the price hike for latency-tolerant workloads that can be scheduled into discounted windows.
Some Claude users are mad that Anthropic’s new watermarks will catch them using it at their jobs, classes (Activity: 1160): The post discusses user backlash to Anthropic adding detectable watermarks/provenance signals to Claude outputs, with concerns that these markers could reveal AI use in workplaces or classes where disclosure may be penalized. A technical edge case raised in comments is that Claude used for proofreading/editing may cause otherwise human-authored text to be flagged as AI-associated, blurring attribution between generation and assisted revision. Commenters were split: one said workplace AI use is encouraged and a watermark would be “affirmation,” while another worried detectors would mislabel their own edited writing as “AI slop.” A separate comment criticized Yahoo for turning a Reddit thread into news, but it added little technical substance.
A commenter with education-sector experience argues that Anthropic-style watermarking is technically weak as an enforcement mechanism because open-weight models are not subject to the same watermarking constraints. They note a likely laundering workflow: use Claude for most generation, then pass the output through an open-weight model to paraphrase and potentially remove or obscure the watermark.
Several comments highlight a boundary problem: if Claude is used for editing, proofreading, formatting dictated text, or restructuring notes, watermarking may label a largely human-authored artifact as AI-generated. The concern is that detectors could conflate legitimate assistive use with full synthetic authorship, creating false accusations in workplaces or schools.
The education-focused comment warns that even improved statistical watermarking can reproduce problems seen with AI detectors: false positives and inequitable enforcement, especially for non-native English speakers or neurodivergent writers whose syntax may appear formulaic. The commenter recommends designing assessments that measure comprehension and AI literacy rather than relying on detection as a blunt academic-integrity tool.
Agent frameworks for developers are still at an early stage, with the likes of Vercel’s eve and Fred Schott’s Flue — both launched this year — setting the early template.
Schott is the creator of the web framework Astro, which led to his company being acquired by Cloudflare in January. He’s just released version 2 of Flue, its first stable release, which has as its foundation React-style “Agent Hooks.”
In Flue, an agent is represented by a JavaScript function. This function “re-renders on every turn,” meaning before every model call.
The addition of hooks came after Schott realized that React’s composability would be a great fit for agent development.
“I originally tweeted that we were building the Astro for agents or the Next.js for agents,” he told us. “But then I realized: maybe no one has even built the React for agents.”
“We’re still trying to figure out who the reactive agents are and the jury is still out… We’re sort of in the jQuery era of agents, not the react era.”
Hooks are authored in TypeScript. According to the Flue 2 launch post, they “let you build dynamic agents that can manage their own state, listen to agent lifecycle events, and even attach different resources and capabilities dynamically to enhance themselves at runtime.”
There are 16 built-in hooks in Flue 2, including useSkill(), useTool(), useSubagent(). You can also add custom hooks.
How Flue evolved via React-style hooks; diagram by Richard MacManus
What hooks open up for developers is that they make an agent much more dynamic, by allowing its configuration to change as a conversation or workflow progresses. Schott said this is needed to build “real support bots, real triage bots,” because they can’t be fully configured in advance. The agent can’t just be static — it has to adapt in real-time to what the user wants or the situation demands.
Agent hooks bring those capabilities to Flue. For example, a support agent might bring in an account management tool after first verifying a user.
File based magic is an antipattern
Schott’s thinking about how to build an agent framework has evolved rapidly since he publicly launched Flue 1 in early May. Initially, he wanted to take existing web framework concepts and apply them to his new agent framework. He uses file-based routing as an example.
“So we kind of naively ported that over to Flue, thinking — great, well, I’ll put your five agents in these five files, and that’ll be the five routes that they expose. But for a lot of people building with Flue, especially the bigger customers, their whole company is one agent. They don’t care about routing. There’s one agent.”
So after the first Flue users showed these early patterns, composability became front of mind for Schott. That led him back to React.
“As you can see from the Flue 2 API, we’re taking it more from React [...] than we are from Astro or Next.js — where it’s less about routing and these website concepts and more about, at its base level, how do you compose an agent on many different things?”
Flue’s central proposition: agents need a harness
A key concept in Flue is that an agent must have a harness — meaning that it’s in an environment where it has access to the context and capabilities needed to accomplish various tasks.
“Instead of you and your code driving the LLM and telling it what to do with scripts, you’re putting the agent into this harness, and it is able to drive itself and work through problems,” explained Schott.
Flue is built on top of Pi, an open source minimal harness. Essentially, Flue is an opinionated take on Pi — adding features that Schott thinks are helpful to developers building agents. For example: hosted agents in Flue 2 are now built with Vite, an open source build tool.
Indeed, Schott likens Pi’s role to the foundational role that Vite now plays beneath Astro.
“I think Pi can serve that role, where it’s the right abstraction — it doesn’t do too much, but it gives the right APIs that then we can go and say, well, let’s have an opinionated take on this that does more.”
Building on Pi meant committing to having a built-in agent harness.
“Our early bet was that the harness is actually not a feature, but it’s fundamental to what you think an agent is,” Schott said. “There is no agent without a harness.”
Building Flue agents with coding agents
The Flue project began earlier this year within the Astro repository, as an issue-triage system. At first, it was an LLM-driven script or workflow reviewing issues. But then, explained Schott, it gained the ability to take actions in the repo.
“It started to transition from just automation in a repo to wanting to take the Claude Code experience, make it headless, make it hostable and run it in the cloud.”
So that’s when the idea of a harness as anchor emerged. Indeed, in his v1 launch post in early May, Schott described Flue as “like Claude Code, but 100% headless and programmable.”
I myself tested out Flue using Claude Code, which guided me through setting up my first Flue agent. And Schott confirmed this is how many developers use Flue.
“We very much are building for them,” he said, regarding AI coding agents. “Our whole onboarding flow is that, you know, pass this prompt to your agent, it’s gonna guide you through it. All of our docs have markdown support.”
Where Flue fits in the agent development stack
The closest comparison to Flue is Vercel’s eve, which also treats the harness as foundational. Vercel and Cloudflare have been known to beef in public, but Schott is generous in his opinion of eve.
“Eve, I think, is the most directly competitive,” Schott said. “It came around at the same time, so it had that same take that a harness is built-in.”
Schott also referenced what he called the “OG agent frameworks,” which came before Flue and so weren’t created with a harness as the central concept. He listed Vercel’s AI SDK, Cloudflare’s Agents SDK, and Mastra (developed by the same team that built Gatsby, a web framework predating Astro).
While these “OG agent frameworks” are all adding harnesses now, Schott considers that an added feature — whereas Flue and eve both have built-in harnesses.
I asked where Flue sits compared to emerging “meta-harnesses,” like Databricks’ Omnigent and perhaps even the self-improving Exo harness.
Note: we’re also publishing our interview with Exo coauthor Alex Krentsel this weekend; it’s worth a watch and has a bonus discussion on OpenClaw architecture!
Schott rightly noted that there’s confusion about what the term meta-harness even means at this early stage. Regardless, he thinks having one API for working across all harnesses would muddle the story for Flue. His framework specifically defines how skills work in Flue, how subagents work, and so on. As he put it, “the framework [Flue] and the harness are very intertwined.”
He personally finds the meta-harness discussion fascinating, and has played with Exo, but says it’s “a different interest scenario that isn’t really related to hosted agents.”
The Cloudflare connection
Throughout the interview, Schott referenced being able to take advantage of his employer Cloudflare’s tooling and infrastructure. But he was also very clear that Flue is an “open source framework for every host,” as he put it, and he wants it to stay that way.
“The best tools are the ones that float above the host,” he said. “That opens the door for the most developer adoption and the most innovation.”
Host portability is one of Flue’s defining principles — and perhaps that’s where the fundamental difference to Vercel’s eve is. While eve can also be self-hosted, it is optimized to take advantage of Vercel’s many features. Of course that’s a known playbook of Vercel, which does the same thing with Next.js.
I also mentioned LangChain’s new Managed Deep Agents offering as an example of hosted agent platforms coming onto the market. However, Schott said a managed agents product is not currently on Flue’s roadmap.
“It’s so early for us, we’re just focused on building the best harness,” he said.
Links to find Flue and Fred online; Richard is at @ricmac. This is a new written interview series we are trying out for subscribers — let us know your feedback!
This is powered by their newest model, Grok 4.6, released today as arguably the second best knowledge work model in the world (as both competitor Cognition and Elon acknowledges)… though it is surely the top by efficiency:
Grok 4.6 is a confirmed 1.5T model that “builds on Grok 4.5 with a particular focus on long-running agents and more ambitious interactive and visual work”. The only training disclosure can be reproduced in full (emphasis ours):
Grok 4.6 underwent a longer supplemental training run than Grok 4.5, with curated model-generated data for reasoning and advanced technical concepts, high-quality engineering data, and an improved optimizer and training recipe. This produced a stronger foundation for the SFT and RL stages that followed.
We then used Grok 4.5 to regenerate the SFT trajectories across reasoning efforts, agent harnesses, and domains such as STEM, software engineering, and knowledge work, and filtered out problematic traces with model-based checks. The resulting SFT checkpoint shows strong performance and improved behavior.
Grok 4.6 is trained on a wide range of agentic RL tasks, including knowledge work, general coding, and domain-specific environments for kernel optimization, web development, computer-aided design, and more.
It is currently unclear if the lack of sandbox escape incidents when it came to training Grok 4.6 is a testament to their infra engineers or an indictment of their researchers.
(that is a joke about current events, don’t get mad)
Frontier Model Day: Grok 4.6, Qwen3.8-Max, DeepSeek V4 Pro, and Microsoft’s MAI-Thinking-1
Grok 4.6 reaches the frontier on price/performance: xAI released Grok 4.6, described as a major step up from 4.5 at the same price. Independent evaluations from Artificial Analysis place it at 61 on the Intelligence Index, roughly in line with GPT-5.6 Sol Max, behind Claude Opus/Fable, with strong agentic results including 88.4% on Terminal-Bench v2.1, 1753 GDPval-AA v2 Elo, and competitive AA-Briefcase performance at far lower cost (AA-Briefcase note). Early arena data from Code Arena also slots it near GPT-5.6 Sol and Claude Fable on webdev tasks. Pricing is a central theme: AA highlights $2/$6 per 1M input/output tokens, materially below frontier peers, while practitioners immediately framed it as the new default for coding and bug-finding workloads (Pawel Huryn, Cognition availability in Devin). xAI says the gains came from a longer supplemental training run, regenerated SFT traces, and agentic RL over coding, web, CAD, and kernel optimization; they also report more self-testing behavior during long tasks (@kimmonismus summary). Elon also said Grok 4.7 is already in flight, with initial training complete and supplemental training on SpaceX internal data planned.
Qwen3.8-Max open weights are out: Alibaba’s Qwen3.8-Max dropped as an open-weight 2.4T total / 95B active MoE. Community notes emphasize its scale, day-0 serving, and long-context/agent orientation: Yuchen Jin called it one of the largest open-weight releases to date; vLLM shipped day-0 support plus vendor-specific 4-bit checkpoints for NVIDIA B300 and AMD MI355X; Together AI and Baseten also announced immediate support. One important caveat from users: the released open-weights variant appears to be text-only, with no vision input in the initial drop (skalskip92).
DeepSeek V4 Pro GA undercuts the market: DeepSeek’s V4 Pro GA rollout immediately drew attention less for “best benchmark in every column” than for economics. Multiple observers highlighted pricing around $0.435/M input and $0.87/M output (kimmonismus), with Cline calling it roughly 57× cheaper than Fable 5 while reporting meaningful gains over the preview, including a 15.8% Terminal Bench increase. Reaction was mixed on capability: some early users found it solid but not clearly ahead of Kimi/Flash on all tasks (Yuchen Jin’s roundup, scaling01, teortaxesTex), suggesting DeepSeek’s next gains may depend more on RL environment and agent work than raw scale.
Microsoft enters with its own reasoning model: Mustafa Suleyman announced MAI-Thinking-1, Microsoft’s first reasoning model “built from scratch,” now available in Foundry. The initial ask from the team is notably practical—Finbarr Timbers specifically requested feedback on tool use—which suggests Microsoft is positioning it as an applied reasoning model rather than just a benchmark entrant.
Solar Pro 4 also moved up a tier: Artificial Analysis reported that Upstage’s Solar Pro 4 jumped from 14 to 42 on the Intelligence Index, with especially large gains on agentic and long-context tasks, though still behind the current top frontier and open leaders on both raw score and price.
Open-Weight Multimodal and Edge Models: Video, Vision, Voice, and Local Inference
LTX-2.5 and the open video stack keep improving: @RisingSayak highlighted that Lightricks’ LTX-2.5 landed in Diffusers with several practical features that matter for local workflows: joint video + 48 kHz audio generation, prompt-controlled clip length, a 2-pass quality mode, tile rendering for lower memory usage, and preprocessing that re-compresses input images to better match training. Ostris AI Toolkit added support the same day. More broadly, several accounts framed the week as an unusually strong run for open multimedia releases, including MiniMax H3, LTX-2.5, LFM2.5-VL-3B, and North Micro Vision (victormustar, multimodalart).
Small VLMs and local multimodal are getting serious: Cohere launched North Micro Vision, an Apache-2.0 open-source small VLM aimed at document understanding, with claims of outperforming Gemma 4 E2B and Ministral 3 3B on a broad visual benchmark mix (results thread). Liquid AI’s LFM2.5-VL-3B was also repeatedly cited as a strong compact vision model, and users demonstrated hybrid local/remote agent stacks—for example, Hermes Agent using DeepSeek V4 Flash for planning plus LFM2.5-VL-3B for local vision.
Speech and sign-language releases were unusually substantive: Google DeepMind announced SL2T, a sign-language-to-text system powering ASL input on Android/Pixel 11. The follow-up notes are technically interesting: body pose tracking happens on-device, translation runs server-side, and the system is optimized for real-world constraints like one-handed signing (detail). Separately, Deepgram launched Flux TTS, a low-latency conversational TTS model claiming ~80 ms response time and mid-call adaptation for voice agents.
Inference, Compression, and Systems: vLLM, Quantization, CUDA Scheduling, and Ranking Infra
vLLM added important infra for giant models and long prompts: vLLM now supports Azure Blob paths for both model loading and KV connectors. The Microsoft/NVIDIA recipe matters operationally: faster weight loading via Dynamo ModelExpress (up to 7.3× faster on H100/A100) and blob-backed KV caching via LMCache + NIXL, trading recomputation for fetches on long-prompt workloads (follow-up).
Compression work is extending the useful life of very large models: LLM Compressor v0.13.0 added REAP expert pruning for MoE models—dropping whole experts based on calibration saliency before quantization—as well as arbitrary 3/5/6/7-bit quantization. On the more extreme end, Unsloth claimed to shrink Qwen3.8-2.4T-A95B from 4.9 TB to 397 GB via dynamic 1-bit quantization, making local execution conceivable on 410 GB+ RAM/VRAM systems. They also showed a 2-bit Nemotron 3.5 Lightning setup sustaining long tool-use sessions in 22 GB VRAM.
GPU kernel authoring is getting safer and more declarative: maharshii highlighted CuTeDSL 4.7.0 Task Scheduling kernels, which let developers explicitly declare warp roles, resources, dependencies, and schedules, enabling static checks for deadlocks, races, and barrier initialization before lowering to GPU code. The same author also posted a concise explainer on the prerequisites behind TMA async copy—acquire/release semantics, mbarriers, and CuTe arithmetic tuples—for people trying to reason about modern NVIDIA memory movement primitives (thread).
Classic recommender/ranking stacks are still quietly delivering wins: François Chollet pointed to Expedia’s migration to a modern Keras 3 setup, reporting 30% faster training and 70% lower inference latency for ranking models (tweet). His follow-up stresses a more strategic point: Keras’s backend-agnostic APIs reduce lock-in if teams later need PyTorch or JAX kernels (note).
Agents, Harnesses, and Developer Tooling: Reliability, Memory, Plugins, and Security
The stack above the model is becoming the main product surface: Several tweets converged on the same theme: many practical gains are coming from harness engineering, memory, approvals, evals, and tools more than bespoke model training. Scott Stevenson restated the argument that RAG and harness engineering beat training most of the time because they personalize per customer, avoid privacy risks, improve in real time, and inherit base-model progress (thread, follow-up). Random Walker added a useful product distinction between delegation agents and collaboration agents, with very different optimization targets around verifiability, latency, and human control (tweet).
Tooling releases reflected that shift: GitHub’s @code introduced Agent Plugins 1.0, packaging skills, MCP servers, and AI extensions together, and separately shipped UX improvements like sticky scroll and better session handling (release thread). OpenAI/Codex-side momentum showed up too, including Codex for Linux. LangChain rebuilt LangSmith dashboards for more useful trace analysis and reporting.
Memory and portable agent state are becoming baseline expectations: Hermes Agent got multiple ecosystem updates, from Raspberry Pi deployment to easy profile export/import and new skills like generating reusable APIs from observed web traffic (Teknium). Managed Deep Agents examples from LangChain focused explicitly on durable memory and recurring workflows such as social-media agents (hwchase17).
Security and governance for agents is becoming concrete: W&B showed a side-by-side agent email example where one agent leaked SSN/card info while another blocked prompt injection and redacted secrets before the model saw them (thread start). The Turing Post raised a more architectural issue around delegated identity: if an agent uses your SaaS credentials directly, revocation and auditing become muddy (tweet).
Benchmarks, Research Directions, and AI-for-Science
AI-assisted math and science claims are getting harder to ignore: The most engaged technical tweet was Steven Strogatz sharing a story that a neurosurgery resident reportedly used ChatGPT 5.6 to solve a significant open problem in numerical linear algebra (tweet). Relatedly, multiple accounts noted another EpochAI open problem apparently falling (scaling01).
New benchmarks target less gamed capabilities: Princeton/MIT collaborators released DiG-bench, a text-based benchmark for discovery rather than standard QA or code tasks; tri Dao specifically praised it for having some of ARC’s flavor without confounding vision issues (tweet). Redwood + Anthropic introduced the Conceptual Reasoning Index, targeting AI-risk-relevant argumentation and conceptual reasoning where feedback is sparse and hard to automate. Vals announced SRE-Bench, focused on binary reverse engineering rather than source-level cyber tasks.
Post-training efficiency and long-context research stood out: Lewis Tunstall summarized Direct On-Policy Distillation, where RL is done on a smaller model and the resulting policy shift is transferred to a larger model using a dense implicit reward, roughly halving pipeline cost in the cited setup. Separately, dair.ai’s summary of new OLMo/Llama/Qwen long-context work argues that four architecture choices—normalization, GQA, pretraining context length, and sliding-window attention—can together cost up to 47% of long-context performance, even when short-context validation looks fine.
Clinical and domain-specific RL is maturing: A thread summarizing Google’s ResidencyRL work reports that training Gemini 3.5 Flash over 49,870 simulated telehealth encounters increased diagnostic accuracy under adversarial conditions from 81% to 88% and reduced missed red flags by 31% (kimmonismus). Snowflake also shared a good counterexample to “bigger always wins”: a new 4B SQL autocomplete model beat their previous 30B-A3B MoE, improving user acceptance while cutting median latency 71%.
AI-for-math headline: @stevenstrogatz shared the numerical linear algebra story involving ChatGPT 5.6.
Accessibility milestone: @GoogleDeepMind announced SL2T for ASL-to-English input on Android.
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Claude Text Watermarking Rollout
Claude now embeds invisible watermarks in all text outputs + signed metadata on files (Activity: 2077): Anthropic says Claude marks some AI-generated/edited content via metadata/provenance signals, not a visible text watermark; the mechanism and persistence depend on file type/workflow and may be lost after editing, export, or platform handling (support article). For plain text, commenters question whether this implies statistical linguistic watermarking versus attached metadata; based on Anthropic’s description, the robust claim is metadata/provenance marking, not an undeletable watermark embedded in arbitrary copied text. Commenters are skeptical of usefulness for text because paraphrasing through another model or local LLM could likely remove detectable signals, and some view any Claude-linkable marking as a privacy/control reason to prefer open-source models.
Anthropic/Claude rollout details: commenters cite the submission statement that Claude models launched on or after August 2, 2026 will embed an imperceptible model-level text watermark intended to survive copy-paste and some editing without changing readability or semantics. Supported file outputs such as .png, .jpg, and .svg will also carry digitally signed C2PA provenance metadata, with third-party detection tooling still forthcoming and older models expected to be updated during a transition period.
A technical concern raised is robustness: for text, users argue the watermark may be removable by paraphrasing through another model, especially a local/open-source one, because rewording can destroy token-level statistical patterns. Another commenter notes this is not unique to Anthropic and points to OpenAI’s provenance/watermarking work: Understanding the source of what we see and hear online.
How would an “invisible watermark” in AI-generated text actually work? (Activity: 878): The thread asks how an invisible text watermark could be embedded in Claude-style LLM output without hidden Unicode; the technical answer is a keyed generation-time scheme that slightly biases token sampling toward pseudo-randomly selected “favored” tokens based on prior context and a secret key, then detects overrepresentation via a statistical score such as a z-score. Commenters note this is robust to copy/paste and minor edits, but degrades under substantial paraphrasing, sentence restructuring, or regeneration by another LLM; Google’s SynthID-Text approach, described in Nature, uses a related tournament-sampling watermarking method. The main skepticism is epistemic: “how would anyone know if it was watermarked?”—i.e., detection depends on access to the secret rule/key or a trusted detector, and robustness claims are limited once the text is heavily rewritten.
A commenter describes LLM text watermarking as a keyed sampling bias: during next-token generation, the model slightly boosts a secret, context-dependent subset of tokens, producing a hidden statistical pattern while preserving fluency. Detection then recomputes the same secret rule over the text and checks whether favored tokens occur above chance, often via a z-score-like statistic; copy/paste and light edits may preserve the signal, while heavy paraphrasing can destroy it.
One linked technical reference is the Nature paper “Scalable watermarking for identifying large language model outputs”, which is relevant to production-grade schemes such as Gemini-style tournament sampling. The discussion also notes that implementations vary by provider, with claims that watermarking can be added at the sampling/model-output layer rather than requiring visible text markers.
A key unresolved technical concern raised is false positives: if detection is purely statistical, naturally written text could coincidentally overuse the “green-list” or favored tokens. This implies practical detectors need calibrated thresholds, long-enough samples, and measured false-positive/false-negative tradeoffs rather than treating watermark detection as a deterministic yes/no signal.
2. Frontier Model Security and Governance Flashpoints
It’s not very often that a paper breaks through to become headline story of the day. For understandable reasons both domestic and foreign, there is renewed interest in the Interpretability Venn Diagram of alignment, security, and chain of thought monitoring, so today’s paper could not have come at a better time:
Since the o1 launch, frontier lab reasoning models have obscured their traces, with cryptographic signatures, for fear of distillation (not that this prevented anyone from Chinese labs accusing them of doing so). The first compromise was responsibly reported by Matthew Green in May, who broke down how it works and figured out how to replay and side channel these indirectly using latency measures. Today’s paper demonstrates that it is possible to DECODE and port these encrypted thoughts to different models/sessions/users… and to dramatically improve open models as a result
The alarming note is here:
“Further, if you ever shared online a Claude Code/Codex session with encrypted reasoning blobs, they can be decoded and leak your personal data.
We did a preliminary scan of ~7,000 public traces and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data.”
(64 appeared exclusively inside the reasoning blocks and nowhere in the visible session.)
Obtain a legitimate encrypted/signed reasoning block from an API response.
Replay that block into a different request—potentially another account/session—to a weaker model from the same provider.
Place it in an assistant/model turn and prompt or prefill the weaker model to transcribe the attached reasoning.
Sample repeatedly, discard refusals, and optionally reconcile multiple noisy transcriptions.
The paper gives concrete templates with some minor variations per model:
Claude: replay the signed thinking block to Haiku 4.5, followed by an assistant prefill such as <thinking-copy>.
GPT: inject the encrypted_content reasoning item multiple times into a fabricated conversation; sample up to 50 outputs. It also describes bypassing an apparent ~50-token verbatim-output threshold using chunked continuations.
Gemini: attach thought_signature to a model turn with a <thought> prefill, then use repeated sampling and reconciliation.
This paper was responsibly disclosed, with several vulnerabilities already fixed, but surely similar attacks still seem possible.
Reasoning-Trace Exposure, CoT Privacy, and Watermarking Debate
Frontier API vulnerability exposed hidden reasoning: A widely discussed disclosure from @kotekjedi_ml claims a vulnerability across frontier APIs allowed extraction of “encrypted” hidden reasoning, with recovered token counts matching billed thinking tokens 1:1 on most queried prompts. In a follow-up, the team reports that a scan of ~7,000 public traces found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data in decoded blobs @kotekjedi_ml. Additional context from @jonasgeiping emphasizes both the immediate privacy risk of sharing traces publicly and the operational-security implications: during the investigation, they reportedly encountered a leaked Hugging Face prod key during the broader cyber incident. Several posts also highlight how difficult monitoring becomes when decoded CoT is terse, fragmented, multilingual, or effectively “neuralese” @jonasgeiping, @scaling01, @eliebakouch. A practical corollary: even if labs hide reasoning, tool interfaces may re-expose it; @_can1357 notes that disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.
What this means technically: Discussion split between “serious privacy/safety problem” and “not a scalable distillation path.” @vipulved argues the attack does not imply practical mass theft of chain-of-thought for model training, framing the encryption more as a stateless distributed-inference protocol optimization than a hard confidentiality barrier. Still, the episode sharpens a few points: public trace sharing is risky; hidden CoT is not a reliable monitoring interface; and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces @BlackHC. In parallel, a separate thread debated AI text watermarking under EU-style compliance pressure. @trq212 said labs are adding watermarking and a text-detection API; critics questioned whether this could bloat outputs or harm code/doc brevity @wightmanr. Others argued the entropy budget is large enough that signatures can be subtle, especially for longer outputs @RyanGreenblatt, @giffmana.
NVIDIA Nemotron 3.5 Lightning and the Small Open Agent Model Push
Nemotron 3.5 Lightning: NVIDIA released Nemotron 3.5 Lightning, a 30B MoE model with roughly 3B active parameters, positioned for always-on agent workloads. NVIDIA and ecosystem posts stress up to 4× throughput, 1M context, open/customizable release artifacts, and support for weights, data, and recipes on Hugging Face @NVIDIAAI. Artificial Analysis provides the most detailed third-party summary: 31.6B total / 3.6B active, OpenMDW-1.1 license, NVFP4 and BF16 weights, median serving near 670 tok/s in pre-release endpoint testing, and a score of 24 on its Intelligence Index—roughly in line with gpt-oss-120b while being much smaller and faster @ArtificialAnlys. Agentic results look particularly strong for the size: GDPval-AA v2 Elo 824 and Terminal-Bench v2.1 24%, both major jumps over Nemotron 3 Nano @ArtificialAnlys.
Distribution and downstream tuning: Lightning shipped fast across the stack: Together AI, Ollama, Baseten, vLLM, Perplexity API, and others. A recurring pattern is pairing a cheaper execution model with a stronger planner: @kimmonismus frames Lightning as NVIDIA’s “local agent workforce,” complementing larger planning models via routing. Harvey reports post-training on Legal Agent Bench improved Lightning from 0% to 8.3% on held-out tasks, beating Opus 4.6 and Nemotron 3 Ultra in that setup while cutting average output from 90k to 37k tokens @harvey. Overall, this release reinforces the current open-model trend: smaller, faster models tuned for high-volume tool use rather than general chat prestige.
Local AI Tooling: Unsloth Desktop, Muse Glimmer Support, and Linux Codex
Unsloth Desktop expands the local stack: @UnslothAI launched Unsloth Desktop, an open-source desktop app for running and training models locally across Mac, Windows, and Linux, with support spanning MLX, GGUF, diffusion image/video, audio, CPU and multi-GPU setups, plus OpenAI-compatible APIs. The notable systems angle is ambition beyond “local chat UI”: tool calling, sandboxed code execution, private search, RAG, MCP, exports, and claims of 2× faster training with 70% less VRAM. Multiple observers positioned it as a more end-to-end local AI operating environment rather than just an LM Studio competitor @TeksEdge, @dessaigne.
Model/runtime support keeps improving: The open/local ecosystem also moved quickly on Meta Muse Glimmer 30B and Nemotron. @mervenoyann highlighted DFlash drafter support for Muse Glimmer in llama.cpp and Transformers, claiming 2–4× generation speedup at small memory cost, with simple llama serve instructions following shortly @mervenoyann. On the model-analysis side, @rasbt gave a useful architectural breakdown of Glimmer: a dense 30B multimodal reasoning model with hybrid local/global attention, extreme KV-cache efficiency (~52 KiB/token BF16 by his estimate), and a design closer to Gemma-family patterns than MoE competitors.
OpenAI finally shipped Linux desktop support: OpenAI announced the ChatGPT desktop app for Linux in preview @OpenAI, with support for Ubuntu 24.04/26.04, Debian 13, Fedora 43/44, x64 and ARM64 packages @OpenAIDevs. More importantly for existing agent users, the desktop app can now import/sync projects, chats, skills, and plugins from other agents into ChatGPT Work and Codex, including automatic updates @OpenAIDevs. This looks like an effort to reduce switching friction and make Codex/Desktop the integration hub rather than a fresh silo.
Agent Products, Benchmarks, and Enterprise Evaluation
Grok Bot is a stronger product signal than another model launch: xAI introduced Grok Bot, pitched as AI teammates with their own cloud computers that can sign into tools and do persistent work. The interesting details from early users are product/ops-oriented rather than model-centric: bots can watch Slack threads and GitHub Actions, repeat scheduled routines, create/manage other bots, and work across linked cloud environments @shaoruu, @n2parko, @sjwhitmore. @kimmonismus notes how deeply this seems tied to Cursor distribution and pricing, hinting at a “virtual coworker” product category where persistent context, logged-in environments, and inter-agent delegation matter more than raw benchmark gains.
Evaluation is shifting toward long-horizon, deterministic, domain-real tasks: LlamaIndex launched ExtractBench, a deterministic benchmark for enterprise document extraction across 370 documents / 4,869 pages / 67 doc types. Its most actionable result is that commercial VLMs can keep precision high while recall collapses below 35% on documents >50 pages, mainly via silent row/list truncation. They also introduced an “Agentic Plus” extraction tier in LlamaParse claiming 95.6% value accuracy at less than one-third the cost of the nearest peer. Artificial Analysis released AA-AnalystAgent, an agentic benchmark for spreadsheet/document quantitative analysis using a pass^5 reliability metric across 80 tasks. Claude Opus 5 leads at 54%, followed by GPT-5.5 at 50% and Claude Fable 5 at 49%; Kimi K3 is the top open-weights model at 39%. The strong theme across both is reliability and workflow correctness over one-shot capability.
Benchmark skepticism is rising: A thoughtful critique from @hrishioa argues many modern evals are being “vibed” rather than engineered carefully, leading to broken scoring, bad aggregation, and even exploitable prompts/sandboxes. That critique lands harder given recent reports of sandbox escapes, outbound network access, and agent reward hacking. Separately, Microsoft research drew attention for a prompt-time “skill compilation” result: @xidulu shared work feeding the previous hidden state at decoding time for free gains, while @dair_ai summarized another paper showing that compact natural-language skills distilled from prior trajectories can recover 55% to >100% of the gap between non-reasoning and reasoning modes on several multi-step agentic tasks, often with 2.7–6× fewer output tokens.
Infra, Verification, and Systems Research
Verifiable inference is moving from theory toward product: @Yogi_Brn launched Attestable with a $20M seed, pitching practical zero-knowledge proofs for AI integrity. The core claim is proving that the correct model ran on the correct inputs and invoked the correct tools, which becomes more valuable as agent traces lengthen. @jaminball says the team reduced ZK overhead by many orders of magnitude from previously impractical levels. The response from @VitalikButerin is notable: he estimates the current approach may already be within single-digit (<10×) overhead relative to raw inference in some settings, and frames that as a stepping stone toward stronger privacy-preserving inference stacks.
Deterministic integer-only inference across hardware: One of the more technically interesting systems posts came from @nathanrs, who reports fully deterministic LLM inference across A100, H100, Apple M5 Max, AMD EPYC, and Intel Xeon by using exact integer arithmetic end-to-end instead of letting nonlinear ops bounce back into floating point. On a Qwen3-0.6B test, all integer runs produced identical hashed logits across devices, with WikiText2 perplexity 20.72 vs 20.95 for fp16 and 106 tok/s CUDA-graphed decode on A100 at batch 1—claimed as 3.6× fp16 eager baseline. If robust, that’s relevant both for reproducibility and for proof-friendly inference.
Compiler/inference portability as an agentic systems target: A smaller but recurring theme is “agents moving down the stack.” Posts around @JvNixon and Infinity describe automated compiler/memory-planner/debugger workflows for running optimized models across heterogeneous chips, with supporters framing software-generated per-chip adaptation as a way to weaken the CUDA moat. Separately, infra vendors shipped more incremental but practical updates: Qdrant 1.19 adds prefix matching on keyword indexes @qdrant_engine, and Together + IBM + NVIDIA announced enterprise inference infrastructure on IBM Cloud @togethercompute.
Top tweets (by engagement)
Reasoning trace vulnerability / hidden CoT extraction: the original disclosure from @kotekjedi_ml and the follow-up privacy findings @kotekjedi_ml were among the day’s most consequential technical posts.
Grok Bot beta: xAI’s agent product launch @bot drew the biggest product reaction, largely because it points to a persistent, logged-in AI coworker UX rather than a simple chatbot iteration.
ChatGPT desktop for Linux + sync/imports: OpenAI’s Linux desktop preview @OpenAI and agent-workflow import/sync support @OpenAIDevs landed strongly with developer audiences.
Nemotron 3.5 Lightning: Jensen’s post @JensenHuang and NVIDIA’s launch @NVIDIAAI marked the most important open-model systems release of the day.
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. Meta Muse Glimmer 30B Release and Local Benchmarks
Introducing Muse Glimmer: an open-weight model optimized for always-on local agent workflows (Activity: 2435): Meta announced Muse Glimmer, a permissively licensed Apache 2.0 open-weight 30B dense multimodal model for always-on local agent workflows, with interleaved text/image input via a dedicated perception encoder, 100+ language training, controllable reasoning effort, and agent-focused training for tool use, long-horizon reasoning, failure recovery, and benchmarks such as DeepSearch QA, MCP-Atlas, τ³-Bench, and SWE-Bench. The post claims ~4-bit quantization reduces the LM to <20 GB, enabling operation in 24–32 GB memory envelopes alongside KV cache, perception encoder, and a DFlash-based speculative decoding drafter with “identical output quality”; weights/resources are linked on Hugging Face, the research blog, and developer docs. A technical comment points to Alexandr Wang saying an open-weight Muse Spark 1.2 release is coming soon on X. Comments are mostly positive but light on technical scrutiny, expressing enthusiasm that Meta is releasing open weights again and jokingly framing Muse Glimmer as “llama 5.”
A commenter cites Alexandr Wang on X stating that an open-weight version of Muse Spark 1.2 will be released soon, which is technically relevant because it suggests Meta may follow Muse Glimmer with a higher-tier or newer open-weight variant. Source: x.com/alexandr_wang/status/2086756152034066792.
Meta releases Muse Glimmer 30B - a new open model (Activity: 450): The image is a promotional benchmark graphic for Meta “Muse Glimmer-30B”, presented as a new open-weight 30B dense vision model under Apache 2.0. It claims competitive results versus Gemma 4-31B and Qwen3.6-27B on agentic/code/math/science benchmarks including MCP Atlas, DeepSearch QA, SWE-Bench Pro, AIME 2026, and SciCode, and advertises that it can run on 18GB RAM/VRAM setups via Unsloth Desktop. Commenters were broadly positive about Meta returning to open model releases, but one noted skepticism about cadence, saying it may be “the strongest agentic model for its size for like three days before they release Qwen,” implying rapid competition from Qwen and pressure on Meta to improve release velocity.
Commenters frame Muse Glimmer 30B as a potentially strong agentic model in the ~30B dense-model size class, but expect it to be quickly challenged by upcoming Qwen releases; one commenter says it may be “the strongest agentic model for its size for like three days before they release Qwen.” The technically relevant concern is release cadence: Meta is seen as needing faster iteration to remain competitive with Qwen and other open-model labs.
A substantive ecosystem point is that the ~30B parameter tier is becoming crowded, with commenters naming Qwen, Google, NVIDIA, and Meta as active players. One commenter hopes Meta follows this release with a similarly sized MoE model, mirroring expectations that Qwen may also expand in that direction.
Muse Glimmer ACTUALLY fits on a single RTX 3090 (Activity: 640): A user reports Meta Muse Glimmer 30B Q4_K_XL GGUF runs on a single RTX 3090 24GB with 262144 context, DFlash speculative draft, mmproj, FlashAttention, and F16 KV cache, using only ~22–23GB VRAM—unlike their tested Q4_K_XL Qwen3.6-27B and Gemma-4-31B, which hit VRAM limits at ~70k/52k tokens with F16 KV or 125k/81k with Q8 KV. They measured ~64–124 tok/s generation under DFlash, ~1400 tok/s prompt processing, and passed a two-needle retrieval test at ~150k tokens, suggesting the model is not effectively capped at 128k; a commenter notes the official Muse-Glimmer-30B-GGUF releases already target 24GB/32GB VRAM, and another reports very compact KV usage: ~1.8 GiB for 131k F16 despite SWA on all layers. Commenters were positively surprised by the KV-cache efficiency, especially given SWA across all layers; one joked that this could further increase RTX 3090 demand/prices.
Users highlighted that Muse Glimmer’s KV cache appears unusually memory-efficient despite SWA on all layers: one report claims a 131k context with F16 KV uses only about 1.8 GiB, making long-context operation feasible on a single RTX 3090.
A commenter noted that the official Meta GGUF builds already target 24GB and 32GB VRAM configurations, including DFlash support, so Unsloth GGUFs may not be required. The referenced official repository is meta-models/Muse-Glimmer-30B-GGUF.
Another technical report claims 256k context + DFlash + mmproj fits in roughly 22–23GB VRAM on an RTX 3090, with observed throughput around 64–124 tok/s. They also noted that a 150k needle test reportedly holds up, but questioned how performance and retrieval quality behave once the context is filled closer to 200k+.
1 Day in and I feel okay saying Muse-Glimmer-30B finally beats 3.6-27B for the size in some use-cases (Activity: 709): OP reports that Muse-Glimmer-30B appears to outperform Qwen 3.6-27B in selected 24GB GPU-class local use cases after ~1 day of testing, especially efficient reasoning, low-bit quantization (iq3_xxs reportedly degrades less than Qwen/Gemma), no-tools trivia/knowledge depth, and OpenCode agent efficiency. They still rate it weaker for general coding—roughly around Gemma4-31B level—but claim it completes agentic tasks faster than 3.6-27B despite similar task success. Commenters echoed strong early results for agentic workflows/tool calling, with one saying Muse-Glimmer-30B “isn’t even close,” but others expect an imminent 3.8 release to erase the lead. One technical criticism was that American models may waste tokens on safety/self-validation before answering.
One commenter reported a few hours of A/B testing where Muse-Glimmer-30B substantially outperformed 3.6 27B specifically in agentic workflows and tool calling, saying “it isn’t even close.” Another user qualified the improvement as strongest for non-coding tasks, while coding performance was left unverified.
A technical concern raised was token inefficiency from safety/alignment preambles: one user asked whether Muse-Glimmer-30B spends many tokens validating that requests are allowed under its policy framework. This was framed as a common issue with some American-aligned models where safety verbosity can reduce practical throughput in interactive or agentic use.
Several comments noted that the comparison may be short-lived because 3.8 is expected imminently and could change the relative ranking versus Muse-Glimmer-30B and 3.6 27B. One dissenting commenter still considered 3.6 27B the stronger baseline overall, suggesting the new model’s advantage may be workload-specific rather than universal.
Early signs that Muse-Glimmer-30B might quantize very well? Share your experiences. (Activity: 354): The image is a social media post from Unsloth AI showing Muse-Glimmer-30B-GGUF running in a chat/coding-agent workflow with visible tool calls, claiming a 2-bit quantized 30B model executed 100+ tool calls while using about 14GB RAM: image. In the Reddit discussion, users question whether 14GB is actually impressive for “2-bit” on a 30B model, while another reports Q4_K_XL on a single RTX 3090 performing well for agentic coding and roughly “on-par with 3.6 27B.” Commenters are split between optimism about Glimmer’s quantization/agentic-coding performance and skepticism about memory efficiency. There is also concern that the model may be overly safety-restricted, with one user citing refusals for code that moves the mouse pointer.
One user reports running Muse-Glimmer-30B as Q4_K_XL on a single RTX 3090 for agentic coding and says it is “performing great,” roughly on par with 3.6 27B in their early testing. Another commenter notes that a 14GB “2-bit” quant is relatively large for a 30B model, implying the packaging/quantization format may include substantial overhead or not be a straightforward 2-bit weight-only footprint.
A technically focused concern is how Glimmer behaves under KV-cache quantization, especially whether degradation from fp16 to q8_0 resembles Qwen or Gemma-style sensitivity. The commenter specifically wants Glimmer added to Anbeeld’s KV-cache benchmark methodology: KV cache quantization benchmarks / KVARn precision tail.
A user testing the BF16 model through vLLM reports disappointing quality versus Laguna-S-2.1, saying Glimmer made many errors that Laguna would not. They suggest the result may be due to early-release issues and plan to retest once the official repo/model release stabilizes.
2. Qwen 3.8-27B and Ling-3.0 Tiny Open Weights
Qwen 3.8-27b coming this week (Activity: 2791): The image is a screenshot of the official Qwen / Alibaba_Qwen X account confirming that Qwen3.8-27B open weights are landing this week, matching the post title’s claim. Comments point to a ModelScope listing for Qwen3.8-2.4T-A95B, noting ModelScope is Alibaba-owned and suggesting the release timing/countdown may be credible. Commenters are already comparing expectations against other Qwen variants, especially asking whether a 35B-A3B-like model is coming because it reportedly performs well on certain tasks with strong speed for its hardware footprint.
There was interest in whether a 35B-A3B-style Qwen variant will arrive, with one user noting that 35BA3B performs “amazing” on certain task types while maintaining strong speed for its hardware footprint, implying demand for smaller active-parameter MoE-style models rather than only larger dense releases.
A Strix Halo owner requested a newer 122B release, saying the current Qwen 3.5 122B feels outdated; this reflects interest in very large local models that can plausibly run on high-memory AMD APU platforms.
inclusionAI/Ling-3.0-tiny · 8B A1.3B MoE· Hugging Face (Activity: 427): inclusionAI released Ling-3.0-tiny, an 8B-parameter MoE with ~1.3B active parameters, positioned by the OP between 4B and 8–12B Qwen/Gemma-class dense models. The model card reports FP8 throughput of ~100–105 tok/s on DGX Spark and 86–90 tok/s on an M4 Pro MacBook, with ~8.34 GiB peak memory at 8K context; commenters also highlight a 256K context window and an AA Bench score of 25 from a shared benchmark image. One commenter compared it favorably against recent LFM small models: IFBench 63.61, Multi-IF 83.15, and BFCL-v4 62.72, beating LFM2.5-8B-A1B and LFM2.5-2.6B on those listed metrics. Commenters were broadly positive about tiny MoE architectures for low-memory, mobile, and edge inference due to high tokens/sec, with one saying it may replace Ling-Mini-2.0 locally. There was interest in larger 15–50B Ling releases and speculation that speculative decoding could push throughput toward diffusion-model-like responsiveness.
Users highlighted Ling-3.0-tiny as an 8B MoE model with roughly A1.3B active parameters, making it attractive for low-memory, mobile, and edge deployments due to expected faster tokens/sec versus denser models. One commenter noted it scores 25 on AA Bench, which they considered notable for this size class.
A technical comparison against recent LFM small models reported Ling-3.0-tiny ahead on instruction-following and tool-use benchmarks: IFBench 63.61 vs 56.47 for LFM2.5-8B-A1B, Multi-IF 83.15 vs 79.93, and BFCL-v4 function calling 62.72 vs 49.73. The same commenter emphasized its 256k context window on an 8B/A1B-style model as a key differentiator.
There was interest in runtime compatibility, specifically whether llama.cpp support exists yet. Another commenter suggested future larger 15B–50B Ling models combined with speculative decoding could significantly improve throughput, potentially approaching the perceived responsiveness of diffusion-style generation pipelines.
This January, four big AI × Pharma tools deals were announced at the huge JPM Pharma conference that takes over San Francisco every year. OpenAI-backed Chai Discovery (now worth $4B) was somehow at the heart despite being all of 2 years old.
The Science team is proud to bring you the first podcast with cofounder Matt McPartlon and product lead Neil Patil to tell the full story!
Editor’s note: not to be confused with Chai AI, which was another top pod of ours.
Pharma suddenly doing big AI tools deals
For the non-pharma people, JPM is JP Morgan’s annual conference for pharma deal-making that takes over San Francisco for a week in January with hundreds of side events, etc. It’s a big thing.
Tools deals for pharma are also a big (new) thing: companies that start as AI for Pharma usually end up building their own drug pipelines instead, and the reason is something like this: convincing pharma to use your tool requires proof that your tool works. Proof means good targets, maybe with good clinical validation. If you have that, then it’s easier to raise money (with a known, if long path to commercialization) or sell (e.g payment in biobucks1) for a specific target than it is to sell to lots of companies on a promise that it will work across their portfolios.
The “we’ll just partner / build our own drug” optionality proved to be the only good path up until January. What changed? In short, the tools got good enough for drug design teams to trust.
Good-enough-to-trust unlocks the ability to scale discovery: get more, better candidates into the lab and animal trials faster. More screening for toxicity, better delivery, etc. This means that what you push to the clinic is more likely to succeed.
Tools also unlock new capabilities: mechanisms that are very hard or impossible to develop using lab-based discovery. Designing an antibody that precisely triggers a very specific molecular cascade takes many years of trial and error. Designing bi-specific antibodies (that bind to two different proteins) is similarly difficult. Good design tools can unlock this.
RJ: The fact that the quality of the model has jumped means you’re enabling things you just plain couldn’t do. So it’s a step change. It’s not an efficiency argument at all, or not so much.
Matt: Yeah, exactly. It’s kind of interesting, even for us — it took me a while to believe in the thesis, actually. I talked to Josh for months before Chai started... It’s like, can I beat a mouse, and then can I do what mice can’t do? And then how many levels of interaction can you just keep building on top of that?
Everyone playing in the structural / binding space has an angle here, and some will be better than others, but Chai is pointing to a different unlock: getting good molecules right out of the gate (meaning they don’t then need as much lab work) means that the iteration time is faster. This turns science into engineering: you can design your systems to reduce friction and hill climb towards one-shotting molecules all the way to the clinic.
This, per-se, is not a new thesis: a16z articulated a version of this in 2020. What has changed is that structural models became binding models (how well doesn’t this molecule bind to this molecule, aka “binding affinity). Binding models unlock design, which has been steadily improving. Chai’s observation is that forengineering problems the best product tends to win, and good technology is a necessary but not sufficient condition.
With that in mind Chai has invested heavily in partnerships that allow them to learn from their Pharma counterparts.
What is kind of cool about working so closely and supporting so many of these partners is we get to really learn about what is the stuff that would be helpful in research. So rather than doing research in a vacuum, based on what would hypothetically be cool, we're able to do informed research based on what our partners have just been organically asking us for help with.
— Neil Patil, (Chai product lead)
This means better UX, such as a molecule editor that is more like a CAD or graphics design program than a chatbot.
Their approach has paid off: since June, Chai has announced three more major deals: Lilly, Novartis, argenx, plus an expansion of their Eli Lily program. This episode is too full of quotable moments for a short blog, so tune in to learn about
Why protein tokens have the highest downstream value of any token
Climbing levels of abstraction as models improve
How Pharma, VC, and research are all just portfolio optimization
"Biobucks" is deal-value for milestone-heavy licensing agreements — the headline number (e.g., "$1.7B deal") is almost entirely contingent on hitting targets. Typically only 2–5% of the total is upfront; the rest pays out only if the drug clears each gate, and most drugs don't.
Login
