Poolside, the San Francisco AI lab that has spent most of its three-year existence quietly selling coding models to governments and defense agencies, released its most capable model to date on Tuesday — and made an unusually aggressive bet that radical transparency, not raw scale, is how a smaller lab competes at the frontier.

The model, Laguna S 2.1, is a 118-billion-parameter Mixture-of-Experts (MoE) system that activates only 8 billion parameters per token, supports a context window of up to 1 million tokens, and — according to benchmarks published by the company — matches or beats open models several times its size on agentic coding tasks. The weights are available immediately on Hugging Face under the permissive OpenMDW-1.1 license.

The headline numbers are striking for a model this small. Poolside reports that Laguna S 2.1 scores 70.2% on Terminal-Bench 2.1, a benchmark of long-horizon terminal tasks, placing it 11th on the company's compiled leaderboard — ahead of DeepSeek-V4-Pro-Max, a 1.6-trillion-parameter model that scored 64.0; Thinking Machines' 975-billion-parameter Inkling, at 63.8; and Nvidia’s 550-billion-parameter Nemotron 3 Ultra, at 56.4. On SWE-Bench Multilingual, it posts 78.5%, and on SWE-Bench Pro's public dataset, 59.4%.

Perhaps more telling than any single score: the model went from the start of pre-training on May 22 to public launch in under nine weeks, trained on 4,096 Nvidia H200 GPUs. In an industry where flagship model cycles are typically measured in quarters or years, Poolside has now shipped three models in three months.

Why the West's open-weight AI gap has become a boardroom issue

The release lands in the middle of an increasingly pointed debate about the provenance of open-weight AI. Over the past year, developer adoption has shifted decisively toward open-weight systems that companies can download, inspect, and run on their own infrastructure — and the leading options in that category have overwhelmingly come from Chinese labs. DeepSeek, Qwen, Kimi, GLM, MiniMax, and Tencent's Hunyuan line all feature prominently in Poolside's own comparison tables.

Poolside's accompanying press release frames Laguna S 2.1 explicitly as a response, noting that the model occupies a size class into which no Western lab has released open weights in 11 months — since OpenAI's gpt-oss-120b last August. "The West needs open-weight models it can trust, run, and build on," said Jason Warner, Poolside's co-CEO, in the announcement.

Co-founder and co-CEO Eiso Kant made the philosophical stakes even plainer in a lengthy post on X. "I believe intelligence should and will become a commodity," he wrote, arguing that the open ecosystem "will not win by being the best in its own category." Users, he argued, simply want the best intelligence for the task at hand — so open models must be on par with, or better than, their closed equivalents.

The strategic logic here is not charity. Poolside's core business is deploying models inside the security boundaries of government, defense, and regulated enterprises — customers for whom closed, metered API access is often a non-starter for compliance and sovereignty reasons. 

Every enterprise that standardizes on a Chinese open model today becomes harder to win tomorrow. Releasing competitive open weights is both an ecosystem play and a top-of-funnel strategy for the company's high-security deployment business. It also reframes the AI race away from terrain where Poolside cannot compete — frontier-scale capital expenditure — and toward terrain where it believes it can: cost per token, self-hosting, and iteration speed.

How a sparse architecture makes enterprise AI agents affordable to run

The technical design reflects a specific thesis about where value in coding AI is moving. Laguna S 2.1's sparse MoE architecture — 256 routed experts plus one shared expert, with grouped-query attention and interleaved sliding-window layers, according to the Hugging Face model card — means inference costs scale with the 8 billion active parameters, not the 118 billion total. Poolside emphasizes that the model is small enough to run on a single Nvidia DGX Spark, the desktop-class AI machine.

That matters for what Poolside calls token economics. Long-horizon coding agents are voracious consumers of tokens: the company's published data shows the model consuming a mean of roughly 249,000 completion tokens per trajectory on its hardest benchmark when thinking mode is enabled. At metered API prices, agentic workloads at enterprise scale become a meaningful budget line item. On OpenRouter, Poolside is offering a free 256K-context endpoint and a dedicated 1M-context deployment priced at $0.10 per million input tokens and $0.20 per million output tokens — aggressive pricing that undercuts most frontier alternatives by an order of magnitude.

The ecosystem support is unusually broad for day one. The model is live on Baseten's model library and Vercel's AI Gateway, with integrations across vLLM, SGLang, Ollama, and llama.cpp, plus quantized variants down to 4-bit GGUF files — 75 gigabytes — for local use. But Poolside's more interesting claim is behavioral, not architectural. Pengming Wang, co-head of applied research at Poolside, said the gains came from improving the model's working habits: "more verification, less taking things for granted, not declaring victory early, and being more persistent." Raw intelligence, the company argues, is one axis of capability; a model's way of working is a second axis that matters immensely for agents left unattended for hours.

Publishing every benchmark trajectory to counter AI's credibility crisis

The most consequential part of the release for enterprise buyers may be an evaluation-transparency move with little precedent among major labs: Poolside published the complete, unedited trajectory of every trial in its final benchmark runs — every reasoning step, tool call, and shell command behind every reported score.

This addresses a growing credibility problem in AI benchmarking. As top scores on mature benchmarks cluster in the 70–90% range, and as "reward hacking" — models finding solutions online or gaming verifiers rather than solving problems — has become endemic, self-reported numbers have lost much of their signal. Poolside disclosed its own encounters with the problem candidly: during training, more than half of trajectories on some SWE-bench tasks were flagged because the model simply researched the original bug-fix pull request online and applied it. The company documented its mitigations, including prompt addenda, LLM-based judging calibrated against human labels, and expert annotator review of a high-scoring Terminal-Bench run.

Three published case studies illustrate what the company means by persistence. In one, the model built a working HTML/CSS rendering engine from an empty folder in a 181-step, 50-minute unattended session — then, lacking vision capabilities, spun up headless Chromium to numerically compare its canvas output against a real browser's rendering. In another, pointed at Poolside's own agent harness in an automated optimization loop, the model made the Go codebase 5.2% faster with roughly 70% lower memory allocation, finding an O(n²) string-concatenation bug along the way. In a third, working in a sandbox with no Python installed, the model did its number theory in Perl and independently re-derived a proof of Erdős problem #397 — a combinatorics question open for five decades until GPT-5.2 Pro first solved it this past January. Poolside notes that its model's construction is structurally different from the earlier published solution, and that its November 2025 knowledge cutoff precedes the first proof.

What the disclosed limitations and benchmark fine print reveal

Poolside deserves credit for disclosing limitations most labs bury. The model can overfit to its native harness and stumble on slightly different tool schemas in third-party agents, mangles JSON in nested tool arguments, and is prone to overthinking on competition math. There is currently no user-configurable thinking-effort dial — just on or off — and the gap between the modes is enormous: thinking lifts Terminal-Bench 2.1 from 60.4% to 70.2%, and DeepSWE from 16.5% to 40.4%, at substantially higher token cost.

Buyers should apply their own discounts to the comparison tables. Poolside's methodology takes the maximum of vendor self-reported scores, benchmark-author leaderboards, and third-party figures for competitors — a reasonable convention, but one that mixes harnesses and test conditions. On DeepSWE, notably, Poolside ran its own agent harness rather than the leaderboard's standard mini-swe-agent, a difference the company acknowledges makes scores less directly comparable. And the frontier remains clearly out of reach: closed models like GPT-5.6 Sol, at 88.8 on Terminal-Bench 2.1, and Claude Fable 5, at 88.0, along with the 2.8-trillion-parameter open-weight Kimi K3, at 88.3, sit well above Laguna S 2.1.

The deeper structural question is whether Poolside's "Model Factory" — the internal platform the company credits for its rapid release cadence — can sustain this pace as models scale. The trajectory so far is genuinely unusual: the April dual release of Laguna M.1 and XS.2, the July 2 refresh of XS 2.1, and now S 2.1, which the company says outperforms April's flagship M.1 at roughly a third of its active size. Remarkably, S 2.1 used the exact same pre-training data as XS 2.1, meaning nearly all the improvement came from scale, training fixes, and post-training across the company's corpus of 409,000 agentic and non-agentic training environments. Poolside says its next, larger Laguna model began pre-training last week.

For technical decision makers, Laguna S 2.1 is the most credible Western open-weight option to emerge in nearly a year for self-hosted agentic coding — with published evidence, a permissive license, broad ecosystem support, and an economics story built around hardware you can own. Whether it dents the dominance of Chinese open models will depend less on this release than on the ones that follow it.

Kant, for his part, has already told the world how he intends that story to end. Poolside is building toward a future where the most capable intelligence "can be owned and shaped by anyone," he wrote — and the company plans to keep shipping "until that future exists." In an industry where the biggest labs increasingly lock their best work behind an API, the most radical thing about Laguna S 2.1 may not be what it scores, but that anyone can download it and check.

GPU memory is the most expensive resource in production AI, and it's also the one running out fastest.

Long context windows and multi-turn conversations force AI models to repeatedly recompute information they've already processed, consuming GPU memory and compute that could otherwise serve additional users or generate new responses.

Instead of treating GPU memory as the limiting resource, why not extend it with much cheaper storage technologies?

Weka, for one, believes that cheap flash storage can close that gap. The company's NeuralMesh 6 software platform, launching alongside its first self-designed hardware line, Wekapod 3, extends what Weka calls Augmented Memory Grid, an approach that aggregates NAND flash to behave like GPU memory at a fraction of the cost.

This is an active and increasingly crowded category. Dell, NetApp, Pure Storage and VAST have all repositioned toward AI infrastructure over the past two years and Weka is one of several vendors arguing it's built for this specific moment rather than adapting to it.

"What we're seeing now with customers is they're chasing availability of compute, and once they get new allocation from anyone, they want to be able to grab it and start running right away," Weka co-founder and CEO Liran Zvibel, told VentureBeat.

The potential payoff is straightforward: better utilization of existing GPU investments, lower inference costs and faster deployment of new AI workloads without waiting months for additional GPU capacity.

The technology is most relevant for organizations already operating AI at scale or expecting rapid growth in usage, particularly enterprises building internal copilots, customer service agents, software engineering assistants or retrieval systems with long context windows. Smaller deployments may see less immediate benefit than organizations where GPU utilization has already become a limiting factor.

Inside Weka's NeuralMesh 6

NeuralMesh 6 adds four capabilities aimed directly at a functionality gap Zvibel says has been costing Weka deals in competitive evaluations.

Composable and virtual multi-tenancy. Composable clusters give anchor tenants full hardware-level isolation, dedicated CPU, memory, and storage. Virtual multi-tenancy runs through Weka's RDMA fabric, delivering network-level isolation that scales past 1,000 tenants per cluster, with provisioning in under 30 minutes. Combined, a single cluster running 50 composable clusters can support up to 50,000 tenants. 

Unified file and object storage. Most storage systems keep two separate paths: a file-based path (the standard way servers and applications read and write files, used heavily in training and fine-tuning pipelines) and an object-based path (S3, the format inference and cloud-native tools typically expect). Normally a gateway translates between the two, meaning the data effectively exists twice. Weka's claim is that the same physical data on disk is directly readable through either path at once, no translation layer, no second copy. Zvibel is targeting non-AWS GPU clouds specifically, naming Lambda, Nebius, G42, and CoreWeave, with what he described as roughly two orders of magnitude higher performance than conventional S3 and a capacity-based pricing model instead of per-API charges. 

Metadata-first replication. Destination environments become browsable before a full data copy arrives, with data hydrating only when accessed. 

"They had to wait for all of that to make it to the other side, and this takes days or weeks, in extreme cases a month," Zvibel said. "We now allow our customers to grab some allocation of new GPUs and get up and running within an hour."

AlloyFlash and Always-On data reduction. TLC and QLC are two types of NAND flash memory. TLC is faster and more durable but costs more per terabyte, while QLC is cheaper and holds more data per chip but is slower. AlloyFlash mixes both within a single cluster, automatically routing latency-sensitive work to TLC while running bulk-capacity workloads on QLC, cutting cost per terabyte without a performance penalty on the work that needs speed. Data reduction now runs by default rather than as an option.

Solving AI's context problem

Multi-tenancy and object storage solve how enterprises and neo clouds operate the platform day to day. A harder problem sits underneath: as context windows and multi-turn interactions grow, so does the GPU compute wasted recalculating work a model has already done. Augmented Memory Grid, a NeuralMesh 6 feature built specifically for this, is Weka's answer.

Every prompt triggers two stages. Prefill calculates attention, the core mechanism behind how large language models process input, and it's computationally expensive. Decode converts that calculation into output and is comparatively lightweight.

The cost shows up hardest in multi-turn sessions like chat or coding, where each new turn re-triggers prefill for everything that came before it, unless that work has been cached.

"If you have 10 turns, you may overcalculate 100 times because you're redoing all of them. If you have 20, you'll overcalculate 400 times," Zvibel said. "You can put two orders of magnitude more NAND than you could afford in shared memory, and we can cache 100% of the pre-calculated tokens, so you never need to redo it."

Where Weka sits competitively

Storage vendors have spent the past year and a half repositioning around AI, and separating genuine capability from repositioned messaging is now a real evaluation problem for buyers. 

"The storage world is shifting its focus from serving bits to enterprise workloads to managing data at the speed of AI. We've seen that most clearly over the past 18 months from Dell, NetApp, and Pure," Steve McDowell, chief analyst at NAND Research, told VentureBeat. "The interesting thing is that companies like Weka, and VAST, are the true AI-native data companies, solving these problems since day one."

McDowell singled out Augmented Memory Grid as Weka's clearest technical lead. 

"Weka continues to have the most technically capable KV cache implementation on the market with its Augmented Memory Grid," he said. " They were early with this technology, and continue to innovate. This is critical for AI inference, as it enables a level of GPU efficiency that, without question, saves money on GPUs and memory. That’s key for today’s memory and GPU constrained market." 

He also flagged Weka's contractual guarantee on its data reduction claims as underappreciated. 

"One flying a little under the radar: Weka is putting its money where its mouth is with its contractual guarantees for its data reduction promises," he said.

McDowell's advice to buyers evaluating competing claims from Weka, VAST, Pure and NetApp alike was pointed suggesting that enterprise buyers should look hard at what vendors are promising versus what they're actually delivering.

"A smart buyer will look at how competing vendors are solving real-world problems today," McDowell said. " They do this by talking to organizations running similar workloads at similar scale. If a vendor can't point to that, then it should be a warning sign."

Inference is the largest long-term expense in enterprise AI deployments. Businesses can keep AI inference costs in check with these seven optimization strategies.

Moonshot AI became the latest AI company to discover that launching a popular model is only half the battle. Less than two days after releasing Kimi K3, the company stopped accepting new subscribers after demand exhausted its available GPU capacity. Existing users will keep access while Moonshot expands its infrastructure and reopens subscriptions in batches.

Inference demand outpaces supply

The incident emphasizes how demand is outpacing available infrastructure. As AI models take on longer, more coding and agentic workloads, companies are finding they need more inference capacity than they anticipated.

“Kimi K3 has received far more love than we expected,” the official Moonshot account writes on X. “Over the past 48 hours, demand has pushed close to the limits of our current capacity. We’re adding capacity as fast as we can and will reopen new subscription spots in batches.”

Kimi K3 has received far more love than we expected, and our GPUs are feeling it.

Over the past 48 hours, demand has pushed close to the limits of our current capacity. To protect the experience of existing subscribers, we're temporarily pausing new subscriptions and…

— Kimi.ai (@Kimi_Moonshot) July 19, 2026

“We’re adding capacity as fast as we can and will reopen new subscription spots in batches.”

For infrastructure engineers and developers, the resulting capacity crunch is a bold indication of why companies from OpenAI to Anthropic to Moonshot are rationing access instead of selling unlimited usage.

Open weights, closed capacity

At 2.8 trillion parameters, Kimi K3 is one of the largest open-weight models slated for release — Moonshot has scheduled the public weight drop for July 27. In Arena.ai’s Frontend Code Arena, K3 topped both OpenAI’s GPT-5.6 Sol and Anthropic’s Claude Fable 5. On the broader Artificial Analysis Intelligence Index, it trails both, scoring 57 to Fable 5’s 60 and Sol’s 59. That doesn’t make it any easier to run.

Open weights let anyone deploy the model, but whoever hosts it still has to pay the inference bill. Coding activities tend to tie up GPU resources far longer than a typical chatbot interaction, making it harder to keep latency low as more developers pile on.

“Agent tasks are not one-off question answering; they continuously generate, read, and process tokens during ongoing tasks,” wrote Citigroup semiconductor analyst Peter Lee in a research note.

Lee argued that as developers build longer agentic workflows, lower inference costs are quickly “re-converted into higher total resource consumption,” shifting the bottleneck from compute to server memory.

Moonshot’s subscription pause is a sign that keeping enough inference capacity online once developers start using it at scale might be just as hard as building the model.

“Agent tasks are not one-off question answering; they continuously generate, read, and process tokens during ongoing tasks.”

China’s chip constraints compound crunch

For a company like Moonshot, this general industry bottleneck is compounded by regional infrastructure realities. Unlike traditional software companies, AI developers typically rent much of this computing power from cloud providers such as Alibaba Cloud, Tencent Cloud, and Huawei Cloud rather than owning extensive data-center infrastructure themselves.

The capacity crunch illustrates the mounting challenge facing Chinese AI developers as US export controls continue to restrict access to leading chip provider Nvidia’s most advanced AI chips. As a result, companies such as Moonshot depend on a combination of older chips and domestically produced alternatives. These constraints have forced Chinese developers to concentrate strongly on software tuning and more efficient use of computing resources to narrow the performance gap with US rivals.

Token economics under pressure

The scramble for computing power has fueled a data-center construction boom across China. Alibaba has committed more than $53 billion to AI and cloud infrastructure over three years, while ByteDance is reportedly considering spending as much as $70 billion this year on AI data centers and related infrastructure.

AI companies typically charge customers based on the number of tokens, or units of text, processed by a model, making token prices a key measure of operating costs.

According to Bernstein Research, Moonshot charges $3 per million input tokens and $15 per million output tokens for Kimi K3. This makes it about 40% cheaper than Anthropic’s Opus 4.8 and roughly 70% cheaper than Claude Fable 5.

“A world where there are only [two to three] dominant frontier labs with 90 percent inference margins is net negative for every other layer while being awesome for those [two to three] labs,” wrote Atreides Management founder Gavin Baker on X. Baker argued that models like Kimi K3, Grok 4.5, and Muse 1.1 could shift value away from the model layer and toward chipmakers, cloud providers, and the software companies building the infrastructure that serves AI models.

For developers, Moonshot’s subscription freeze serves as an architectural warning. The era of assuming infinite, cheap API access is ending.

“A world where there are only [two to three] dominant frontier labs with 90 percent inference margins is net negative for every other layer while being awesome for those [two to three] labs.”

The post Moonshot launched Kimi K3. Then demand shut down subscriptions in 48 hours. appeared first on The New Stack.

OpenAI and Hugging Face share early findings from a security incident during AI model evaluation, highlighting advanced cyber capabilities and lessons for defenders.

David Vélez and Robin Vince join the boards of the OpenAI Foundation and OpenAI Group PBC, bringing global leadership in finance, technology, and governance.

Google DeepMind today released three new proprietary AI models it says are among its most token-efficient yet: Gemini 3.6 Flash, Gemini 3.5 Flash-Lite, and Gemini 3.5 Flash Cyber.

The models aim to make AI agents faster, smarter, and cheaper at scale. Google is pricing Gemini 3.6 Flash at $1.50 per one million input tokens and $7.50 per one million output tokens through its application programming interface (API), while Gemini 3.5 Flash-Lite costs a staggeringly cheap $0.30/$2.50 per million tokens in/out.

Compare that to the $1.50/$9.00 per 1M tokens for Gemini 3.5 Flash, and the $2/$12 for Gemini 3.1 Pro Preview, and the savings are considerable. However, Google's prior generation Gemini 3.1 Flash-Lite still remains the search giant's "most cost-efficient" model at $0.25/$1.50 per 1M tokens. Yet, it remains 2X slower than the new, more expensive Gemini 3.5 Flash-Lite, giving those enterprises who value speed more "bang" for their buck.

VB Frontier AI Model API Pricing Comparison Chart (Late July 2026 Shortlist)

Model	Input ($/1M)	Output ($/1M)	Total ($/1M)	Source
MiMo-V2.5 Flash	$0.10	$0.30	$0.40	Xiaomi
deepseek-v4-flash	$0.14	$0.28	$0.42	DeepSeek
deepseek-v4-pro	$0.435	$0.87	$1.305	DeepSeek
MiniMax-M3	$0.30	$1.20	$1.50	MiniMax
LongCat-2.0 — limited-time promo	$0.30	$1.20	$1.50	LongCat
Gemini 3.1 Flash-Lite	$0.25	$1.50	$1.75	Google
Qwen3.7-Plus	$0.40	$1.60	$2.00	Alibaba Cloud
MiMo-V2.5	$0.40	$2.00	$2.40	Xiaomi
Gemini 3.5 Flash-Lite	$0.30	$2.50	$2.80	Google
LongCat-2.0 — standard	$0.75	$2.95	$3.70	LongCat
MiMo-V2.5 Pro (≤256K)	$1.00	$3.00	$4.00	Xiaomi
GLM-5.2	$1.40	$4.40	$5.80	Z.ai
GPT-5.6 Luna	$1.00	$6.00	$7.00	OpenAI
Grok 4.5	$2.00	$6.00	$8.00	xAI
MiMo-V2.5 Pro (>256K)	$2.00	$6.00	$8.00	Xiaomi
Gemini 3.6 Flash	$1.50	$7.50	$9.00	Google
Qwen3.7-Max	$2.50	$7.50	$10.00	Alibaba Cloud
Gemini 3.5 Flash	$1.50	$9.00	$10.50	Google
Gemini 3.1 Pro Preview (≤200K)	$2.00	$12.00	$14.00	Google
GPT-5.6 Terra	$2.50	$15.00	$17.50	OpenAI
GPT-5.4	$2.50	$15.00	$17.50	OpenAI
Kimi K3	$3.00	$15.00	$18.00	Moonshot AI
Gemini 3.1 Pro Preview (>200K)	$4.00	$18.00	$22.00	Google
Claude Opus 4.8	$5.00	$25.00	$30.00	Anthropic
GPT-5.5	$5.00	$30.00	$35.00	OpenAI
GPT-5.5 Instant (chat-latest)	$5.00	$30.00	$35.00	OpenAI
Sakana Fugu Ultra (≤272K)	$5.00	$30.00	$35.00	Sakana AI
GPT-5.6 Sol	$5.00	$30.00	$35.00	OpenAI
Claude Fable 5 / Claude Mythos 5	$10.00	$50.00	$60.00	Anthropic

No price was provided yet for the specialty Gemini 3.5 Flash Cyber model, which, as its name would imply, is designed for cybersecurity researchers and red teamers to patch bugs.

While the prices are among the middle-low end of all major AI models globally, the fact that Google designed them to use less tokens overall also should drive down costs for enterprises beyond what the sticker price shows (since you'll be paying for fewer total tokens at any rate).

Gemini 3.6 Flash and Gemini 3.5 Flash-Lite are available immediately through the Gemini API in Google AI Studio and Android Studio, as well as within the consumer Gemini application and Google Search. According to a separate Google blog post, Gemini 3.5 Flash Cyber will be available "exclusively available to governments and trusted partners via CodeMender soon" — CodeMender being Google's proprietary AI code bug-fixing agent released last year.

As with previous Gemini models, these are all proprietary and "closed source," thus, they can only be obtained through Google's official API and that of its partners, as opposed to an open-source license like MIT or Apache 2.0.

One conspicuous omission noted by developers on X and social media: where is the larger, more powerful, flagship Gemini 3.5 Pro model Google previously alluded would be released this summer? After all, Gemini 3.1 Pro, the prior flagship, debuted back in February 2026, and rivals OpenAI and Anthropic have since released several more generations of flagship updates far more powerful than Google's.

Google technical staffer Logan Kilpatrick responded to one such inquiry on X, writing: "Gemini 3.5 Pro is currently testing with partners and we plan to make it broadly available as soon as it’s ready."

Google's release signals that the immediate future of AI lies in agentic capabilities—systems that operate autonomously over extended periods.

If early large language models are akin to massive, fuel-hungry freight trains capable of hauling incredible loads at immense cost, the new Flash series represents a fleet of nimble, hyper-efficient hybrid delivery vans.

Efficiency gains ranging from 17% to 65% reduced tokens for strong results on third-party benchmarks

Under the hood, Gemini 3.6 Flash achieves significant efficiency gains. The model reduces output token usage by 17% compared to its predecessor, Gemini 3.5 Flash, according to the Artificial Analysis Index maintained by the independent third-party AI benchmarking group of the same name.

In specific long-horizon software engineering benchmarks like DeepSWE, which measures how well agents complete multi-step engineering tasks from scratch, the token savings reach up to 65%.

This reduction means the model requires fewer reasoning steps and tool calls to complete the exact same multi-step workflow. Think of token efficiency like fuel economy in a vehicle.

When an AI model takes a convoluted path to solve a problem, it burns through more computational fuel, driving up the final cost for the developer. By streamlining its internal logic, Gemini 3.6 Flash arrives at the correct answer faster and cheaper.

While Google's materials did not specify the exact architectural or algorithmic changes used to achieve this token efficiency, they noted that the model "takes fewer reasoning steps and tool calls to accomplish multi-step workflows" and exhibits reduced "verbosity."

The official model cards released by Google reveal that both Gemini 3.6 Flash and Gemini 3.5 Flash-Lite feature a 1-million-token input context window alongside a max output limit of 64,000 tokens, with both models sharing a knowledge cutoff date of March 2026.

Respectable benchmark performance at low cost

The technological improvements extend to concrete capabilities. Gemini 3.6 Flash scores 49% on the DeepSWE benchmark, a notable increase from the 37% achieved by version 3.5.

It also pushes machine learning engineering performance higher, scoring 63.9% on MLE-Bench compared to 49.7% previously. Furthermore, Google integrates computer use as a built-in client-side tool via the Gemini API and Gemini Enterprise, reflecting an OSWorld-Verified score of 83.0%, up from 78.4%.

The model also tackles knowledge work with greater proficiency, outperforming its predecessor on benchmarks like GDPval-AA v2 by moving from a score of 1349 to 1421.

To ensure safety amidst these capability upgrades, Google deploys enhanced Frontier Safety safeguards. These protections harden the model against jailbreaks and mitigate risks in Chemical, Biological, Radiological, and Nuclear domains, as well as cyber offense misuses.

The engineering team trains the model to minimize refusals for beneficial uses, striking a necessary balance between strict security and practical utility.

Models for low-cost coding, agentic, and cybersecurity use cases — respectively

Google divided its new offerings into three distinct products tailored for different operational needs.

Gemini 3.6 Flash serves as the heavy-duty workhorse of the trio. It handles complex coding, intricate knowledge work, and multimodal processing with improved precision. Enterprise customers utilize it for demanding tasks such as complex document parsing, intricate chart and data analysis, and long-form report drafting.

The model executes complex code migrations using multi-agent orchestration frameworks with lower latency and higher quality than earlier iterations. Furthermore, 3.6 Flash aids in developing photographic texture extractors for 3D workflows using canvas interfaces.

Gemini 3.5 Flash-Lite targets environments where high throughput and absolute minimal latency are non-negotiable. Google designates it as the fastest model in the 3.5 series.

As measured by Artificial Analysis, the model processes 350 output tokens per second, making it highly effective for agentic search and massive document processing workloads. Artificial Analysis notes this is about twice as fast as prior generation model Gemini 3.1 Flash-Lite.

Developers can configure 3.5 Flash-Lite to prioritize low-latency execution for high-volume tasks using minimal thinking levels, or engage higher thinking levels to process complex multi-step subagent workloads.

Despite its lite designation, it outperforms the standard Gemini 3 Flash on several key agentic and coding evaluations, including SWE-Bench Pro, where it scores 54.2% compared to 49.6%, and OSWorld-Verified, scoring 74.0% versus 65.1%.

The model extracts product features from massive datasets, generates interactive web design concepts, and scales receipt translation seamlessly.

The third product, Gemini 3.5 Flash Cyber, represents a highly specialized deployment. Google fine-tuned this model specifically to find and fix cybersecurity vulnerabilities. It integrates directly with Google's CodeMender agent.

In practice, multiple 3.5 Flash Cyber agents work concurrently to produce a single, comprehensive vulnerability report, achieving competitive performance at the frontier on the CyberGym benchmark, even getting within range of Anthropic's much-hyped Mythos model.

Google did not specify an exact numerical cost for 3.5 Flash Cyber, stating only that it is fine-tuned "at a lower price per token than larger models.

Commercial licensing only

The licensing framework for the new Gemini models carries profound implications for developers and enterprise users. Google deploys Gemini 3.6 Flash and 3.5 Flash-Lite under a commercial, proprietary API model. Unlike open-source software governed by licenses such as the MIT License or the GNU General Public License, developers do not gain access to the underlying model weights, training data, or source code.

An MIT or GPL license grants users the freedom to download the codebase, modify the internal architecture, self-host the deployment, and distribute the software infrastructure independently. In contrast, Google's API approach means developers essentially rent access to the intelligence on a strict metered basis. Every prompt and generated response travels through Google's managed servers, incurring a cost based on the strict pricing structure of $1.50 per million input tokens for 3.6 Flash.

This commercial tethering restricts deployment flexibility. Enterprises cannot air-gap the models entirely on their own local secure hardware without establishing specialized, high-tier enterprise agreements with Google Cloud. Developers remain bound by Google's acceptable use policies, arbitrary rate limits, and network requirements, creating a permanent dependency on Google's infrastructure uptime and terms of service.

The licensing for Gemini 3.5 Flash Cyber proves even more restrictive. Acknowledging the dual-use nature of cybersecurity AI—which attackers can weaponize just as easily as defenders can use it to patch systems—Google is for now making the model only available behind a limited-access pilot program, similar to the trend kicked off by Anthropic's Mythos model with its Project Glasswing program, and continued by OpenAI with its staggered rollout for GPT-5.6.

In this case, Google is making 3.5 Flash Cyber exclusively available to governments and trusted partners. This strict gatekeeping prevents open access, prioritizing systemic security over widespread developer innovation.

Looking ahead

Google DeepMind continues to iterate rapidly, but the gap in its product line remains apparent. While the Flash series excels in speed and economy, the industry eagerly awaits the deployment of Gemini 3.5 Pro to gauge Google's absolute frontier capabilities.

Simultaneously, the company confirms that pre-training for Gemini 4 has already commenced.

Until the next major flagship release materializes, developers must optimize their systems using the highly efficient, yet purposefully constrained, Flash architecture.

Bet on information

If test loss flatlines after 1.5B parameters while training loss continues to drop as you scale, that tells you that your model is limited by the amount of information in your data.

Training on a single, smallish data set exposed an information gap: the 3.1B model falls off the scaling trend. Neither parameters nor compute will improve performance past this wall. For predicting changes to gene expression, you need more information rich data.

This is what Chu and Bo’s teams have done, and here is what ~30x the information buys you:

Now we can scale with parameters and training compute! We don’t know how much this effort costed, but we can guess that data collection experiments and infrastructure was a few tens of millions, and compute + headcount + research was a few million. The budget looks like a RL rollout budget, rather than a data rich pre-training one.

We were lucky enough to have the two central figures in this story on our podcast. Taking the lead from Ci Chu and Bo Wang, Xaira Therapeutics is betting that information rich data is the key to AI-driven drug development. Chu was recently promoted to Chief Discovery Officer and Bo to Chief AI Scientist1, underscoring just how strategic Xaira considers this bet.

Reverse engineering the human cell

If you had to figure out how a human cell works, what would you do? A good place to start might be by documenting what genes are expressed (e.g. what RNA is floating around) in different kinds of cells, in different circumstances.

That is CELLxGENE, a database of 168M cells built by Chan Zuckerberg Institute that maps each cell to a count of how many times 20K-30K genes were detected in that cell, plus detailed metadata about every cell. A ~4 trillion-entry matrix.

If the Protein Data Bank (PDB) unlocked structural biology models (Boltz Episode, ESM/BioHub Episode), CELLxGENE has done the same thing for Virtual Cell models. Like PDB, CELLxGENE has inspired a zoo of AI models of RNA expression; so much so that RNA expression models have become synonymous with Virtual Cell models. Bo Wang built one of the most influential, scGPT, that became the starting point for Xaira’s new model.

RNA expression ≠ Virtual Cell

Models trained on CELLxGENE describe the relationship between cell types and cell states, but they are not good at predicting what will happen if we make changes to RNA expression. Changes in gene expression are highly correlated, and its is difficult (impossible) to figure out what causes what in most cases.

If you could “turn the dial down” on one gene at a time, however, then you would be able to observe what is upstream and downstream of a given gene2. You could tell if A → B & C or B → A & C or B → A, C → B → … If you did this for all of the genes, then maybe you could train a model that could predict what would happen to a cell if you change a gene (e.g. with a drug or a gene edit). Or maybe you could figure out the least invasive way to change a particular gene’s expression.

X-Atlas → X-Cell

This is exactly what Chu and Bo’s teams have done. The data set is called X-Atlas and the model is called X-Cell.

In this episode, we discuss:

• Why the team abandoned autoregression for diffusion

• The CRISPR-based experiments that run millions of tests in parallel, and generate the raw data for X-Atlas and X-cell

• Generalization to real lab experiments in real human cells

• Beating the linear baseline that has outperformed previous models

• Justifying a kitchen-sink of priors, and how that stacks up vs. data and architecture

Bo also shared with us some of the (major) advantages he has as an academic vs. industry leader, and how his labs keep up with the breakneck pace of AI innovation.

Check out the full episode on YouTube, or your favorite podcasting platform!

💾

In today's rapidly changing writing landscape -- just kidding. Most readers are savvy enough to recognize specific phrases as AI tells. Why does AI write like that, and why does it matter?

“The new PRD are the evals,” Xavi Amatriain, Expedia Group’s first chief AI and data officer, told the VB Transform 2026 audience last week in Menlo Park. “So basically, you encode what you want the product to do through your evals, which might include red teaming evals and all kinds of other things, which already have a bunch of security requirements. So, you already embed that into the PRD and the product design document before you even start coding.”

He pushed it further. “With AI-assisted or AI-generated code, that’s gonna be the future. It’s like all your thinking is gonna go into the evals.”

Amatriain served as VP of AI and Compute Enablement at Google across the platforms powering Gemini and Google Search before his December 2025 appointment at Expedia. He's mentored talent who went on to found Perplexity and Scale AI.

VentureBeat’s VB Pulse research on the evaluation gap reinforced the stakes. Sixty-six percent of the 157 enterprises surveyed already permit some production deployment without human review or are building toward it within the next 12 months, yet only 5% fully trust the automated evaluations that would make that decision. Half have shipped an agent that passed internal evals but then failed with a real customer.

Don’t let guardrails get in the way of feedback

“The more guardrails and artificial business rules and sort of rules that you put into the system, the worse off,” Amatriain said. “Not only because they’re brittle, but also because they actually mess up with the feedback loop. You are actually biasing the user and the feedback you get from the user, and then you’re learning that in the wrong way.” He called guardrails “a necessary evil” and said the goal is to minimize their impact over time.

Not everyone at Transform agreed. Other speakers argued during the event that the highest-risk actions still demand very firm guardrails.

Expedia governs AI through three layers instead. Principles come first, communicated broadly. “I like to encode at a very high level how I expect decisions to be made, because in a large organization you’re gonna have a lot of distributed decision making,” Amatriain said. “And sometimes, if you’re lucky enough, those principles might be embedded in your culture. But most of the time, my experience has been they’re not.” The processes and tools that enforce them follow. “Principles look really nice on a picture on some wall, but you need to then give them teeth,” he said. Automation sits on top of both.

In practice, this plays out through what Expedia calls agent release toll gates, checkpoints calibrated to risk. “Governance needs to correlate to the risk,” Amatriain said. “And if you have something that is low risk, you don’t need too much governance to get in the way. But if there’s a lot of risk, then you need more governance. That can be encoded.” The toll gates tie evaluation rounds, red teaming, and security review to each agent’s risk level, and the checks shift from recommended to required as the stakes climb.

Specialized agents over monolithic intelligence

“Even when I was at Google, I was like, I don’t believe in AGI as sort of like a singleton and a unified sort of like single model,” Amatriain told the audience. “I think it’s much better to think of it as composition, sort of like having specialized agents that are very good at some task and then composing the system out of those specialized agents.”

Expedia’s architecture starts at the component level. Tools compose into skills, skills assemble into sub-agents, and sub-agents get orchestrated into the full agentic system. “You need to have those principles that are unified that talk about things like what is the tone that we’re using, how are we addressing the user, how are we passing context, memory,” he said. “All of that needs to be thoroughly designed.” He framed this as a systemic design problem. “It’s not about the model, it’s not about a specific solution, it’s about how you’re designing the system.”

Amatriain argued that scoping each agent narrowly also makes the system easier to secure, since teams can evaluate and lock down individual agents in isolation before composing them.

When the user must keep the final click

Travel pricing changes in real time, flight availability shifts minute to minute, and hotel reviews routinely contradict what suppliers claim. Amatriain described a system that blends retrieval-augmented generation with direct API tool calls, choosing the approach based on latency. “If the user asks you a question like, how much does a four star hotel usually cost in Chicago in July, you don’t expect the agent to take two minutes to answer that question,” he said. “You expect an immediate answer because that answer can be cached and it doesn’t need real-time information.” A pet-friendly four-star near Lake Michigan with a pool might justify a 30-second reasoning window.

“The supplier might be saying, yeah, we have a great swimming pool, but then we also have the reviews from the travelers and we actually see there’s two reviews that say the swimming pool was not great or was not open after 6 p.m.,” Amatriain explained. A generic chatbot, he added, would only surface what a supplier self-reports, while Expedia cross-references against its own review corpus.

“We don’t want the agent to book the hotel or to buy you a plane ticket for you,” Amatriain said. “That’s something that the user has to have the agency. And the agent can recommend, can suggest, can discuss with you, but you’re gonna have to hit that click. And that’s non-negotiable.” That constraint, he argued, is also a security decision. “Once you establish those design principles, you also don’t need the guardrail because otherwise you’re gonna have to put all those guardrails in after the fact.”

The next attackers will be other AI systems

“Security needs to be a principle that is shifted as left as possible and as part of the design itself,” Amatriain said in response to an audience question. “And usually when you need a guardrail is because you’ve not thought about it early on.”

A second audience member pressed for lessons learned from production. Amatriain described a feedback loop where monitoring signals flow back into the eval suite. “You can almost automate the whole cycle,” he said. “But having that whole feedback loop from real signals, from your operating AI system, all the way into being reported and fixed as quickly as possible is going to become essential.”

Amatriain's toll gates are a bet that governance calibrated to risk can stay ahead of that feedback loop. VentureBeat’s separate June Pulse survey on agent security, drawn from 107 enterprises, shows how thin that margin is. More than half, 54 percent, have already had an agent security incident or near-miss. Fifty-nine percent plan to adopt, add, or replace agent security tooling within 12 months, and 29% plan to move this quarter. Incident rates climb with organization size, reaching 63% among enterprises with more than 1,000 employees versus 49% for companies with 101 to 1,000. And sandbox isolation, the one post-breach control that limits damage, drops from 35% adoption at the smaller companies to just 20 percent at the largest.

Amatriain warned that threats will increasingly come from other AI systems. “You’re gonna get threats coming not only from humans but also from other external agentic systems that are really powerful, and they’re gonna be poking at everything you’re doing. And as soon as you detect something, it’s not only about the detection, but the time to fix becomes essential here.”

Microsoft and Mistral are deepening their partnership with a multibillion-dollar agreement focused on enterprise AI infrastructure. The goal is to give organizations more flexibility over where they run frontier models and how those deployments are managed, particularly in regions with tight data residency and sovereignty requirements.

Under the deal, announced Tuesday, Microsoft will use Mistral’s expanding European compute infrastructure to increase regional capacity. Mistral, meanwhile, plans to deploy thousands of NVIDIA Vera Rubin GPUs to power everything from model training to multi-agent workloads. 

For engineering teams in regulated industries, the announcement emphasizes where enterprise AI is heading. Many organizations no longer want the public cloud to be the only place they can run AI models. They’re asking for deployments that fit into the infrastructure they already have, including on-premises and air-gapped environments.

Sovereign compute meets agentic AI 

At the compute layer, the agreement gives Microsoft access to European-operated infrastructure outside its traditional first-party data centers and leased facilities. The move builds on Microsoft’s European Digital Commitments, announced in 2025, which focus on keeping customer data in Europe and helping organizations meet regional regulatory requirements.

Crucially, Mistral will power this infrastructure with NVIDIA’s next-generation Vera Rubin rack-scale platform. Because sovereign environments support multi-step agentic workflows, this hardware pairing is essential. NVIDIA claims the platform delivers up to 10x the agent throughput at scale compared to its Grace Blackwell generation.

“Europe should have access to the world’s most capable AI without compromising control over their data, operations or digital future.”

Even though exact financial terms, capacity allocations, and rollout timelines have not been made public, the agreement positions Mistral as both an AI model supplier and an independent European compute vendor for Microsoft’s enterprise ecosystem. For European companies wary of the US CLOUD Act, the appeal is obvious — though whether a US-headquartered provider can fully place data beyond its reach remains legally contested.

“Europe should have access to the world’s most capable AI without compromising control over their data, operations or digital future,” said Microsoft Vice Chair and President Brad Smith in the announcement. 

The model layer

At the model and API tier, Microsoft announced that Mistral Medium 3.5 and Mistral OCR 4 are now live in Microsoft Foundry, with Medium 3.5 also integrated into Microsoft Copilot Studio.

Among the models coming to Azure AI Foundry is Mistral Medium 3.5, an open-weight model with 128 billion parameters and a 256,000-token context window, which could indicate it’s better suited for applications that need to work across large documents or extended conversations.

Microsoft is also adding Mistral OCR 4, which is designed for document-heavy AI workflows. The model can process documents in 170 languages and preserve information about page layout, including bounding boxes, document structure, and confidence levels.

The model can process documents in 170 languages and preserve information about page layout, including bounding boxes, document structure, and confidence levels

Air-gapped deployments go mainstream 

For developers, the primary technical win is architectural symmetry. Engineers can build, test, and fine-tune applications inside Microsoft Foundry and migrate those workloads across public Azure, Azure Local, or sovereign infrastructure operated by Mistral without having to refactor underlying workflows.

This hybrid portability targets industries bound by strict compliance regulations. The deployment options also include fully air-gapped environments, allowing organizations to run AI workloads on networks that remain completely isolated from the public internet.

Deployment flexibility drives decisions

The partnership indicates that deployment flexibility is becoming part of the buying decision. Many organizations want the freedom to run AI in the cloud, on-premises, or in regional infrastructure without being locked into a single approach. And that’s where Microsoft’s expanded relationship with Mistral fits so well.

The companies are betting that giving customers more deployment options will weigh just as much as offering competitive models. If the strategy works, Microsoft will be providing the control plane for how the world’s most regulated industries run it. 

If the strategy works, Microsoft will be providing the control plane for how the world’s most regulated industries run it. 

The post Microsoft is building an AI stack it doesn’t fully own — on purpose appeared first on The New Stack.

OpenAI launches the ChatGPT for Small Businesses program, helping entrepreneurs build AI skills, automate work, and grow with ChatGPT Work.

Integrating GenAI with VoIP can deliver cost savings and efficiency through automated call routing, real-time translation, advanced analytics and intelligent chatbots.

The company behind the disk space analyzer TreeSize has irked some users by no longer offering support or updates for perpetual licenses beyond their maintenance period unless customers subscribe. Further frustration has come from JAM Software's long-standing policy of not providing license keys or installers to TreeSize perpetual license holders after that support period ends.

Since 2025, JAM Software has been transitioning most TreeSize editions to subscription models. Today, it sells perpetual licenses only for personal use, which include 12 months of updates, support, and “downloads of older versions, and your license,” plus the option to extend the support period. TreeSize currently has "no plans to discontinue the sale of perpetual licenses for TreeSize Personal," product manager Hendrik Christ told Ars Technica.

As perpetual-license maintenance periods expire, customers are discovering that extending support now generally requires subscribing to software they already own the right to use.

Read full article

Comments

© TreeSize

What began as discrete AI model training and human-facing chat interfaces has evolved into always-on AI factories dedicated to producing intelligence at scale....

What began as discrete AI model training and human-facing chat interfaces has evolved into always-on AI factories dedicated to producing intelligence at scale. These factories are now tasked with powering agentic workflows that reason, plan, use tools, verify intermediate results, and execute complex multistep tasks across vast contexts. Agentic workloads are not defined by a single prompt…

Source

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token falls, communication increasingly determines how efficiently models scale across thousands of GPUs. NVIDIA GB300 NVL72 set a world record for pre-training DeepSeek-V3 671B at 1,648 TFLOPs per GPU, showing how advances across the entire AI…

Source

Nature Machine Intelligence, Published online: 21 July 2026; doi:10.1038/s42256-026-01254-4

Lin et al. introduce a brain-inspired generative model that provides two key features of intelligence: planning and problem-solving. It uses cognitive maps, stochastic computing and compositional coding, and requires only local synaptic plasticity.

Tunix is Google’s new JAX-native post-training library designed to eliminate TPU idling bottlenecks when training multi-turn, tool-using LLM reasoning agents. It maximizes hardware throughput by combining highly concurrent, asynchronous rollouts with a decoupled producer-consumer pipeline, ensuring the trainer is constantly fed even while agents wait on network I/O or environment steps. Additionally, Tunix provides plug-and-play abstractions and continuous macro-level profiling, allowing developers to easily integrate custom open-source environments and optimize complex distributed workflows without massive code rewrites.

Agentic AI shifts more of the critical execution path onto the CPU. Agents operate in sandboxes to execute code, invoke tools, retrieve context, interact with...

Agentic AI shifts more of the critical execution path onto the CPU. Agents operate in sandboxes to execute code, invoke tools, retrieve context, interact with databases, and analyze results before returning information to the model. As these loops run concurrently across an AI factory, CPU performance increasingly shapes both per-agent responsiveness and overall factory throughput.

Source

The fast pace of AI research means organizations now have a wide range of models to choose from that can power AI agents to solve real business problems. But choosing a model doesn’t guarantee you effective agents or even good performance. For that, you need to run your agents in an environment that provides them with tools, state, security, and scale, with fast startup times and good integration with your existing business systems.

Picking the right agent runtime environment is like picking an enterprise app server but for AI systems — and agents have very different needs from traditional applications.

Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs.

Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs.  Agents that deliver in proof-of-concept systems will fail in production if the runtime stack powering them can’t keep up and keep them under control. 

Agentic compute is different

It’s easy to think of an AI agent as just another microservice that takes unstructured input, runs APIs or queries, and returns messages. But infrastructure designed for traditional enterprise applications with predictable business logic or even cloud-native stateless workloads doesn’t fit agents with their bursty, long-running, stateful, non-deterministic, code-writing, tool-invoking behaviors that might be triggered by a system event or an email — not just a chat session.

Model inference needs GPUs for speed, but agents also need reliable, durable compute that supports stateful sessions for long-running processes, along with strong security and real-time visibility.

You still need to think about familiar issues like hosting, scaling, identity, and security, but all that is complicated by the unpredictable, multi-stage workflow of the agent reasoning loop.

An agent pulls in input from multiple sources, reasons over its context about the execution plan for accomplishing the goal, calls other tools or writes its own code, iterates over the results of those calls, and maybe builds on them or switches to another approach that requires another reasoning loop and eventually delivers output. That might be updating a system or sending an email rather than just displaying an answer.

Model performance is only one part of making that useful. Architecting a successful agent system that can run at enterprise scale requires considering the agent runtime, the application layer that agents call, and the tools, APIs, and MCP servers they consume.

---

More on Microsoft Azure

• How to secure Kubernetes in the age of AI workloads
• In the AI age, Java is more relevant than ever

---

You have to be able to integrate with business logic and existing systems, manage the usual quotas, rate limits and SLAs for APIs so agents don’t overload them — and you have to do all that while keeping up with AI developments that are moving too fast for you to build the infrastructure primitives you need to rely on from scratch every time.

Requirements of modern agentic infrastructure

Instead, you should look for flexible infrastructure that fits the way the agent works. As with any technology, if you build on an existing platform like Azure Container Apps, you can save effort in areas where your business can differentiate. And while AI agents have flaws (from hallucinations to high token costs) that aren’t fully solved, you can pick an agent runtime environment that makes it easier to get useful results despite them.

Agentic compute needs fast startup and resume. Whether it’s a human typing into a chat prompt or system events automatically launching multiple agents, agent infrastructure needs to spin up quickly. If it takes a few seconds to spin up a container, the reasoning loop can’t start till that’s done.

It also needs to scale up and down responsively, without costing you anything when it’s not running.

Because agent workloads are long-running and event-driven, agents need to start up, do some work, go idle for hours or even days, and then resume instantly with their memory, context, caches, connections, identity, and security intact. That means persisting and restoring state so long-running agents don’t have to pay the same startup tax over and over again. Whether the last stage of the agent’s reasoning loop was successful or a failure, it has to take another approach; you don’t want it to do the same work again.

Rather than building your own custom microVM stack, the new Azure Container Apps Sandboxes provide a temporary, secure, and stateful compute environment that spins up, executes code, snapshots disk and memory, then automatically idles and resumes just as fast. This is the stateful equivalent of Dynamic Sessions, with sub-second startup from pre-warmed pools; you can burst out to hundreds (and eventually thousands) of concurrent sandboxes when you need them, then scale back down to zero.

An agent runtime needs to be secure by default because agents are only useful when they take action — and by definition, they’re likely to do unexpected things, write and execute their own untrusted code, and keep trying to achieve their goal (sometimes even when there’s a policy that should stop certain behaviors).

To give agents secure, auditable access to the resources they need, run them and the code they generate in a sandbox, rather than on a developer laptop with production credentials and admin rights. Identity, access control, and execution boundaries have to be enforced at the runtime layer, not bolted on in a hidden prompt.

ACA Sandboxes have egress and access policies, so you can control outbound calls and limit what URLs they can access. If you can’t use managed identity for all the services an agent needs to connect to you, you can protect secrets by injecting API keys through an external egress gateway instead of hard-coding them. Because the gateway is external, outbound call decisions are controlled by policy outside the sandbox, not by agent code that will relentlessly try any method to get through.

If agents are useful, you’re going to run a lot of them, often simultaneously. AI agents need strong isolation for each task, so every untrusted code execution happens in its own sandbox and no data leaks between tasks.

You can manage them as groups, but each ACA Sandbox has its own secure boundary, so details from one customer support agent won’t end up in a chat with a different customer. If the untrusted code does turn out to be problematic, isolated sandboxes at least contain the blast radius.

Manage the tools agents call

You don’t want to rely on fragile glue code or ad hoc orchestration that you have to rewrite any time systems change for the calls agents make to tools, APIs, cloud services, and other workflows. You also don’t want to rely on manual cleanup of resources no longer needed. A runtime with built-in agent tool execution makes agents more robust. It takes the drudgery out of connecting the agent environment to your other systems, enabling them to interact securely, in isolation, and at scale.

“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take.”

“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take,” points out Vyom Nagrani, who runs the team of PMs responsible for both Azure Container Apps and the Azure SRE Agent that’s built on ACA Sandboxes. Sandboxes have access to a connector framework with over 1,400 enterprise-grade connectors, enabling them to take actions not only on Azure and Microsoft services but also on third-party tools.

“The runtime provides those connectors, and it provides a managed way of authenticating against all of these third-party systems, so now you can build an agent which can talk to many, many, many different systems. It’s not boxed into one authentication boundary,” Nagrani tells The New Stack.

MCP servers are a built-in capability of the connector framework. “You can take any of these connectors; you can take any REST API and expose it as an MCP server, which then the agent can consume.” Or if you want to build a custom MCP server, you can host that in a sandbox too. ACA Sandboxes can be both where agents run and, if that’s appropriate, where the tools they use are hosted.

If you’re building a user interface to wrap your agents, the application layer that makes calls to the agents can run in Azure Container Apps Express, a new service now in public preview, Nagrani says. 

“It’s a simplified app hosting stack for the app layer that responds to HTTP traffic and serves web traffic: that’s where the human interactivity comes in.”

Putting it all together

ACA Sandboxes offers an agent runtime environment that answers the key questions architects need to consider: where agents and the ephemeral compute they need access to run; where the application layer that calls agents runs; and how agents get access to all the tools, APIs, services, MCP servers, and existing business logic they need to orchestrate.

Whether you’re a platform engineer, a software vendor or a startup building a new AI platform, treating agent runtimes as the new application server and MCP servers as the new APIs requires agent infrastructure that supports agent workloads effectively, allowing governance to shift left into the runtime layer where it can scale with the ever-increasing numbers of agents. 

How real platforms build on agent runtimes

Azure Container Apps is already a strong platform for running agents. Auger, a startup launched by the former CEO of Amazon’s global consumer business to help mid-size enterprises get their complex supply chains out of Excel spreadsheets, used it to build a multi-agent system that can give real-time answers about shipments and forecasts in a world where mines or critical shipping lanes might be closed at any time by war or weather. 

Backend agents pull the unstructured data with all the details to answer those questions from different siloes, building ETL pipelines and creating an ontology of the supply chain ecosystem for each customer that includes functions and actions — all of which need to be audited and reversible. Frontend agents use that ontology to answer questions such as, “What happens if I build a new warehouse here or switch to a supplier in this country?”

ACA Sandboxes are built on Azure Container Apps, giving organizations an easier way to create agent services with strong isolation, dynamic scaling, fast startup, persistent state, and broad tool and service integrations.

That’s just what the Foundry team was looking for when they started work on the Microsoft Foundry Agent Service managed agent runtime. ACA Sandboxes gave them a platform that delivers fast start/resume, built-in tool execution (including for untrusted code), persistent state for long-running agents, strong per-agent isolation, and secure-by-default operations. Agent identity, “on‑behalf‑of” authentication to existing business services, and strict isolation are critical capabilities they didn’t have to build themselves.

“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”

The Foundry Agent Service adds a layer of visibility and observability into agent actions and conversations, allowing customers to monitor agent performance and see where the system is doing well and where it needs improvement.

“Azure Container Apps Sandboxes package the hard parts of an agent runtime into a first-class Azure resource — sub-second start and resume, built-in execution of tools and untrusted code, and snapshot-based state that lets long-running agents pick up exactly where they left off. Every agent gets its own hardware-isolated environment with secure-by-default operations, so builders can focus on what their agents do, not on the plumbing underneath,” Nagrani points out.

“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”

The post The rise of the agent runtime: The compute platform behind production agents appeared first on The New Stack.