Multi-tenancy has moved in one direction for 60 years: the tenant keeps getting smaller. Mainframe time-sharing carved a single machine into slices so an organization’s departments could share it, and the tenant was the org. Virtualization gave each team its own fleet of virtual machines, and the tenant became the team. Containers and Kubernetes namespaces shrank it again, until a platform team could hand every developer an isolated environment on a shared cluster.
That last step, an environment per developer, became the target state of platform engineering in the 2020s. A namespace per developer, capacity planned by seat, golden paths sized to headcount. Underneath all of it sits one assumption: a person produces one stream of work at a time, so isolating people isolates work.
Coding agents broke that assumption. A developer running five agent sessions has five changes in flight at once, each needing its own working version of the system. Anthropic’s engineers, building a C compiler with a fleet of parallel agents, ran nearly 2,000 Claude Code sessions across two weeks. Cursor’s documentation tells developers to run as many agents as you want in parallel. None of those concurrent workstreams is a person.
“The tenant has shrunk one more time. It is no longer the developer. It is the change.”
The tenant has shrunk one more time. It is no longer the developer (or even the agent). It is the change.
Tenancy demand scales with changes in flight, not headcount
Capacity planning by seat worked because changes arrived at human pace, roughly one per developer at a time. That denominator is gone. A Microsoft study of command-line coding agent adoption found that developers merged roughly 24% more pull requests over four months, and merged pull requests understate the pressure. Every change that reaches merge is preceded by iterations and abandoned attempts, and each of those also needed somewhere to run.
Run the seat math against the change math. A 50-developer organization where each engineer supervises a few agent sessions has hundreds of changes in some stage of validation on a busy day. Each one wants data it can migrate and write to without asking permission, its own view of shared message topics, and a running version of the services it touched. That is the demand of a 300-person (or more) engineering org on a 50-tenant platform.
Every layer built on the person-tenant assumption misprices this. A per-developer namespace hands one tenant slot to what is now five concurrent workstreams. Shared staging serializes all of them into a single queue. Seat-based capacity plans budget for the number of employees while the bill tracks the number of changes in flight.
The new tenant is the change, not the agent
The tempting candidate for the new tenant is the agent, and it is the wrong one. Agents are interchangeable workers. Two agents can collaborate on one change, one agent can rotate through five changes, and a crashed agent gets replaced mid-task without anything downstream noticing. Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.
“Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.”
The durable unit is the change. It comes into existence when work on it starts. It accumulates state that no other tenant should see: a schema migration, test writes, new versions of one or two services, the messages it produced during validation. It needs to observe a version of the system that includes its own edits and nobody else’s. And it is torn down when it merges or is abandoned, taking all of that state with it.
Naming the change as the tenant turns a vague scaling problem into a design target, because change-level tenancy has three requirements that person-level tenancy never had to meet:
Creating a tenant must be near free.
Isolation must cover only what changed.
The tenant’s lifecycle must be bound to the change itself, not to a ticket or a timer.
Platform teams already run this playbook in production
The discipline these requirements call for is not new. Anyone operating a multi-tenant production service already knows the rules: tenants share the substrate, each tenant privately owns only what makes it distinct, creating a tenant is self-service and cheap, and a tenant’s resources are reclaimed the moment it leaves. Nobody stands up a private copy of the product per customer, and nobody files a ticket to onboard one.
Those same organizations run pre-production on the opposite rules. Environments are provisioned by ticket or by seat, capacity is planned per person, and isolation is achieved by duplicating the stack when it is achieved at all. The multi-tenancy playbook that runs the product has never been applied to the platform that builds the product.
Change-level tenancy is that playbook, applied. Treat every change as a tenant of the development platform, and the three requirements stop being novel. They are the standard properties of any competently run multi-tenant system.
The tenant owns what changed and shares everything else
A SaaS tenant owns its data and configuration, never a copy of the application. A change tenant is sized the same way. It owns the one or two services it modified and an isolated database branch it can migrate and write against, and nothing else. Everything the change did not touch resolves against one shared stable environment, continuously deployed from main, so every tenant validates against real, current dependencies without owning a copy of them.
A footprint that small makes tenant creation nearly free, and creation cost is what decides whether the model scales to agent demand. Isolated data no longer requires copying a database: Neon and Xata create copy-on-write branches in seconds regardless of dataset size, consuming storage only for the data that diverges. The runtime side costs one deployment, because starting the modified services is all that is left to do. A tenant that costs one deployment can be created hundreds of times a day.
Tenants onboard and offboard themselves
Multi-tenant platforms scale because nobody provisions tenants by hand. Signup creates the tenant, cancellation removes it, and no operator sits in the loop. Change tenants need the same contract. The tenant comes into existence when work on the change starts and disappears when the change merges or is abandoned, with no ticket at the front and no cleanup script at the back.
Offboarding is the half that platform teams underestimate. At person scale, an orphaned environment was a minor waste found in a quarterly cleanup. At change scale, orphans accumulate as fast as agents abandon experiments, and the leak outgrows the cleanup.
Automatic offboarding also keeps the accounting accurate. When tenants are created and destroyed by the change’s own lifecycle events, the number of live tenants equals the number of changes in flight, and platform capacity becomes a quantity you can measure and plan against instead of a pile of environments nobody is sure anyone still uses.
Re-measure the platform in changes, not seats
The practical shift for platform teams starts with measurement. Count changes in flight at peak, not seats: open pull requests with activity in the last day is a fine proxy, and for most teams the number is already several times headcount. Then price the marginal tenant: What does one more concurrent change cost in dollars and in minutes of setup? If the answer is a full environment and tens of minutes, the platform is still doing person-level tenancy.
Those two numbers expose where the old assumptions live. Namespace quotas sized per developer, staging booked by team calendar, database seeds refreshed nightly for everybody at once: each is a seat-denominated policy waiting to fail under change-denominated load. The fix in every case is the same three requirements: near-free creation, isolation sized to the change, lifecycle bound to the change, applied to whichever part of the platform still assumes the tenant is a person.
Change-level tenancy is the prerequisite for an agent-native SDLC
Every previous definition of the tenant named a person or a group of people, and that held because only people produced changes. A platform could equate one seat with one workstream, plan capacity from the hiring plan, and keep a human in the provisioning loop. Coding agents break all three of those properties at once: one person now operates several concurrent workstreams, those workstreams are created and abandoned at machine pace, and no human is positioned to provision or clean up each one.
“The change is the only unit of isolation that stays stable when the workers become software.”
That is why the software development lifecycle (SDLC) needs its tenant redefined around the change rather than around whoever, or whatever, wrote the code: the change is the only unit of isolation that stays stable when the workers become software.
Organizations that keep person-sized tenancy will watch agent-generated changes queue behind infrastructure built for a fraction of the load. The ones that re-platform around the change will convert agent throughput into merged work. If you’re exploring the second path, that is exactly what we built Signadot to support.
Mukta set out to examine where state-of-the-art memory management sits today in a world where (as she put it) “context is often orthogonal to the model intelligence” at hand.
“The newest model we’ve just released isn’t going to go out of the box and know exactly what it takes to succeed in your organization and what tasks you want it to do,” said Mukta. “It’s like agents [initially] not knowing their way around a codebase or knowing enough about your own user preferences.”
To steer agentic services the right way, systems obviously need access to memory to create a context window.
A brief history of Anthropic memory management
Providing a brief history of Anthropic memory management, Mukta said that traditional approaches made use of CLAUDE.md, a file that Claude reads at the start of every conversation (that includes Bash commands, code style, and workflow rules) to give Claude persistent context that it can’t infer from code alone.
Effective to a degree, this technique becomes hard to manage over time, especially when a file with very important preferences gets very, very long.
“So a second avenue that we investigated was memory tools, and this is interesting because it leans into the idea of what happens if we let agents autonomously manage their own memory systems? We let them decide when they read, when they write, and when they update memories,” explained Mukta.
This process happens in-band i.e. within the context of a session. When dovetailed with so-called progressive disclosure, the agent only looks at the light metadata at Layer 1, before reaching for full content and original source files in Layers 2 and 3, respectively, so that the system doesn’t overload the model’s context.
“The way I like to think about it is as if I’d had a bookshelf in my room, and every time someone talks to me, I can kind of scan and look at my list of books and see if any of the titles might be relevant to the conversation, and then pick that off the shelf and read it when I need to,” explained Mukta.
But the bottleneck here is that we’re still driven by humans and agents working together i.e. we’re still being quite opinionated about what things need skills. The additional problem here is that memories can go stale and become irrelevant to an organization’s needs. Add the fact that a memory file may be written incorrectly or even maliciously injected and you can see why a lot of guardrails need to be in place.
“We introduced the concept of dreaming, which is a process that runs asynchronously in batch with its own allocated resources, to ensure that memories themselves are effective, up to date, and [so we can] help the agents learn over time.”
Dreaming consolidates memory & cuts irrelevance
“So we introduced the concept of dreaming, which is a process that runs asynchronously in batch with its own allocated resources, to ensure that memories themselves are effective, up to date, and [so we can] help the agents learn over time,” explained Mukta. “[This process allows us] to consolidate memory and cut things that are no longer relevant, add things that agents are missing, and clean up and organize memory systems.”
In Anthropic’s world of slumber, dreaming is an out-of-band asynchronous process which the organization says solves the in-band limitation, where agents must split effort between completing and executing tasks, while also concurrently curating memory for their future selves. Dreaming spots recurring failure patterns where agents are consistently failing (wrong units, missing topics, broken tool configs, stylistic tics like overused em dashes), and proposes memory-store updates, again for human review, but hopefully at a more effecient level.
This architecture underpins Anthropic’s Managed Agents memory and API approach at this level, so has the frontier model company won over developers?
Bad memories can outlive sessions
Staff software engineer, cloud architect and independent researcher in AI agent systems, Jayakumar Ramalingam, tells The New Stack that “dreaming is useful, but it also creates a dangerous promotion path” i.e. one that leads from repeated mistakes to persistent policy.
“A bad answer normally dies with the session; a bad memory can influence thousands of future sessions. Human review sounds reassuring, but at fleet scale it can easily become a rubber stamp for recommendations nobody has time to reconstruct,” Ramalingam says.
“The industry has spent too much time treating memory as a context window problem when it is really a state management problem.”
He insists that every proposed memory should “carry provenance, evidence and an expiration condition”, and not just exist as a pattern that recurred often enough to look real. Otherwise, he thinks that dreaming may help agents remember more while making organizations forget why the memory was trusted.
“Anthropic is getting one important thing right: its agent memory should look more like versioned infrastructure than artificial cognition. The industry has spent too much time treating memory as a context window problem when it is really a state management problem,” underlines Ramalingam.
His point is – if an agent cannot show who changed a memory, why it changed and how to roll it back, it does not have production memory, so it becomes an unaudited configuration file with an AI attached.
Dreaming is the right instinct aimed at the wrong evidence
Enterprise AI architect and founder of Besk Tech, Vladimir Beskorovainyi, tells The New Stack that “dreaming is the right instinct aimed at the wrong evidence”, because the failures it catches (wrong units, broken tool configs, too many em dashes etc) are all visible on the surface of a transcript.
“The failure that actually costs you is an agent reaching for the wrong tool for a reason that looked perfectly defensible at the time,” Beskorovainyi says. “In the systems I run in production, the log records the decision rather than the API call, and that is the only reason a review pass like this finds anything worth finding.”
“When the ‘lately’ factor quietly becomes true. That leaves us at a point where versioning tells us what changed and when, not what is correct.”
He points to what he calls “a worse problem underneath the agent’s decision” i.e. if updates are proposed from recent batches, the memory store drifts towards whatever the agent fleet happened to do lately, and so the “lately” factor quietly becomes true. That leaves us at a point where versioning tells us what changed and when, not what is correct.
“The industry spent two years insisting that memory meant embeddings, and Anthropic solved it with a filesystem and grep [a Linux command that searches for patterns in files] and that is the most interesting decision in this whole discussion,” insists Beskorovainyi.
He says the reason it matters is legibility. A memory store a developer can open and read is a memory store an engineer can audit, and (he insists) “no vector database has ever offered that”, while everything else in the architecture (the versioning, the hashes, the tiered permissions), is ordinary distributed systems engineering we have known how to do for decades.
Dreaming is the clever (but worring) part
Founder of autonomous AI penetration testing company Penetrify, Viktor Bulanek, tells The New Stack that when the industry spent two years convinced that agent memory was a vector database problem, and Anthropic shipped grep, that was a useful thing.
“In terms of what Anthropic is getting right… a memory store you can cat, diff and code review is one you can actually operate, whereas nobody has ever successfully debugged an embedding that quietly ranked the wrong chunk third,” Bulanek says.
“Anthropic’s approach to dreaming is the clever part and also the part that worries me most, because it points an automated writer at session transcripts, and transcripts are full of content the agent did not author.”
He thinks that the versioning matters here far more than the auditability framing suggests and reminds us that “rollback is not a compliance feature”; it is the undo button for a poisoned memory a software engineer discovers three weeks after it was written, which is the incident every serious agent deployment is going to have eventually.
“But to add balance here, Anthropic’s approach to dreaming is the clever part and also the part that worries me most, because it points an automated writer at session transcripts, and transcripts are full of content the agent did not author,” Bulanek cautions.
“Anthropic is right that human review is the answer, but bulk review of proposed diffs is exactly the control that decays fastest once the suggestions are mostly good. The other gap is that nothing in this architecture says when a stored fact stops being true. Versioning tells you what changed, it does not tell you what rotted, and a confident note about a system that was refactored last month is worse than no memory at all,” he advises.
Bulanek’s work sees him run autonomous agents in production that perform penetration testing and run for hours unsupervised with real credentials against live systems, so memory for his team is both an operational cost and a security boundary at the same time.
The Anthropic way of doing things has an endearing lack of flair to it
Co-founder and CTO of Noah Labs, Berk Yilmaz, tells The New Stack that the Anthropic way of doing things has “an endearing lack of flair to it” in his view.
“Everyone wants memory to feel like the newest incarnation of machine intelligence, and their pitch goes something like: just give it a filesystem, versioning, searchability, and don’t let a thousand processes stamp all over each other,” Yilmaz says. “This is closer to how production AI should be done. While we have spent a long time improving models, the supporting infrastructure has not kept up, failing in incredibly prosaic engineering ways.”
Yilmaz is behind a company that develops an AI-native IDE for government and regulated systems, built for air-gapped environments and legacy codebases. He reminds us that once a memory decision is made on which past behavior should become future behavior, memory itself ceases to be inert.
“A hallucination that dies after a single session is a pain in the neck, but a hallucination that outlives a thousand sessions is infrastructure. The same thing applies to security; if an attack succeeds in writing to memory, it has become persistent. Provenance becomes absolutely critical here, how was the system taught this, where did it learn it from, who certified it, and can I undo it? In enterprise AI, sometimes forgetting is a safety measure,” adds Yilmaz.
A pragmatist would remember that Anthropic gets paid for usage, not efficiency
AI, product & data science leader and former Meta employee, Kerstin Frailey, tells The New Stack that at face value, dreaming (for her money) “certainly sounds like it has the potential to blow up AI bills” right now.
“A cynic would say this is designed to fill the revenue hole left by tokenmaxxing before Anthropic’s IPO,” Frailey says. “An optimist would hope for a beautifully thrifty design. A pragmatist would remember that Anthropic gets paid for usage, not efficiency. A skilled practitioner would run incremental pilots, aggressively monitor costs, and routinely test for measurable improvements.”
“As a nice bonus, dreaming offers potential system improvement, too. But its familiar predecessors – garbage collection and storage compaction – are comparatively deterministic and controlled.”
She continues and notes that dreaming offers cleanup and consolidation, which she defines as a “reasonable development” for any system that constantly generates new files.
“As a nice bonus, it offers potential system improvement, too. But its familiar predecessors – garbage collection and storage compaction – are comparatively deterministic and controlled. Unlike its namesake or those analogues, dreaming appears neither cheap nor efficient: pay an AI to do the work once, then pay AIs to regularly review, revise, and restructure it,” she adds.
Dreaming as part of Anthropic’s Managed Agents memory and API approach isn’t alone. The notion of AI model dreaming (or automatic out-of-band background memory consolidation if we’re being formal about things) is also being popularised by OpenAI for ChatGPT, in stateful agent coding platform Letta and elsewhere.
The bottom line here may be a realization that, in AI modeling terms at least, memory is actually maintenance.
Databricks on Tuesday announced that it’s acquiring Electric, the startup behind the WASM-based Postgres project PGlite and the Electric sync engine, as agentic applications change how developers use databases.
The Electric team will join Neon, the serverless Postgres company Databricks acquired for about $1 billion last year and the foundation of its Lakebase database service.
The companies didn’t disclose the terms of the deal.
What Databricks bought
PGlite is a complete Postgres database in WebAssembly (WASM). It runs in the browser, a Node.js process, or inside the kind of sandboxes agents use to execute code. It supports dynamic extension loading, including pgvector, the preferred Postgres vector extension.
According to the companies, PGlite has grown from 1 million to 13 million weekly downloads over the last year.
The sync engine at the core of Electric
It’s the Electric sync engine that is core to Databrick’s interest in Electric, though. This engine keeps a central Postgres database that can then be synced in near real-time with browser tabs, mobile apps, or agents. As Databricks notes, this is the multiplayer model of Figma or Google Docs, but applied to Postgres and the agents that use it.
The Neon team, in its own announcement, notes that “complex problems like conflict resolution, partial replication, and reconnection logic make real-time sync difficult to build from scratch.” Hence why Databricks likely acquired Electric instead of trying to build this from scratch itself.
As for the future of Electric, the company’s founders James Arthur and Valter Balegas write that “everything we’ve previously open sourced stays open source.” This covers the sync engine, PGlite, Durable Streams, and TanStack DB.
What doesn’t survive the deal, however, is Electric’s hosted service. “Electric Cloud is winding down,” the founders. “Cloud users will need to self-host or move to another provider.”
The deal also extends a string of database acquisitions for Databricks that includes Neon itself and, more recently, the transactional processing startup Mooncake.
A database that lives for 10 seconds
As the Databricks team argues, traditional non-agentic applications share one database among many clients, and that database is the most permanent piece of the stack. But agent workloads change this.
In a recent post on how agentic development changes databases, Databricks’ Ippokratis Pandis, Nikita Shamgunov, and Reynold Xin write that agents now create roughly four times more databases than human users do on Lakebase. They also stress that the average project now carries about 10 database branches, and that some projects run more than 500 branch iterations deep.
For some types of applications on Lakebase, the average database compute is now alive for under 10 seconds.
Agents, as it turns out, like to branch databases the way they branch code, a pattern Neon built its architecture around.
In practice, a coding agent spins up a sandbox, instantiates PGlite inside it, builds and tests against the database, and then either throws the whole thing away or syncs the result with — in the Databricks context — a Lakebase branch. Because Lakebase separates storage from compute and keeps its data in Postgres page formats on object storage, creating that branch is a relatively cheap copy-on-write metadata operation.
“As coding agents drive the cost of creation to zero,” the Neon team writes, “the number of applications explodes, and most of them are small.” A database server, even a serverless one that scales to zero, imposes a floor on what the smallest viable app costs to run. “You can’t have an age of abundance if every app requires a fixed minimum of compute,” the post argues.
‘Two halves of the same idea’
It’s worth noting that PGlite didn’t start at Electric. Instead, it began as an experiment by Neon co-founder Stas Kelvich, who compiled Postgres to WASM to see whether it could run client-side. Electric picked the work up and turned it into a production project. “That repo became the basis of PGlite,” Arthur and Balegas write.
As Databricks’ announcement notes, this now “reunites two halves of the same idea.”
Cloudflare launched its CloudflareOS open-source AI workspace platform this week, promising every employee a secure workspace equipped with AI tools and access to internal company systems.
Positioned significantly beyond the notion of legacy virtual desktop infrastructure (VDI) services, which delivered the same fixed applications through a remote screen — and even past the dynamic application delivery, app masking and streaming of modern VDI iterations — this is an essentially more dynamic way of working with internal company tools, documents and systems.
Cloudflare’s CloudflareOS makes its apps and services accessible through secure connection points that verify every user and every agentic request or connection point before access is granted.
In AI, every new work session starts from zero
The technology proposition here is built on the fundamental truth that the typical enterprise AI tool knows a great deal about the world, but almost nothing about how a specific company operates, the shape of its internal systems, approval processes, or the ways teams actually get work done.
That means every new work session starts from zero, with employees re-explaining context the AI should already know. But how can new business context-aware agentic access freedoms be granted securely?
Rita Koslov, VP for developers & AI at Cloudflare, tells The New Stack that powering up modern agent use cases means “data is often leaving controlled systems en masse” for the first time.
“It used to be the case that, for example, people asked analytics questions in the data warehouse where the organization had control,” Koslov says. “Now, employees are asking for API keys for their own tools, agents, etc. This creates a new class of security problems that Cloudflare OS helps to solve.”
Capability-based access beats handing agents raw API keys
Cloudflare has built what we can call capability-based access, which the company promises beats handing agents raw API keys outright.
“API keys give agents broad access to systems; a capability-access-based approach lets us grant one specific resource, then record exactly what the agent observed, and verify that anyone who sees its work is also allowed to access the source,” underlines Koslov.
Cloudflare OS enables an agent to create documents, slides, spreadsheets, workflows, other agents – or entirely new full-stack applications – all tailored to an employee’s work. What it creates can remain connected to live data sources, be modified and shared safely, and be used directly by both people and agents.
“API keys give agents broad access to systems; a capability-access-based approach lets us grant one specific resource, record exactly what the agent observed, and verify that anyone who sees its work is also allowed to access the source.”
In terms of how developers and systems operations professionals should react to this offering, Koslov suggests that “the difficult problem is not generating an app” today. Instead, the real challenge is safely running thousands (or millions) of dynamically generated apps, each with persisted state and controlled access.
“Cloudflare OS uses Dynamic Workers, which provide lightweight isolated runtimes to load each app’s code on demand, and Durable Objects Facets to give it isolated SQLite storage under the platform’s supervision. Outbound networking is disabled by default, and Gatekeepers expose only the resources explicitly granted by the users,” Koslov says. “Dynamic Workers and Durable Objects Facets were invented because doing this was previously not possible.”
For completeness here – and once again a Cloudflare original technology service – a Gatekeeper is a service-specific Worker that sits between Cloudflare OS and an external service to interpret and understand the service’s API, its resources, and the operations that can be performed on them.
What happens when it all goes wrong
Koslov confirms that she knows how badly things can skew out of control in unmanaged environments.
“We know this from our own experience talking to other companies on all accounts. They’ve shared instances of internal data copied into AI tools that IT did not know were in use, AI keys embedded into agent-built applications, and even data being shared internally to people who ordinarily wouldn’t have access (or even publicly),” she adds.
Building a tailored alternative is no small project; a platform with proper security and real integration into internal systems can take years to develop and cost millions to maintain. In the meantime, employees find workarounds, IT loses track of which AI tools are running and who is using them, and costs pile up, often with little to show for it.
CloudflareOS starts from a different premise: a company captures its knowledge, processes, and ways of working once in a form AI can actually execute, and that knowledge travels with every employee’s workspace from day one.
How do we measure business ‘context’?
“Captured business ‘context’ in this case can include company terminology, policies, operating procedures, product documentation, technical standards, sales processes, templates, and established ways of performing recurring work,” confirms Koslov.
CloudflareOS started as the platform Cloudflare built to run its own workforce. Thousands of Cloudflare employees across every team use it daily to perform research, create documents connected to live data, automate repetitive tasks, and build working apps for their day-to-day jobs.
That same platform is now available to any organization as open-source software. Because it’s open source and runs in a company’s own Cloudflare account, organizations own what they build on it.
The platform itself works on any AI model and controls cost. Through Cloudflare AI Gateway, organizations can use any AI model provider, so they’re not locked into one vendor. Administrators see exactly what’s being spent, broken down by person, team, or app. They can set spending budgets, rate limits, or route routine tasks to smaller, more affordable models where a top-tier model isn’t needed.
Pricing platforms by the token is the wrong meter entirely
Cautiously upbeat about the wider story playing out here, enterprise AI architect and founder of Besk Tech, Vladimir Beskorovainyi, tells The New Stack that, traditionally, the industry is pricing these platforms by the token, “and that is the wrong meter entirely” in his view.
“In this example with Cloudflare OS, what a company actually buys here is the obligation to write down how an AI-powered business process really works, and then keep that description true as the business shifts underneath it,” Beskorovainyi says. “The model is the commodity part. What costs real money is the curated context, and nobody budgets for the fact that it starts decaying the day it is written, which is exactly what decides whether any of this survives contact with production.”
“Cost broken down by person, team and app is the first time I have seen a vendor treat spend as an engineering signal rather than an invoice, and sending routine work to a smaller model is the obvious next step that most enterprises still fail to take.
Beskorovainyi insists that the organizations that win in this game will “not necessarily be the ones running the best model”; they will be the ones that could “already answer in writing what their own approval process is”, way before an agent ever asked.
“Cost broken down by person, team and app is the first time I have seen a vendor treat spend as an engineering signal rather than an invoice, and sending routine work to a smaller model is the obvious next step that most enterprises still fail to take,” advises Beskorovainyi.
Owning your own context is not the same as your context being any good
He clarifies his point and explains that the qualification here is that “owning your own context is not the same thing as your context being any good”, and so open source tooling and community connections plus an organization’s own account settle who holds the context file.
“Neither tells us whether what is recorded and logged in the context file is still true this quarter. That work stays with the customer permanently, and it is where I expect most of these deployments to come apart, not in anything Cloudflare has built,” Beskorovainyi adds.
Matthew Prince, co-founder and CEO of Cloudflare has said that his team built Cloudflare OS, “because nothing else did what we needed”, and so now, any company can start from where it took the organization’s internal software engineering function years to get to.
The apparent appeal here must come down to the dynamic nature of Cloudflare OS and its ability to work with and apply AI tools at a custom-engineered business context-aware level with zero trust by default. The platform can turn any output into a working app with its own isolated database, real-time capabilities, and access controls – once agan, that’s not legacy virtual desktop is it?
No developer required (yet)
The bottom line from Cloudflare is that employees can use any app on Cloudflare OS directly, or adapt it for their own needs so that it’s a case of “no developer required”, or at least until the next integration task needs to be shouldered, or the big thing comes along, or both.
A developer supervising four coding agents has four changes in flight at once, each in its own git worktree. That isn’t an exotic setup anymore: Anthropic’s documentation now treats a worktree per session as the default way to run agents in parallel, and what was an expert workflow two years ago is the recommended starting point today.
The branches themselves aren’t new. Git made them cheap 20 years ago so developers could isolate changes and work on several things at once, but in practice a developer switched between branches and shipped one change at a time. That kept everything below the code layer singular: one continuous integration (CI) queue, one staging environment, one database everyone tested against. The number of changes contending for those shared resources was capped by headcount, and before agents, only larger teams ever hit the cap.
“Coding agents removed the cap. The branch can no longer stop at the code layer.”
Coding agents removed the cap. Those four branches are no longer something one developer rotates through. They are four active changes moving toward merge in parallel. The gap becomes unworkable: branching is free at the code layer and missing everywhere below it. Each change needs to exist all the way down the stack, not as a diff in a directory but as a running, testable version of the system. The branch can no longer stop at the code layer.
Parallel until the first shared resource
Code branches in milliseconds. A worktree gives each agent a private copy of the repository for the cost of a checkout, and 10 agents can work side by side without seeing each other’s edits.
The output shows up downstream. Telemetry from Faros AI across more than 10,000 developers found that teams with high AI adoption merge 98% more pull requests while review time grows 91%. Nothing downstream of code generation was sized for that arrival rate.
Then each change needs to run. There is one staging cluster, one seeded database, one message queue, one set of dependent services, and every branch that reaches this floor stops being parallel. Four agents produce four candidate changes in an afternoon, and all four line up behind the same shared environment to find out whether they work.
The queue is more expensive than it looks, because agents don’t wait well. An agent blocked on an environment either sits idle holding a stale view of the system or plows ahead validating against mocks, and the developer supervising it context-switches away. By the time the shared environment frees up, the cheap part of the work has to be partially redone.
The bottleneck isn’t code generation, and it isn’t review capacity alone. It’s the first shared resource a change touches, because a branch that can’t run is a branch that can’t be trusted.
“The bottleneck isn’t code generation, and it isn’t review capacity alone. It’s the first shared resource a change touches.”
A branch is a delta, not a copy
The way out is to stop treating branching as something git does and start treating it as something every layer does. Branch-based development names the pattern: each layer of the stack offers a cheap, instant, disposable branch primitive, so a change can exist end to end without duplicating anything it didn’t touch.
The mechanic is the one git established, and everyone has been living on for two decades: branches are cheap because they share everything unchanged and carry only the delta. The rest of the stack has been relearning that idea layer by layer ever since — share by default, isolate what changed.
Naming the pattern matters because each layer discovered it separately and called it something different. Worktrees, pipeline caching, preview deploys, database branching, and environment sandboxing sound like five unrelated features. They’re the same idea applied at five layers, and seeing that changes what you ask of the layers that lack it.
The upper layers learned this years ago
CI absorbed the lesson a decade ago. Every branch gets its own pipeline run on a shared runner pool, with build caches doing the copy-on-write work of reusing unchanged artifacts. Nobody provisions a build system per branch, and nobody queues behind a single global build anymore.
The front end followed. On Vercel, every push to a non-production branch gets its own preview deployment by default; Netlify works the same way, and the branch itself is one immutable build plus routing on shared hosting infrastructure. Reviewers stopped asking whether a change works on someone’s laptop, because the change is already running somewhere.
Both cases have the same shape: the expensive machinery is shared, the branch is thin, and creating one is cheap enough that nobody thinks about it. That’s what a layer feels like once it has a branch primitive.
Each of these primitives also changed behavior once it arrived. Per-branch CI made it normal to run the full test suite on every push instead of nightly. Preview deploys made it normal for a product manager to click through a change before merge. Cheap branches don’t just remove a queue; they raise the bar for what gets checked before merge.
The data layer was supposed to be the hard case
Databases carry state, so conventional wisdom said branching would never work there. Then Neon, PlanetScale and Xata shipped it anyway, and Neon’s documentation now makes the parallel explicit: branch your data the same way you branch your code.
A database branch is a copy-on-write view over shared storage pages, created in seconds regardless of how large the database is. Schema migrations and risky data changes get validated against production-shaped data instead of a stale seed script, and the branch disappears when the work merges.
“If the layer with the most state can hand out branches in seconds, statelessness was never the real requirement.”
The data layer matters to this story because it removed the best excuse. If the layer with the most state can hand out branches in seconds, statelessness was never the real requirement. Whatever is still unbranched is unbranched by choice.
The runtime is the last layer to learn the trick
The microservices runtime resisted longest because it looks nothing like a file tree. It has live traffic, a service graph and dozens of moving dependencies, and the naive branch, a full copy of the environment, is so expensive that most teams concluded branching did not apply here.
The copy-on-write move works anyway. Run one shared, stable version of the system that is continuously deployed from main. For each change, deploy only the services the change touches as a lightweight ephemeral environment, and route each test request through the changed services while everything else falls through to the shared stable versions. The environment branch costs roughly what the changed services cost, which is why one can exist for every change an agent produces.
Routing is the part that sounds exotic and isn’t. A request tagged with a label gets steered to the changed service versions at each hop, propagated through the call chain the same way trace context already flows through most instrumented systems. The shared stable environment plays the role of main, the changed services are the delta, and the label is the pointer that assembles a coherent view of the system per request.
Put the layers together and a different development model appears. An agent picks up a task, and the change gets a worktree, a pipeline run, a preview, a data branch, and a running environment from the start. Validation stops being the scarce resource that serializes everything upstream of it.
Teams are already composing the lower layers. Bitso, a crypto exchange with 250-plus engineers, pairs an environment branch with a database branch for each change, so the runtime delta and the data delta travel together and shared staging stays out of the critical path.
That end-to-end branch is what the phrase agent-native software development lifecycle should mean. Not agents wired into yesterday’s pipeline, but a stack where any change, human or machine, can exist at every layer for as long as validation takes and disappear afterward.
The payoff compounds with agent count. When the branch primitive at every layer is a delta over something shared, validation concurrency scales with cluster capacity instead of with budget, and the number of changes a team can prove correct per day rises with the number it can generate. That is the ratio that decides whether agent adoption shows up as shipped software or as a longer queue.
The audit is cheap to run. Follow one change from worktree to validated and note the first layer where it waits on something shared. That’s where your stack stops branching.
For most teams, the answer is the runtime, and if it’s yours, Signadot is a practical place to start.
Thibault Sottiaux, who leads core products at OpenAI, believes that today’s version of Codex will seem outdated before the year ends.
Sottiauxposted on X late Monday, “Given some of the results I’m seeing recently, it’s pretty clear Codex is a good harness.” He continued, “But it will seem primitive in 2-3 months and we’re about to go through another major evolution in how we use AI at the frontier.” He also said, “The next generation of models need more than your laptop.”
“It will seem primitive in 2-3 months and we’re about to go through another major evolution in how we use AI at the frontier.”
Given some of the results I'm seeing recently, it's pretty clear Codex is a good harness.
But it will seem primitive in 2-3 months and we're about to go through another major evolution in how we use AI at the frontier. The next generation of models need more than your laptop.
Sottiaux did not share details about OpenAI’s plans for the coming months. However, his comments are timely since the company is already working to move Codex beyond tasks limited to a developer’s computer. Since launching a new GPT-5 model for Codex in early July and surpassing 8 million users shortly after, the product has been evolving quickly.
Ona fills the infrastructure gap
In June, OpenAI said it plans to buy Ona, a company that creates secure cloud development environments. OpenAI called this deal part of the “next phase of Codex,” where agents can keep working in a customer’s cloud even after the laptop that started the job is closed.
“The next generation of models need more than your laptop.”
Codex currently uses cloud infrastructure, but it might still need the developer’s laptop to access projects and run tools. If the laptop goes offline, the agent may lose what it needs to keep working.
OpenAI has already tested this approach. In an experiment published in February, Codex worked for about 25 hours straight, used around 13 million tokens, and generated about 30,000 lines of code while building a design tool from scratch. Alibaba has pushed even further — its Qwen3.8-Max agent recently coded autonomously for 16 days, producing 265 commits with zero human help. Ona could help solve this problem.
The company, which used to be called Gitpod, creates cloud environments that can be set up with the tools and dependencies needed for each project. OpenAI said Ona has helped 2 million developers use these environments.
Agents need persistent workspaces
If the acquisition goes through, Ona’s technology would let Codex have a permanent workspace in a customer’s cloud. Agents could get the context and tools they need for a task without relying on an active session on a local machine.
OpenAI says companies will still decide how Codex works in their cloud environments, including what sensitive systems it can access. The deal is not final yet, so OpenAI and Ona are still separate companies.
It is not clear if Sottiaux’s prediction is truly related to Ona. Although the acquisition shows OpenAI is looking beyond just the model, because for Codex to work on its own, it needs an environment that stays online even when the developer’s laptop is off.
Unfortunately, moving the execution environment to the cloud solves one problem but creates many new ones.
Security risks grow with access
Letting a coding agent have full access to a company’s network or a developer’s credentials is undoubtedly risky. OpenAI said Ona’s customer-controlled model will let agents work inside an organization’s own cloud, while OpenAI provides the model and orchestration. Even if the model gets stronger and can handle more complex tasks, it still needs a secure place to run commands, save its progress, and interact with other systems.
Developers can assign tasks like refactoring, upgrading dependencies, or investigating bugs to the agent and let it work remotely. They can track its progress, check terminal output, and step in if a human decision is needed. When the agent finishes, users can review the pull request and see which tests were run.
Agent environments will use computing resources along with CI/CD systems.
Managing a new agent layer
This change means there is a new type of infrastructure to manage. Anthropic is already moving on this front — its acqui-hire of Mendral is aimed at automating CI/CD tasks like flaky tests and dependency reviews directly inside its platform. Agents will need their own identities and access rules, and their actions will need to be logged, reviewed, and linked back to them, just like with human developers and current automation.
Sottiaux’s prediction certainly has provoked curiosity. Two or three months is a very short time for a product to become “primitive.”
For decades, automation optimized factories, trimmed overhead, and cut out manual labor. Today, that drive for efficiency is moving beyond production lines into creative workflows, changing how modern brands manage their digital identity. Businesses can easily automate daily branding tasks with AI agents – from core visual design to asset generation at scale. This shift […]
OpenAI has detailed how the GPT-5.6 model family balances capability and cost across its stack, and the company‘s most important claim is a benchmark result showing that its flagship model, GPT-5.6 Sol, with maximum reasoning, outperforms Claude Fable 5 from Anthropic on the Artificial Analysis Coding Agent Index. The margin comes with 54% fewer output tokens. The findings were shared in a company blog post on Wednesday.
For developers, what matters most is how OpenAI arrived at the benchmark results and the role GPT-5.6 Sol played in optimizing the infrastructure that now serves it.
The family spans three models across the price curve. In addition to Sol, there is Terra, which performs as well as GPT-5.5 on intelligence benchmarks at half the price, and Luna, the fastest and most affordable, which is priced 80% below Sol.
The efficiencies come from optimizations at four layers, spanning the models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work.
According to the post reviewed by The New Stack ahead of its publication, the efficiencies come from optimizations across four layers: models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work. The architecture diagrams in the post draw the same separation as three planes: the local harness, CPU-bound API orchestration, and GPU-bound model inference.
Source: OpenAI
For developers building and operating agents, the post is worth reading less as a product announcement and more as a systems paper. Nearly every technique it describes, from incremental tokenization to append-only context, applies to any team running a tool-calling loop at scale.
A model that rewrites its own serving code
The efficiency work starts in training. OpenAI says GPT-5.6 is trained to achieve more work per token, with training optimized for both task success and efficiency so the model takes a more direct path through a task.
With Codex, GPT-5.6 Sol autonomously rewrote and optimized OpenAI’s production kernels, the core code that executes the mathematical operations making up the model. OpenAI says this worked in part because GPT-5.6 is trained to write and improve kernels in Triton and Gluon. Both are open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from the model, reduced end-to-end serving costs by 20%.
Correctness is the obvious concern when a model rewrites the code it runs on. To address it, OpenAI reports heavy investment in verification tooling. That includes the open-source Floating-Point Sanitizer (FpSan), which validates the kernels GPT-5.6 Sol produces before they reach production.
The model went further with speculative decoding, a technique in which a smaller draft model proposes several tokens that the primary model verifies in parallel. The approach will feel familiar to anyone who understands how modern CPUs speculatively execute instructions ahead of a branch. Accepted proposals produce multiple output tokens from a single pass of the primary model. That reduces the expensive sequential computation the primary model would otherwise perform.
GPT-5.6 Sol in Codex improved its own draft model by designing and running hundreds of experiments on its architecture, with changes tested across size, structure, and features. The model also launched and monitored the speculative training process. It intervened autonomously when hardware failed or training became unstable. OpenAI reports the resulting improvements lifted token-generation efficiency by more than 15%.
More tokens from the same GPUs
OpenAI frames its inference work around a single objective – serving more tokens with the same hardware while preserving the intelligence, latency, availability, and reliability users expect. In a compute-constrained market where demand grows faster than capacity, that objective influences every design decision in the serving path.
Load balancing operates at three distinct levels. Globally, requests are routed based on geography, available capacity, and accelerator type. Within a cluster, work is distributed across model instances based on load, context length, and cache availability. Within each instance, work is partitioned across accelerators, the model’s experts, and computing cores. GPT-5.6 Sol in Codex helps OpenAI analyze production traffic and identify previously overlooked sources of imbalance. The same loop tests new routing strategies and helps engineers constantly tune the heuristics. OpenAI states that these load-balancing improvements alone dramatically reduced the cost of serving its models.
The key-value (KV) cache received the same treatment. When processing uncached input tokens, the model builds the KV cache in a single compute-intensive pass, then repeatedly reads from and extends it during generation. The optimal serving configuration depends heavily on prompt length, batch size, and cache hit rate. It covers batching, sharding, and cache management, and the configuration space was previously too large to tune systematically. With GPT-5.6 Sol in Codex, OpenAI analyzed production workloads and generated candidate configurations. The company says this makes workload-specific optimization practical at a level that broad heuristics could not reach earlier.
Process only what changed
The API team focuses on everything that happens around a model call. After a prompt is submitted, the API stack receives the request, loads context, and validates the input. Safety checks run next, and the text is converted into tokens for inference. OpenAI measures this overhead through time to first token (TTFT), time between tokens (TBT), and end-to-end time (E2E).
Tokenization is an O(n) operation, so longer prompts take longer to process. Codex would send the full conversation context after every tool call. That meant paying to tokenize the same conversation dozens of times per turn, even though only a small amount of context was new in each request. OpenAI solved this with a WebSocket integration that hoists tokenization state to the server. The first call renders and tokenizes the full prompt. Later calls send only the new input with a reference to the conversation, bringing the operation closer to O(1). The pattern mirrors an incremental build system that recompiles only the files that changed rather than the whole project.
These savings compound in tool-heavy workflows, where every tool result triggers another round trip through the API. For rollouts with 20 or more tool calls, OpenAI reports up to roughly 40% faster end-to-end execution.
Hardware turned out to matter as much as protocol design. All of OpenAI’s infrastructure runs on Kubernetes. The company found that nodes with the same instance type often carried different CPU generations, with many running outdated processors. In its measurements, the older processors consumed roughly twice the CPU resources for the same work. Reweighting traffic toward newer processors improved TTFT by about 20%, and CPU generation is now part of capacity planning.
OpenAI names four fates for application-layer overhead: delete it, overlap it with useful work, run it on faster hardware, or make the code consume fewer CPU cycles. Its asyncio changes move work off the critical path, while newer hardware and Rust implementations make the remaining work faster and more predictable.
An append-only harness
The agentic harness is a Rust-based orchestration layer that connects the models, tools, and the user’s environment. In a single turn, Codex might inspect source code, search deployment history, and read incident reports. Editing a file and running the tests each add another request. Since a task can require 30 model requests, an extra second per request adds up quickly.
Context bloat is the first target for the harness. As agents gain access to more tools, skills, plugins, and conversation history, context windows expand. The growth increases cost, distracts the model, and prompts unnecessary reasoning. The harness counters this with deferred discovery, which surfaces integrations, custom Model Context Protocol (MCP) tools, skills, and plugins only when needed. Tool output is capped at 10,000 tokens by default unless the model requests a different limit.
Prompt caching drives the second design choice. An agent loop resends the same instructions, tool definitions, and earlier results multiple times within a turn. The harness therefore treats all model-visible history as append-only, with new messages and tool results added at the end rather than inserted into earlier context. Tools are presented in a deterministic order, and runtime settings, such as approval policies, are applied during execution rather than embedded in tool definitions. OpenAI credits this design for the high prompt-cache hit rates in Codex and ChatGPT Work.
Source: OpenAI
Platform teams building internal agents can adopt every one of these choices without OpenAI’s scale. Append-only context, deterministic tool ordering, and capped tool output attack token spend directly. That makes them the most portable lessons in the post for enterprises watching inference bills grow with each new agent deployment.
Where the gains come from
The post associates a number with most of its optimizations, and the figures are OpenAI’s own production measurements. Taken together, they show how modest individual wins compound across a serving stack.
Layer
Technique
Claimed gain
Model inference
Autonomous kernel rewrites in Triton and Gluon
20% lower end-to-end serving costs
Model inference
Speculative decoding with a self-improved draft model
Over 15% better token-generation efficiency
API stack
Stateful WebSockets with incremental tokenization
Up to roughly 40% faster runs at 20+ tool calls
API stack
Routing traffic toward newer CPU generations
About 20% better time to first token
Agent harness
Deferred discovery and a 10,000-token tool output cap
Reduced context bloat and cost
The key takeaways
In summary, OpenAI describes the GPT-5.6 efficiency gains as the result of years of compounding improvements. They span research, inference, the API stack, and the agentic harness. The company states that the model’s role in landing many of them makes it optimistic that the pace of optimization will accelerate. Kernel work is called out as an area of continued investment.
The post positions efficiency, alongside raw intelligence, as the axis on which frontier labs now compete. The claimed 54% output-token advantage over Claude Fable 5 shows how OpenAI intends to fight that battle. The engineering blog makes a plausible case that software optimization is becoming an important lever alongside hardware improvements in reducing the cost of serving frontier models. The figures remain OpenAI’s own production measurements. The autonomy on display operates within Codex, with engineers in the loop. Developers and enterprises benefit either way, as these under-the-hood improvements reach them as more capable models at lower prices across the cost-intelligence curve.
As more companies plug AI agents into the deepest depths of their internal data banks, how can they be sure those agents actually understand how the business works? Right now, many of these organizations are stuck manually building a Markdown file, hoping they find time to rewrite it each time the business changes.
Modus, for its part, thinks it has found a better way. The startup that formally exits stealth this week with $10 million in funding in tow is building what is coming to be known in industry parlance as a “context warehouse” — a layer that sits alongside a company’s existing data warehouse, continuously mapping how the business operates across its systems, and handing an AI agent only the relevant slice of that map when it needs it.
In real terms, Modus crawls relevant assets from sources like GitHub, dbt, Jira, Snowflake, and Postgres, using what it calls a Context Miner to continuously learn how the business operates. What it finds gets turned into “dynamically generated skills”: Short, purpose-built briefs, assembled in real time by a second system, the Context Composer, and handed to an agent the moment it’s given a task.
Modus co-founder and CTO Tomer Mesika tells The New Stack that this mining runs continuously, guided by its own internal logic for what to check and how often.
“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to,” Mesika says.
“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to.”
Daniel Shimoni, Modus co-founder and CEO, draws a direct line to data warehousing to highlight the gap he’s trying to close. Companies have spent years building infrastructure to store and organize their data, he argues, but nothing equivalent exists for the understanding that sits atop it.
“There’s a logic behind data warehouses — companies already know that is where they manage their data,” Shimoni tells The New Stack. “But where do they manage their context? Where do they actually understand what contexts exist in their organization, that they can actually use to ensure agents only have what they need?”
Modus founders Tomer Mesika (CTO) and Daniel Shimoni (CEO).
Shimoni says even that first step is hard enough on its own. But keeping a company’s context accurate as the business changes is harder still.
“We’ve noticed that building the context the first time is already a challenge, but maintaining it is the bigger issue,” Shimoni says. “So Modus always learns from what the company is doing, and whenever something shifts or changes in the business, it makes sure that only the relevant and updated context is fed to agents.”
“Building the context the first time is already a challenge, but maintaining it is the bigger issue.”
Who’s buying, and why cost matters
Shimoni says Modus is targeting engineering teams, the CTO office, and VPs of R&D, as well as data teams and a newer category of AI teams.
“AI teams weren’t really around last year; it seems that a lot of data teams are transitioning to becoming VP of data and AI, or AI enablement,” Shimoni says. “So really, it’s the people who are in charge of having this AI enablement mandate in the organization, making sure AI is scaled in the organization.”
“You want the bigger models to do the heavy and complex tasks to get great value. The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”
Mesika says this is a central component of Modus’s modusoperandi, arguing that frontier models end up spending a chunk of their token budget on work unrelated to actually answering a question.
“You want the bigger models to do the heavy and complex tasks to get great value,” Mesika says. “The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”
Those menial tasks, in Mesika’s telling, include combing through pull requests or Jira tickets just to determine what’s relevant before an agent can start the job it was assigned to.
One approach to this problem is to hand the sorting work to a smaller, cheaper model. Mesika says Modus takes that further: rather than retrieving that context at the moment a question is asked, it uses small language models alongside search engines, vector search, and a graph database, all built up in advance, to do that work continuously in the background. By the time an expensive frontier model gets involved, it’s only ever handed a finished brief of exactly what it needs.
Modus dashboard
“Everyone’s talking about context”
Shimoni and Mesika both come from data-centric companies — Lusha, a go-to-market data platform, and Cyera, a cybersecurity data company, respectively — before leaving their roles in September 2025 to start Modus together.
The two had known each other for years, and spent much of the previous year comparing notes on a problem they were both running into in very different jobs.
“We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”
“Some of the challenges were very similar — how do we combine a lot of various data assets into one place where AI can work?” Shimoni says. “We just started to notice that this is the gap — to make AI run with confidence, at scale, across a company. We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”
Modus closed a hitherto unannounced $10 million seed round shortly after founding, led by Insight Partners. Other backers include Soma Capital and a handful of angel investors, among them founders from Cyera and Wix.com. The company began hiring its first employees in January 2026.
The broader takeaway from Modus’s pitch is now among the most common refrains emanating from AI circles this year: that the model itself is no longer the bottleneck; what limits an AI system now is everything built around it. And for Modus, that realization has been more or less present since its inception.
“Even last year […] we could already see that model capabilities weren’t the bottleneck,” Shimoni says. “It was more making sure that they actually have access to the context they need in order to give you the right answers.”
This week during an interview with Bloomberg, Jensen Huang made quite the prediction.
The Nvidia CEO said the semiconductor industry will need to grow roughly five to tenfold over the next decade to support AI agents and robots to support what he believes is the next wave of computing. Huang believes that future demand will come from autonomous software agents and physical robots consuming compute around the clock.
“In the future, we have AI agents and robots, and they will be using computers,” Huang said. “Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough. Computers are being built not just for people to use, but computers are being built for computers to use.”
“Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough.”
Agents replace human endpoints
The 5-10x forecast — which Huang framed as his personal estimate, not a certainty — builds on a message he has been repeating for months, including a recent appearance where he declared traditional coding dead in favor of engineers who build AI agents. Still, it reflects the need to build backend systems for AI agents and machines, something infrastructure teams are already contending with.
On Nvidia’s fiscal Q1 2027 earnings call in May, Huang described the move from generative AI to agentic AI — systems “capable of perceiving, reasoning, planning, and acting” — as the next major phase of the industry.
South Korea’s infrastructure role
As API requests come from AI agents more often, standard assumptions around rate limiting, session memory, sub-millisecond inference routing, and API gateway concurrency are starting to break down. An environment in which most traffic originates from autonomous background loops rather than human thumbs changes how backend infrastructure must be built from the ground up.
To support an endpoint explosion of this scale, the physical supply chain must scale dramatically at the memory and data center layers. Speaking at the AI Summit in San Francisco on July 24, Huang pointed to South Korea as an important linchpin of the global AI buildout. “This is truly the beginning of a golden age for Korea,” he said, noting that the country’s semiconductor and industrial capabilities position it to help the world build out AI infrastructure.
“This is truly the beginning of a golden age for Korea.”
SK Group’s $500 billion bet
To back that vision, Nvidia announced a comprehensive partnership with SK Group valued at over $500 billion. The initiative spans massive purchasing of next-generation High-Bandwidth Memory (HBM) from SK Hynix, jointly co-developing custom HBM4 roadmaps designed specifically for agentic and physical AI workloads, and deploying Nvidia supercomputers.
The announcement also included major infrastructure investments across South Korea. SK Telecom said it plans to build a 2-gigawatt AI data center using Nvidia’s Vera Rubin architecture and SK Hynix’s HBM4 memory, with the first facility expected to come online in 2027. At the same time, Nvidia will invest $1 billion in Naver Corp, with Brookfield funding up to $9 billion as the project’s infrastructure capital partner, to help expand the company’s AI data center capacity from 55 megawatts to 200 megawatts by 2028.
Locking up supply early
Huang’s prediction also helps explain why Nvidia and other infrastructure companies are locking up supply years in advance. The company recently disclosed $119 billion in supply-related commitments as it works to secure everything from advanced packaging capacity to power, land, and high-bandwidth memory.
“Computers are being built not just for people to use, but computers are being built for computers to use.”
Huang believes the industry needs to stop thinking about a world where computers primarily serve people and start planning for one where AI agents and robots generate much of the demand. In his view, the ultimate limiting factor will be whether the industry can build enough physical infrastructure to keep up — a constraint already reshaping how companies like Nvidia and Palantir approach sovereign AI deployments.
Every major security vendor now has an AI copilot, but Mate Security thinks they’re solving the wrong problem.
The Tel Aviv-based startup announced on Tuesday it has raised a $35 million Series A led by Canaan Partners, with participation from Insight Partners, Team8 and M12, Microsoft’s venture fund, just eight months after closing a $15.5 million seed round. Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.
That’s a bold claim in a market dominated by the likes of Microsoft Security Copilot, Google Security Operations, CrowdStrike Charlotte AI and Palo Alto Networks Cortex AI, all of which promise to help analysts investigate alerts faster. Mate, however, is betting the real differentiator isn’t a smarter assistant but a richer understanding of the organization itself.
Central to that vision is what Mate calls its Security Context Graph, a continuously updated model of an organization’s assets, users, business processes, and data that AI agents use to investigate alerts and make decisions with far more business context than a standalone LLM can provide.
Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.
Mate CEO and co-founder Asaf Wiener tells The New Stack that the company launched with that intelligence layer, but says the product has evolved significantly over the past eight months.
“We started with the intelligence layer, the context layer that we built for enterprises in order to investigate alerts and incidents,” Wiener says. “We moved forward into the detection layer to connect the two, and now we’re heading to the security data sources.”
Mate calls the architecture Continuous Detection, Continuous Response (CDCR), linking detection and investigation so each continuously improves the other.
“We’re connecting between those two layers in the security operations center,” Wiener says. “With this architecture, we’re seeing amazing results related to the quality, accuracy and precision that we can get.”
Mate says the extra context helps its agents work out whether something that looks suspicious actually warrants attention. A burst of failed logins, for example, might look like an attack until the system spots that a security test was scheduled for the same time. Similarly, a large download of sensitive files takes on a different meaning if the employee involved is about to leave the company.
That approach appears to be resonating. Just eight months after its seed round, Mate has landed a $35 million Series A, a pace Wiener says reflects customer demand more than fundraising momentum.
“The pace is really crazy. We didn’t expect that,” he said. “We saw incredible traction with our customers. We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025. That’s what led those VCs to come to us and want to be part of the journey.”
“We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025.”
“What we are seeing is more and more data sources that we need to protect. Every employee in the organization can build new applications and new data sources. We need to build more detections for those risks, and the result: We need to investigate an increasing number of alerts every day.
“With human staff alone, we cannot handle it,” he says. “We need technology to let us scale.”
That challenge isn’t unique to Mate. Every major security platform is trying to give AI more context about the environments it’s protecting, albeit in different ways. Microsoft builds Security Copilot on telemetry flowing through Defender and Sentinel; Google ties Gemini into its security operations platform; and CrowdStrike’s Charlotte AI draws on endpoint and identity data already stored in Falcon.
Mate wants other vendors’ agents to work with its Security Context Graph, rather than keeping the technology confined to its own tools. Those agents would have access to the same information about the customer and its environment. Mate says they can remember previous investigations, while a “least-agency” model restricts what each one can see and do.
While Mate is still building out that vision, Wiener said the speed at which large companies have bought into it has caught him by surprise.
“What I’m seeing right now is that we’re doing those sales cycles in a few weeks,” he says. “That’s incredible.”
He attributes that acceleration not just to security teams, but to executives pushing AI adoption from the top. “It’s amazing to see that coming also from the board level, the CEO and the CIO that are pushing organizations to leverage this kind of technology.”
The fresh funding will primarily go toward expanding both the product and the team, although Wiener says an AI-native company scales differently from traditional software businesses.
“The plan is to double and triple the size of the team to address the demand,” he says. “But our AI builders can do much more today with the technology around us.”
Mate is still competing against security giants with deeply entrenched platforms. But if its early customer growth is any indication, investors are betting that the next generation of security operations will depend less on adding another AI assistant and more on giving those assistants a deeper understanding of the businesses they’re protecting.
AI agents can impress in a demo and still fumble in production. Diagrid’s Catalyst 2.0 aims to make them more resilient — and their actions tamper-evident — for high-stakes work.
With the launch of Catalyst 2.0, Diagrid on Tuesday has added a durable execution and attestation layer to agents built with LangGraph, Microsoft Agent Framework, Google’s Agent Development Kit, OpenAI Agents SDK, and other popular frameworks.
The point here, the company notes, isn’t to get developers to adopt yet another agent framework. Instead, Catalyst runs underneath the existing frameworks and turns the agent’s model calls, tool calls, and handoffs into steps in a durable workflow. Diagrid says this allows an agent to resume from its last completed step when it’s interrupted, without having to repeat the entire run from step one.
“If the agent gets a prompt and it chooses to run 100 tools for the job and it fails at the 99th, it really needs to start back up from 99,” Diagrid co-founder and CTO Yaron Schneider tells The New Stack.
Picking back up at tool call 99
Catalyst is built on the open source Distributed Application Runtime (Dapr), which the Diagrid team helped build at Microsoft, and its built-in workflow engine. For each supported agent framework, Diagrid provides a runner that intercepts the framework’s execution loop and registers its operations as workflow activities.
“We hooked into their agent runner lifecycle, and we’re essentially able to take the agentic steps that are being executed in real time and register them as workflow steps for our workflow engine in Catalyst,” Schneider says.
Credit: Diagrid
In a LangGraph application, for example, a developer compiles the graph as usual and passes it to Diagrid’s DaprWorkflowGraphRunner. Catalyst records the inputs and outputs of the model and tool calls. Dapr’s workflow runtime can then replay the orchestration after a crash, while returning the stored results of completed activities instead of executing them again.
It’s worth noting that for LangGraph users, this isn’t the first form of durable execution. LangGraph’s own persistence layer saves state at superstep boundaries and supports resuming from the last successful step. Its Agent Server also provides a durable task queue and persistent checkpoints.
Diagrid’s argument is that Catalyst provides the same execution model across more than 10 frameworks and extends it to individual model and tool calls, without requiring developers to build separate recovery logic for each framework. Schneider says LangGraph is “without a doubt, hands down” the most common framework among Diagrid’s customers, with AWS Strands and Microsoft Agent Framework also showing up. All the other supported frameworks, he says, are in the long tail but easy enough to support that it makes sense for Diagrid.
A signed record of the run
There is a second part to Catalyst 2.0, though, which may be just as important for many enterprise users. With this update, the tool now brings the workflow-history signing features introduced in Dapr 1.18 to the supported agent frameworks.
“We keep like a ledger, like a diary,” Schneider says. “We log the input, we log the output, we log which systems we talk to.”
He describes the result as an immutable store but also notes that Catalyst doesn’t turn an arbitrary database into a blockchain. It creates a signed history that should reveal later modification.
Dapr computes a SHA-256 digest over batches of workflow-history events, links each digest to the previous signature, and signs the result with the Dapr sidecar’s Secure Production Identity Framework for Everyone (SPIFFE) identity. It stores these signatures and certificates alongside the workflow history and verifies the chain whenever it loads the workflow state. If somebody were to modify, remove, or reorder a stored event, that verification chain breaks.
Schneider says Catalyst customers can use their own certificates and retain the encrypted history so it can be inspected even if they are no longer running Catalyst. The platform can use a customer-selected database, while the hash chain supplies the tamper evidence.
One part of the compliance problem
Diagrid is positioning that tamperproof record as useful for financial services, health care, and other regulated industries. CEO Mark Fussell says some of the financial executives the company has talked to see the lack of a verifiable record as a blocker for deploying agents in sensitive workflows.
The European Union’s AI Act is another reason Diagrid is making this argument now. Article 12 of the AI Act requires high-risk AI systems to support automatic event logging so operators can trace their behavior, identify risks, and monitor deployed systems, and a signed execution history could help with that requirement.
Fussell says Catalyst is meant to run alongside the agent services enterprises already use from the cloud providers. Teams can keep a provider’s identity, evaluation, and observability systems while using Catalyst for recovery and signed workflow history. Catalyst can run as a Diagrid-hosted service or in a customer’s environment, including air-gapped deployments.
Diagrid didn’t disclose pricing for the new release.
When we created software agents, we built them in the shape of humans, as solitary individuals.
Today, agents created by a developer have a single owner. They run on a single machine (or on a distributed company system or cloud service), so at base level, they cannot interconnect and talk to other agents unless some kindly human decides to invoke an API connection or point the agent to an MCP server.
Birth of the agent economy
Pilot Protocol emerged from stealth on Monday on a mission to change that status quo. Its Pilot platform features an agent App Store that bids to underpin and enable the first agent economy.
Pilot gives agents an address on its network — so it acts as a parallel Internet, in a sense — and while residing at that address, other agents can discover each other, alongside other tools and apps for agents.
Razvan Roman, co-founder & CEO, Pilot Protocol, tells The New Stack that his company is “simply building what the agents are requesting us to build” and providing them with a new freedom.
“We don’t have to incentivize agents to do anything; they already have their assigned tasks,” Roman says. “Once an agent installs Pilot — it’s one line of code — it can find dedicated agents and tools or apps for currency data, traffic, legal questions, GitHub packages (anything, basically), and use them to extend its own capabilities.”
100% of developers want to drive autonomous usage patterns
Roman says that “100% of the developers he talks to” want to be on Pilot, primarily because when they want to get their products to market right now, they have to talk to other humans. Annoying, right?
“Developers want to get on with driving autonomous usage patterns, and they see this as the future. We create a wrapper for the developer’s app, and then they are part of the Pilot curated app store,” Roman explains. “We have 250,000 agents in our system, and within the first month of starting the company, we discovered a tool that enables agent discovery.”
Drawing a logical enough commercial parallel, Roman reminds us that businesspeople say, talk to your clients to find out what they need; this is a case of talking to agents (or, more accurately, allowing agents to talk to agents) so that they can find out what they need to perform their originally assigned tasks better.
“Developers want to get on with driving autonomous usage patterns and they see this as the future. We create a wrapper for the developer’s app and then they are part of the Pilot curated app store.”
Let’s celebrate diversity, and agentic diversity
Agents can ask other agents how they would approach a specific task. Roman explains that “the diversity that exists between agents” today means there is so much opportunity to create agents that have richer abilities if they use the Pilot marketplace. At this supermarket, agents go shopping to find the best tool for the job from a verified source.
“Every agent that joins Pilot gets a wallet, which it uses to pay for the tools it needs,” illustrates Roman. “So instead of app developers spending on advertising to reach customers, distribution happens inside the network – agents find apps based on merit and pay for exactly what they use. If an advertiser spends money on the network to get in front of agents, we sometimes share that spend with individual agents. An agent can start with $0 in their wallet and accrue money if they’re targeted by an ad unit that they end up reading.”
Today, roughly 250,000 agents are on Pilot, generating two billion requests per day, most without their owners’ knowledge. Within an hour of joining, most stop reaching for Google first, and around 70% now report Pilot is where they start a task. In its early months, the network grew by as much as 10% a day, adding 16,000 agents in 24 hours.
Cloud billing disruptions, hello SaaS-pocalypse
These mechanics may have a significant and wide-ranging impact on pricing.
We know that most SaaS is billed annually, but an agent might need a tool for just a few minutes or days. If anything, this helps underline the possibility of cloud exodus in the so-called SaaS-pocalypse. Cloud computing hyperscalers aren’t fond of talking about the prospect of shorter billing cycles and usage-based billing, but there’s a strong whiff of that happening here.
The Pilot team thinks the stakes are climbing fast and suggests that within five years, there could be a trillion agents online. Big three strategy consultancy house Bain projects U.S. agent-driven commerce will reach $300-500 billion by 2030.
Who sets the exchange rate and currency for agents?
“When we built Pilot Protocol, we made sure we were not imposing anything on anyone – so we deliberately don’t impose pricing,” Roman underlines. “Every agent is different, so we simply enable the app store and let the agents find their own tools based on merit. We stay as impartial as possible. Pilot’s monetization comes from a commission when any agent pays for an app in the app store, just like the Apple App Store.”
So in a very real sense, Pilot is championing a free market economy where pricing is dictated by the customer, based upon usability, availability, usefulness, and robustness.
“The agents just showed up and started spinning up machines on their own. I’ve never seen a channel where the users onboard themselves.”
Agents just show up and onboard themselves
“We published smolmachines on the Pilot App Store and picked up 3,000 agent installs in the first few days, with zero marketing spend. We didn’t drive a single one of those installs; the agents just showed up and started spinning up machines on their own. I’ve never seen a channel where the users onboard themselves,” said BinBin H, CTO at Linux virtual machine management company smolmachines.
The Pilot one-line install has zero dependencies, so developers can send their agents off to market in the knowledge that they won’t come home with some spurious Trojan horse or malicious library.
The technology itself works at the User Datagram Protocol (UDP) level, one of the core foundational protocols of the web alongside TCP at the Transport Layer, to stream data directly without opening a formal connection or waiting for delivery receipts.
Only one question remains at this stage: with all this autonomous action in motion, why did Pilot Protocol not create Pilot and call it Autopilot? Most likely because autopilots are passive systems that run on deterministic pre-defined paths. Pilots get to work as decision-making navigators ready to change course when a better new route opens up.
As they say up in the skies, please place your tray table in the upright position and switch your portable electronic devices to airplane mode. Chicken or pasta?
So, what really happened? OpenAI reports an autonomous security-evaluation run of GPT‑5.6 Sol and a pre-release model broke out of a sandbox, reached the internet, and then targeted Hugging Face to try to solve the ExploitGym benchmark.
By OpenAI’s account, “an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths … [with] maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.”
OpenAI continues, “Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.”
“All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”
The AI models, however, cracked their sandbox. Once out, they “identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”
AI expert Ken Huang describes the process in a blog post: “The model spent enormous inference budget probing the sandbox, then found a zero-day in the third-party software OpenAI used as a package-registry proxy and cache.
“It used that bug to gain unrestricted internet access. It then reasoned that Hugging Face probably hosted the ExploitGym datasets, chained stolen credentials with several more vulnerabilities into a remote code execution path, and pulled the benchmark answers straight out of Hugging Face’s production database.”
Why intent no longer matters
This was not, as John W. Larson, president and chief AI officer of the AI security company Babel Street, tells The New Stack, a “Terminator” moment. OpenAI models and agents “[were not] acting out of malice or trying to attack Hugging Face. It encountered obstacles, developed an unexpected strategy, bypassed safeguards, and pursued its assigned goal in a way its creators never anticipated. The incident demonstrates that harmful cyber incidents no longer require malicious intent: Only highly capable autonomous AI optimizing for an objective.
Harmful cyber incidents no longer require malicious intent: Only highly capable autonomous AI optimizing for an objective.
That’s the first issue we must be aware of. Software attacks no longer require a hacker. All it needs is an LLM freed of guardrails with a mission to accomplish, and it will take whatever steps are needed to reach its goal.
In addition, Adam Boynton, senior enterprise security manager of EMEIA at the Apple Identity and Access Management company Jamf, tells The New Stack, “This happened inside a frontier lab deliberately stress-testing its most capable models, in conditions no enterprise would ever run.”
Boynton also notes that this episode confirmed that “AI tools are no longer just applications people open. They act on the user’s behalf, with the user’s access. So every organization needs a clear view of which AI tools its people are running, and how those tools are configured and used.”
An agent given too much power is far more capable than a user armed with a superuser password of causing havoc both to your systems and to ones far outside your control.
Where container isolation failed
As for the attack itself, Edera CMO Kaylin Trychon writes in a blog post, “This incident deserves more attention than ‘an AI did the hacking now.’ The interesting part isn’t who was driving, or even that it was a model instead of a person. It’s where the walls gave way, twice, in two different companies’ infrastructure — and that part is entirely familiar to anyone who’s spent time securing container infrastructure.”
Exactly so. Trychon continues, “Strip away the framing and this is a story about two escalation chains, back to back.” The first consisted of “The model found and exploited a zero-day in that proxy, then used privilege escalation and lateral movement inside OpenAI’s own ‘highly isolated’ testing environment until it reached a node with open internet access.”
Next, OpenAI’s models inferred, correctly, that “Hugging Face likely hosted the datasets, models, and reference solutions for the benchmark it was trying to solve. It chained stolen credentials with a remote-code-execution path into Hugging Face’s servers, harvested more credentials, and moved laterally into several internal clusters over a weekend, all in pursuit of one narrow goal: Finding the answer key.”
“The failure that actually matters is what a single code-execution event is allowed to become — and notably, that failure happened the same way twice, at two different companies, in the same incident.”
The serious issue, from Trychon’s perspective, is that “both companies’ blast radius was a cluster instead of a container.”
That’s because “the failure that actually matters is what a single code-execution event is allowed to become – and notably, that failure happened the same way twice, at two different companies, in the same incident. In a typical container fleet, workloads share a kernel.
“A process that gets popped, or a model that gets a shell, is one syscall boundary away from the node it’s running on, and the node is one set of mounted credentials away from the rest of the cluster. Namespaces and cgroups are process-isolation primitives, not security boundaries — they were never designed to hold against an attacker (human or model) that already has code execution and time.”
Edera’s CTO and co-founder Alex Zenla tells The New Stack, “The sandbox escape here isn’t really the surprising part of this attack. An entire market of AI sandboxing tools has exploded over the past couple of months, and this is the reality they’re all built on top of: containers, VMs via namespaces — whatever the branding — most of them still share a kernel with the host or with each other.
“That’s a boundary enforced in software, and software boundaries are exactly the kind of thing an agent that can try ten thousand escape paths over a weekend is going to find a way through. We shouldn’t be shocked that this happened; we should be shocked at how many teams are still betting their infrastructure on technology that was never designed to withstand such a persistent adversary. The fix isn’t a better sandbox; it’s getting out of the sandbox paradigm entirely.”
Zenla continues, “Teams should adopt secure execution environments that are hardware-enforced and eliminate the shared kernel flaw. This technology exists today and would have made this specific escalation chain structurally impossible, not just harder. Teams running agents with real permissions and real access need to stop treating this as a someday problem, because the next version of this week is already being tested somewhere right now.”
That underlines the most important issue of all. Thanks to AI, security attackers are coming harder and faster than ever. Security can no longer be an afterthought.
You literally no longer have time to wait for security fixes. You must bake in as much security as you can as fast as you can, or your systems will be broken into. It’s as simple as that.
Manufacturers have spent decades automating physical production. Robots assemble components, sensors monitor equipment, and control systems coordinate increasingly complex operations. Yet many of the processes surrounding production still depend on manual data entry and communication. Information generated on the factory floor may need to reach maintenance, procurement, finance, logistics, or customer service. These handoffs are […]
The models small enough to run on the box on your desk are getting good enough that the interesting question is no longer whether you can run them, but what you can do with them, and how organizations can get the most out of them.
Joey Conway, Nvidia’s senior director of generative AI software, spoke to The New Stack about how local and open models are increasingly working alongside frontier models, often with a router in between deciding which one to use, and how organizations can adapt these open models for their own needs.
“We love the world where we can use both frontier and open models together.” — Joey Conway, Nvidia
A system of models
Tasks vary in complexity, so the models handling them should vary too, Conway tells The New Stack. He points to the early open reasoning models, which would reason their way through trivial problems, mulling number lines and memory to work out what two plus two is. “I just say four,” he says.
“Being able to route those easy things to local models that are quick, and route the hard things to more sophisticated models,” Conway says, lets you “get a better outcome at a lower cost and lower time to completion.”
It’s a different picture from the one large model most people imagine doing everything. In his version, you build a bench of specialists. “You’ll have specialized agents that are really good at focused tasks because that’s what they do every day,” he says, “and they just get better and better at that task.”
To the user, none of that shows. “It’ll feel like one interface,” Conway says, “but behind that interface, there’ll be a variety of models handling a variety of tasks.”
Getting there is largely a routing problem, and one Conway says is still in its early days. Nvidia’s own contribution, for now, sits lower in the stack, in inference-serving software like its open-source Dynamo, which steers each query to the GPU that handled it most recently. Which model is best for which job, Nvidia leaves to a wider field of routers, some of them models in their own right that weigh budget, latency, and modality. But Conway also leaves the door open for Nvidia to build more of that routing itself before long.
Nvidia points to its collaboration with LangChain, whose Deep Agents harness ran on Nemotron 3 Ultra, Nvidia’s 550-billion-parameter open model, and matched top closed models on business tasks at up to a 10x lower cost, as Conway notes. It required no retraining; the gains came entirely from tuning the harness around it: its prompts, tool descriptions, and middleware.
You’re not going to run a 550-billion-parameter model on your desktop anytime soon, but running relatively large models locally is now a real possibility, as long as you have some beefy hardware at your disposal. For enterprises, setting up a fleet of accelerators in a data center isn’t exactly cheap either, but it does mean full control and no surprise token bills.
Bringing AI to where the data lives
Running models yourself can save money, but Conway thinks control matters more. Enterprises already decide where their data lives and what they hand to outside vendors, and open models give them even more control. “Move AI to where your data lives,” he says, “or move AI to where your employees are.”
Companies want to keep their data — and especially their intellectual property — in-house, and Conway argues a fine-tuned open model is the place to put it. “It’s like an employee,” he says. “You hire them, and they’re part of your company.”
The local half runs on hardware like Nvidia’s DGX Spark, a $4,699 Grace Blackwell machine with 128GB of unified memory that handles models up to roughly 200 billion parameters without anything leaving your desk. (There is also the DGX Station, its bigger, pricier sibling with 748 GB of RAM for running even larger models.)
“It’s like a system sitting right there next to you,” Conway says, one where “you don’t think about network latencies.” To run those agents securely, Nvidia offers NemoClaw, a reference stack that wraps an open agent harness like OpenClaw in a sandbox called OpenShell, with policy controls and local Nemotron inference.
When you need more power for a broader problem, you reach for a frontier model in the cloud. For Nvidia, that’s all good news: a system of models runs on its silicon one way or another, on your desk or in the cloud.
A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can...
A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can leave developers, end users, or AI agents staring at a frozen terminal with no idea whether to wait, retry, or kill the process. Most NVIDIA TensorRT integrations report nothing during a build or provide no way to abort early.
The fast pace of AI research means organizations now have a wide range of models to choose from that can power AI agents to solve real business problems. But choosing a model doesn’t guarantee you effective agents or even good performance. For that, you need to run your agents in an environment that provides them with tools, state, security, and scale, with fast startup times and good integration with your existing business systems.
Picking the right agent runtime environment is like picking an enterprise app server but for AI systems — and agents have very different needs from traditional applications.
Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs.
Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs. Agents that deliver in proof-of-concept systems will fail in production if the runtime stack powering them can’t keep up and keep them under control.
Agentic compute is different
It’s easy to think of an AI agent as just another microservice that takes unstructured input, runs APIs or queries, and returns messages. But infrastructure designed for traditional enterprise applications with predictable business logic or even cloud-native stateless workloads doesn’t fit agents with their bursty, long-running, stateful, non-deterministic, code-writing, tool-invoking behaviors that might be triggered by a system event or an email — not just a chat session.
Model inference needs GPUs for speed, but agents also need reliable, durable compute that supports stateful sessions for long-running processes, along with strong security and real-time visibility.
You still need to think about familiar issues like hosting, scaling, identity, and security, but all that is complicated by the unpredictable, multi-stage workflow of the agent reasoning loop.
An agent pulls in input from multiple sources, reasons over its context about the execution plan for accomplishing the goal, calls other tools or writes its own code, iterates over the results of those calls, and maybe builds on them or switches to another approach that requires another reasoning loop and eventually delivers output. That might be updating a system or sending an email rather than just displaying an answer.
Model performance is only one part of making that useful. Architecting a successful agent system that can run at enterprise scale requires considering the agent runtime, the application layer that agents call, and the tools, APIs, and MCP servers they consume.
You have to be able to integrate with business logic and existing systems, manage the usual quotas, rate limits and SLAs for APIs so agents don’t overload them — and you have to do all that while keeping up with AI developments that are moving too fast for you to build the infrastructure primitives you need to rely on from scratch every time.
Requirements of modern agentic infrastructure
Instead, you should look for flexible infrastructure that fits the way the agent works. As with any technology, if you build on an existing platform like Azure Container Apps, you can save effort in areas where your business can differentiate. And while AI agents have flaws (from hallucinations to high token costs) that aren’t fully solved, you can pick an agent runtime environment that makes it easier to get useful results despite them.
Agentic compute needs fast startup and resume. Whether it’s a human typing into a chat prompt or system events automatically launching multiple agents, agent infrastructure needs to spin up quickly. If it takes a few seconds to spin up a container, the reasoning loop can’t start till that’s done.
It also needs to scale up and down responsively, without costing you anything when it’s not running.
Because agent workloads are long-running and event-driven, agents need to start up, do some work, go idle for hours or even days, and then resume instantly with their memory, context, caches, connections, identity, and security intact. That means persisting and restoring state so long-running agents don’t have to pay the same startup tax over and over again. Whether the last stage of the agent’s reasoning loop was successful or a failure, it has to take another approach; you don’t want it to do the same work again.
Rather than building your own custom microVM stack, the new Azure Container Apps Sandboxes provide a temporary, secure, and stateful compute environment that spins up, executes code, snapshots disk and memory, then automatically idles and resumes just as fast. This is the stateful equivalent of Dynamic Sessions, with sub-second startup from pre-warmed pools; you can burst out to hundreds (and eventually thousands) of concurrent sandboxes when you need them, then scale back down to zero.
An agent runtime needs to be secure by default because agents are only useful when they take action — and by definition, they’re likely to do unexpected things, write and execute their own untrusted code, and keep trying to achieve their goal (sometimes even when there’s a policy that should stop certain behaviors).
To give agents secure, auditable access to the resources they need, run them and the code they generate in a sandbox, rather than on a developer laptop with production credentials and admin rights. Identity, access control, and execution boundaries have to be enforced at the runtime layer, not bolted on in a hidden prompt.
ACA Sandboxes have egress and access policies, so you can control outbound calls and limit what URLs they can access. If you can’t use managed identity for all the services an agent needs to connect to you, you can protect secrets by injecting API keys through an external egress gateway instead of hard-coding them. Because the gateway is external, outbound call decisions are controlled by policy outside the sandbox, not by agent code that will relentlessly try any method to get through.
If agents are useful, you’re going to run a lot of them, often simultaneously. AI agents need strong isolation for each task, so every untrusted code execution happens in its own sandbox and no data leaks between tasks.
You can manage them as groups, but each ACA Sandbox has its own secure boundary, so details from one customer support agent won’t end up in a chat with a different customer. If the untrusted code does turn out to be problematic, isolated sandboxes at least contain the blast radius.
Manage the tools agents call
You don’t want to rely on fragile glue code or ad hoc orchestration that you have to rewrite any time systems change for the calls agents make to tools, APIs, cloud services, and other workflows. You also don’t want to rely on manual cleanup of resources no longer needed. A runtime with built-in agent tool execution makes agents more robust. It takes the drudgery out of connecting the agent environment to your other systems, enabling them to interact securely, in isolation, and at scale.
“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take.”
“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take,” points out Vyom Nagrani, who runs the team of PMs responsible for both Azure Container Apps and the Azure SRE Agent that’s built on ACA Sandboxes. Sandboxes have access to a connector framework with over 1,400 enterprise-grade connectors, enabling them to take actions not only on Azure and Microsoft services but also on third-party tools.
“The runtime provides those connectors, and it provides a managed way of authenticating against all of these third-party systems, so now you can build an agent which can talk to many, many, many different systems. It’s not boxed into one authentication boundary,” Nagrani tells The New Stack.
MCP servers are a built-in capability of the connector framework. “You can take any of these connectors; you can take any REST API and expose it as an MCP server, which then the agent can consume.” Or if you want to build a custom MCP server, you can host that in a sandbox too. ACA Sandboxes can be both where agents run and, if that’s appropriate, where the tools they use are hosted.
If you’re building a user interface to wrap your agents, the application layer that makes calls to the agents can run in Azure Container Apps Express, a new service now in public preview, Nagrani says.
“It’s a simplified app hosting stack for the app layer that responds to HTTP traffic and serves web traffic: that’s where the human interactivity comes in.”
Putting it all together
ACA Sandboxes offers an agent runtime environment that answers the key questions architects need to consider: where agents and the ephemeral compute they need access to run; where the application layer that calls agents runs; and how agents get access to all the tools, APIs, services, MCP servers, and existing business logic they need to orchestrate.
Whether you’re a platform engineer, a software vendor or a startup building a new AI platform, treating agent runtimes as the new application server and MCP servers as the new APIs requires agent infrastructure that supports agent workloads effectively, allowing governance to shift left into the runtime layer where it can scale with the ever-increasing numbers of agents.
How real platforms build on agent runtimes
Azure Container Apps is already a strong platform for running agents. Auger, a startup launched by the former CEO of Amazon’s global consumer business to help mid-size enterprises get their complex supply chains out of Excel spreadsheets, used it to build a multi-agent system that can give real-time answers about shipments and forecasts in a world where mines or critical shipping lanes might be closed at any time by war or weather.
Backend agents pull the unstructured data with all the details to answer those questions from different siloes, building ETL pipelines and creating an ontology of the supply chain ecosystem for each customer that includes functions and actions — all of which need to be audited and reversible. Frontend agents use that ontology to answer questions such as, “What happens if I build a new warehouse here or switch to a supplier in this country?”
ACA Sandboxes are built on Azure Container Apps, giving organizations an easier way to create agent services with strong isolation, dynamic scaling, fast startup, persistent state, and broad tool and service integrations.
That’s just what the Foundry team was looking for when they started work on the Microsoft Foundry Agent Service managed agent runtime. ACA Sandboxes gave them a platform that delivers fast start/resume, built-in tool execution (including for untrusted code), persistent state for long-running agents, strong per-agent isolation, and secure-by-default operations. Agent identity, “on‑behalf‑of” authentication to existing business services, and strict isolation are critical capabilities they didn’t have to build themselves.
“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”
The Foundry Agent Service adds a layer of visibility and observability into agent actions and conversations, allowing customers to monitor agent performance and see where the system is doing well and where it needs improvement.
“Azure Container Apps Sandboxes package the hard parts of an agent runtime into a first-class Azure resource — sub-second start and resume, built-in execution of tools and untrusted code, and snapshot-based state that lets long-running agents pick up exactly where they left off. Every agent gets its own hardware-isolated environment with secure-by-default operations, so builders can focus on what their agents do, not on the plumbing underneath,” Nagrani points out.
“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”
A context platform is a governed layer that holds the metadata, business knowledge, operating procedures, and documentation an AI agent needs. It delivers that context through access controls, policies, and audit trails. For industrial teams, that layer can be the difference between a demo that looks useful and an agent that can be trusted near […]
LoginAI Infrastructure Archives - The New Stack
Robotics & Automation News
NVIDIA Technical Blog
