Perplexity shipped Portable Computer this week, the local-first version of its Computer agent running on an Nvidia DGX Spark workstation, and the bill is steep: A DGX Spark starts at $4,700, and even an aging 24GB RTX 3090 sells well above $1,500.
The architectural choice beneath the surface deserves as much attention as the price tag: Most platforms building agents address reliability with greater intelligence. That typically includes a larger orchestrator model, a planner model, or a critic model reviewing the work.
Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.
Perplexity separated the two jobs instead of stacking them. Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.
The loop controller is code, and the decisions are still a model
Perplexity uses the term “orchestrator” to refer to the runtime controller rather than the planning model. That controller assembles context, enforces policy, and executes approved tool calls inside an OS-level sandbox. The company says it is deterministic code, not yet another model. The local model proposes the next action, including which tool to call and when to ask a cloud advisor for help. The split resembles a control plane architecture, where reasoning suggests, and inspectable software retains authority. Nate Kupp, Perplexity’s vice president of Computer Enterprise and Infrastructure, tells The New Stack that the harness accounted for most of the engineering work.
Same weights, better scores
Perplexity kept the base model and the silicon constant, putting Qwen3.8-27B on the same DGX Spark across three agent stacks. On its internal Local Knowledge Work Bench, a held-out set of 53 tasks, the company reported 82.6% for Computer. Pi scored 77.6% and Hermes 74%.
On ParseBench-100, a subset covering charts, layouts, tables, and formatting, the gap widened considerably. Perplexity reported 65.1% for Computer, while Hermes reported 34.6% and Pi reported 13.9%.
Because the base weights were constant, the gap measures the system around the model rather than a better model. That does not mean weights stopped mattering. Perplexity post-trained Qwen into PPLX 27B and reported 85.4%, above its own base-model number. Harness engineering and post-training are significant factors of this approach.
The word harness also covers a lot of ground, including prompts, tool schemas, context management, verification hooks, and document processing. Orchestration code is one part of that surface.
The security boundary lives outside the model
The sandbox is the boundary, not the determinism. Perplexity says the sandbox restricts processes, filesystem paths, and network access. If the sandbox is unavailable, the harness disables itself before making any tool call. Deterministic code is valuable here because it makes policy easy to inspect and helps the system fail safely. Deterministic code can still ship a vulnerability or faithfully execute a permitted mistake.
The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.
The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.
The same discipline shapes how context gets spent. Perplexity reports that Qwen3.8-27B advertises a 260,000-token context window but begins to struggle beyond 100,000 tokens. The harness therefore keeps the core prompt and toolset small and loads skills on demand. Commonly used connectors became command-line tools rather than full Model Context Protocol definitions sitting permanently in context.
Deterministic execution cannot rescue reasoning that exceeds the local model. On Terminal Bench 2.1, Perplexity reported 59.6% running locally. Letting the local agent consult Claude Opus 5 raised it to 73.0%, compared with 82.4% when Opus 5 worked alone. All of this remains vendor-reported evidence on a bench that the company has yet to open-source.
For enterprises evaluating local agents, the component to scrutinize is the layer that grants and denies authority, because that is where the platform’s engineering is most evident.
Put several AI bots to work, and a mistake by one may not stay within its assigned task. For example, that error could reach another bot’s files and login credentials, or even the computer running them all. Two releases this month offered companies very different ways of containing that risk.
On August 17, Nous Research announced that its Bot Mode would ship bundled and enabled by default in Hermes Agent v0.20.3, turning agent profiles into a roster of named bots that hand off work to one another. About a week earlier, SpaceXAI launched Grok Bot with almost the same interface: a sidebar of named teammates who sign in to your tools and keep working long after you close your laptop.
The interface converged within a week, but the answer to the question every platform team has to ask did not: When one bot goes wrong, what can it reach?
Four projects have now come to their own answer, and no two of them agree.
Grok Bot draws the line around the user account.
Hermes draws it around the profile.
OpenClaw draws it around an optional runtime sandbox.
Taken together, the documentation shows an industry converging on the persistent coworker interface far faster than it is converging on what constitutes an identity or a security boundary for it.
Four projects, four written answers
Every one of these products now offers the same surface. You create several named agents, assign them different jobs, and have them pass work among themselves. The naming convention alone suggests separation, since a bot called Expense Manager and a bot called Talent Scout sound like they occupy different rooms in a shared office.
The documentation says otherwise, and it says something different in each case. The unit of isolation is the account in one product, the profile directory in another, an opt-in container in a third, and the deployment topology in the fourth. Those four units are not interchangeable, and an operator who assumes the roster itself is the boundary will be right in exactly one of the four cases.
Is Grok Bot confused about what it wants to be?
SpaceXAI’s launch post leads with the promise that bots have their own computer. The documentation, last updated the same day, describes a single persistent cloud computer assigned to the user account rather than to any individual bot. Browser cookies and signed-in sessions are shared across the roster, files are visible to every bot, and command-line credentials are shared. One bot can pick up work that another bot saved.
Each bot gets its own screen on that machine, which allows several of them to run browser and desktop tools in parallel. SpaceXAI is direct about what those screens are not. The documentation calls them “separate work surfaces, not separate security boundaries.” It then instructs operators to keep a credential or file off the machine entirely if another bot on the account cannot use it.
The consequences run further than credentials. Signing in for one bot makes that session available to the others because the browser is shared. Installed connectors are account-wide, and their availability is not isolated to a single bot. The shared workspace sits at /workspace and is designed to survive computer updates and recovery, so the durable state is shared across the whole roster.
None of this is an implementation accident. It is what makes handoffs between bots cheap, and cheap handoffs are the product. But an operator reading only the launch page would build a mental model that the documentation contradicts, and that gap is where the risk sits.
Hermes gives each bot its own profile
Nous took the opposite architectural position. In Hermes, a bot is a profile, and each profile has its own configuration, memory, skills, credentials, and chat history stored in its own directory on disk. Handoffs between bots run as real invocations against the named profile, rather than as a shared context blob passed around within a single process.
Nous shipped the teammate protocol as part of v0.20.3, alongside the MCP 2.x SDK migration and a set of runtime hardening changes. The company archived the standalone plugin repository once the merge was completed. Bot Mode is on by default, so a Hermes user who updates gets the roster without opting in.
Two qualifications matter before anyone reads that as containment. A separate credential store does not guarantee different credentials, since what ends up in a new profile depends on how the operator created it and what they edited afterward. And every profile still shares the host machine, its operating system user, and its filesystem permissions. What Hermes documents is workstation-level separation of agent state, a meaningfully stronger default than a shared cloud account, but not the same as isolation.
OpenClaw’s sandbox is off by default
OpenClaw documents the most complete boundary of the four. When the sandbox is enabled with the Docker backend, agent tool execution runs inside isolated containers. At the same time, the gateway remains on the host, and the scope can be selected per session, per agent, or shared across agents. Each scope gets its own workspace. Auth material lives per agent under an agent-scoped auth profiles file. Operators can configure network isolation, resource limits, and allow-or-deny tool policies on top of it.
The documented default for that sandbox mode is off. That is a defensible choice for a project most people run on a laptop, where the container overhead buys little against a single-user threat model. The underlying setup behavior deserves more attention. If sandbox prerequisites fail during setup, the script resets sandbox mode to off rather than refusing to start, so an operator who intended isolation and encountered a Docker socket issue ends up running without sandbox isolation. The documentation also warns against mounting the host Docker socket into agent sandbox containers and flags the CLI container’s shared network namespace with the gateway as a trust boundary in its own right.
ClawFleet answers the same question by moving it into the deployment topology. The project documents a wrapper that puts each OpenClaw or Hermes agent in its own Docker container with an isolated filesystem and network. It lists roughly 500 MB of memory per OpenClaw instance and 150 MB per Hermes instance. That cost is why the other three projects make the boundary optional or skip it, and naming the number makes the trade-off legible.
How to choose which bot is right for you
Scenario
Documented fit
Rationale
Persistent work that must continue with the laptop closed
Grok Bot
The only one of the four with a vendor-run always-on cloud computer, at the cost of one shared credential surface for the whole roster
Several agents with genuinely different credential sets on one workstation
Hermes
Per-profile stores are the documented default, and Bot Mode ships on
Untrusted or multi-tenant agent sessions
OpenClaw with sandbox enabled
Per-agent or per-session container scope with configurable network and tool policy, provided the operator turns it on and verifies it
Isolation as the deployment model rather than a runtime setting
ClawFleet
Container per agent with separate filesystem and networking, at a documented memory cost per instance
Each project gives operators different advice
The operational guidance diverges as sharply as the architecture. OpenClaw’s docs read like infrastructure documentation, naming specific hazards such as the Docker socket and the shared network namespace, and telling the operator what not to do. Hermes documents the profile layout and the protocol, then leaves policy to the operator. Grok Bot’s guidance is largely the warning itself, an instruction to treat the account as the boundary and to keep sensitive credentials off the shared machine entirely.
Grok Bot carries a second disclaimer worth reading alongside the first. Sensitive actions route through an approval mechanism. SpaceXAI documents the categories that trigger it, including sending messages, publishing content, purchases and transfers, deleting data, and touching production. Enforcement runs through an LLM classifier. Cursor‘s documentation for that same engine states plainly that the classifier is not a security boundary and can make mistakes. A buyer evaluating the product therefore finds the phrase twice, attached to the two mechanisms they would most reasonably assume protect them.
AI agents still lack identities of their own
Enterprises can adopt any of these four products today and get real work done, and the honest reading is that all four are engineering their boundaries in good faith against different threat models. What none of them provides is an identity for the agent. In every case, the bot borrows the operator’s credentials, whether from a shared cloud browser, a profile directory, or a container volume, and the entire security conversation boils down to how far those borrowed credentials travel.
“There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled, the account, the profile, the runtime, or the container.”
That is why the four answers differ so much. There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled: the account, the profile, the runtime, or the container. Expect to see that gap close on the identity side rather than the agent side, through scoped delegation and per-agent credentials issued by the identity provider, rather than being copied from the human. Until then, the useful move for platform teams is unglamorous and specific. Read the security page before the launch page, because for this class of product, they describe different things.
OpenAI has detailed how the GPT-5.6 model family balances capability and cost across its stack, and the company‘s most important claim is a benchmark result showing that its flagship model, GPT-5.6 Sol, with maximum reasoning, outperforms Claude Fable 5 from Anthropic on the Artificial Analysis Coding Agent Index. The margin comes with 54% fewer output tokens. The findings were shared in a company blog post on Wednesday.
For developers, what matters most is how OpenAI arrived at the benchmark results and the role GPT-5.6 Sol played in optimizing the infrastructure that now serves it.
The family spans three models across the price curve. In addition to Sol, there is Terra, which performs as well as GPT-5.5 on intelligence benchmarks at half the price, and Luna, the fastest and most affordable, which is priced 80% below Sol.
The efficiencies come from optimizations at four layers, spanning the models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work.
According to the post reviewed by The New Stack ahead of its publication, the efficiencies come from optimizations across four layers: models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work. The architecture diagrams in the post draw the same separation as three planes: the local harness, CPU-bound API orchestration, and GPU-bound model inference.
Source: OpenAI
For developers building and operating agents, the post is worth reading less as a product announcement and more as a systems paper. Nearly every technique it describes, from incremental tokenization to append-only context, applies to any team running a tool-calling loop at scale.
A model that rewrites its own serving code
The efficiency work starts in training. OpenAI says GPT-5.6 is trained to achieve more work per token, with training optimized for both task success and efficiency so the model takes a more direct path through a task.
With Codex, GPT-5.6 Sol autonomously rewrote and optimized OpenAI’s production kernels, the core code that executes the mathematical operations making up the model. OpenAI says this worked in part because GPT-5.6 is trained to write and improve kernels in Triton and Gluon. Both are open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from the model, reduced end-to-end serving costs by 20%.
Correctness is the obvious concern when a model rewrites the code it runs on. To address it, OpenAI reports heavy investment in verification tooling. That includes the open-source Floating-Point Sanitizer (FpSan), which validates the kernels GPT-5.6 Sol produces before they reach production.
The model went further with speculative decoding, a technique in which a smaller draft model proposes several tokens that the primary model verifies in parallel. The approach will feel familiar to anyone who understands how modern CPUs speculatively execute instructions ahead of a branch. Accepted proposals produce multiple output tokens from a single pass of the primary model. That reduces the expensive sequential computation the primary model would otherwise perform.
GPT-5.6 Sol in Codex improved its own draft model by designing and running hundreds of experiments on its architecture, with changes tested across size, structure, and features. The model also launched and monitored the speculative training process. It intervened autonomously when hardware failed or training became unstable. OpenAI reports the resulting improvements lifted token-generation efficiency by more than 15%.
More tokens from the same GPUs
OpenAI frames its inference work around a single objective – serving more tokens with the same hardware while preserving the intelligence, latency, availability, and reliability users expect. In a compute-constrained market where demand grows faster than capacity, that objective influences every design decision in the serving path.
Load balancing operates at three distinct levels. Globally, requests are routed based on geography, available capacity, and accelerator type. Within a cluster, work is distributed across model instances based on load, context length, and cache availability. Within each instance, work is partitioned across accelerators, the model’s experts, and computing cores. GPT-5.6 Sol in Codex helps OpenAI analyze production traffic and identify previously overlooked sources of imbalance. The same loop tests new routing strategies and helps engineers constantly tune the heuristics. OpenAI states that these load-balancing improvements alone dramatically reduced the cost of serving its models.
The key-value (KV) cache received the same treatment. When processing uncached input tokens, the model builds the KV cache in a single compute-intensive pass, then repeatedly reads from and extends it during generation. The optimal serving configuration depends heavily on prompt length, batch size, and cache hit rate. It covers batching, sharding, and cache management, and the configuration space was previously too large to tune systematically. With GPT-5.6 Sol in Codex, OpenAI analyzed production workloads and generated candidate configurations. The company says this makes workload-specific optimization practical at a level that broad heuristics could not reach earlier.
Process only what changed
The API team focuses on everything that happens around a model call. After a prompt is submitted, the API stack receives the request, loads context, and validates the input. Safety checks run next, and the text is converted into tokens for inference. OpenAI measures this overhead through time to first token (TTFT), time between tokens (TBT), and end-to-end time (E2E).
Tokenization is an O(n) operation, so longer prompts take longer to process. Codex would send the full conversation context after every tool call. That meant paying to tokenize the same conversation dozens of times per turn, even though only a small amount of context was new in each request. OpenAI solved this with a WebSocket integration that hoists tokenization state to the server. The first call renders and tokenizes the full prompt. Later calls send only the new input with a reference to the conversation, bringing the operation closer to O(1). The pattern mirrors an incremental build system that recompiles only the files that changed rather than the whole project.
These savings compound in tool-heavy workflows, where every tool result triggers another round trip through the API. For rollouts with 20 or more tool calls, OpenAI reports up to roughly 40% faster end-to-end execution.
Hardware turned out to matter as much as protocol design. All of OpenAI’s infrastructure runs on Kubernetes. The company found that nodes with the same instance type often carried different CPU generations, with many running outdated processors. In its measurements, the older processors consumed roughly twice the CPU resources for the same work. Reweighting traffic toward newer processors improved TTFT by about 20%, and CPU generation is now part of capacity planning.
OpenAI names four fates for application-layer overhead: delete it, overlap it with useful work, run it on faster hardware, or make the code consume fewer CPU cycles. Its asyncio changes move work off the critical path, while newer hardware and Rust implementations make the remaining work faster and more predictable.
An append-only harness
The agentic harness is a Rust-based orchestration layer that connects the models, tools, and the user’s environment. In a single turn, Codex might inspect source code, search deployment history, and read incident reports. Editing a file and running the tests each add another request. Since a task can require 30 model requests, an extra second per request adds up quickly.
Context bloat is the first target for the harness. As agents gain access to more tools, skills, plugins, and conversation history, context windows expand. The growth increases cost, distracts the model, and prompts unnecessary reasoning. The harness counters this with deferred discovery, which surfaces integrations, custom Model Context Protocol (MCP) tools, skills, and plugins only when needed. Tool output is capped at 10,000 tokens by default unless the model requests a different limit.
Prompt caching drives the second design choice. An agent loop resends the same instructions, tool definitions, and earlier results multiple times within a turn. The harness therefore treats all model-visible history as append-only, with new messages and tool results added at the end rather than inserted into earlier context. Tools are presented in a deterministic order, and runtime settings, such as approval policies, are applied during execution rather than embedded in tool definitions. OpenAI credits this design for the high prompt-cache hit rates in Codex and ChatGPT Work.
Source: OpenAI
Platform teams building internal agents can adopt every one of these choices without OpenAI’s scale. Append-only context, deterministic tool ordering, and capped tool output attack token spend directly. That makes them the most portable lessons in the post for enterprises watching inference bills grow with each new agent deployment.
Where the gains come from
The post associates a number with most of its optimizations, and the figures are OpenAI’s own production measurements. Taken together, they show how modest individual wins compound across a serving stack.
Layer
Technique
Claimed gain
Model inference
Autonomous kernel rewrites in Triton and Gluon
20% lower end-to-end serving costs
Model inference
Speculative decoding with a self-improved draft model
Over 15% better token-generation efficiency
API stack
Stateful WebSockets with incremental tokenization
Up to roughly 40% faster runs at 20+ tool calls
API stack
Routing traffic toward newer CPU generations
About 20% better time to first token
Agent harness
Deferred discovery and a 10,000-token tool output cap
Reduced context bloat and cost
The key takeaways
In summary, OpenAI describes the GPT-5.6 efficiency gains as the result of years of compounding improvements. They span research, inference, the API stack, and the agentic harness. The company states that the model’s role in landing many of them makes it optimistic that the pace of optimization will accelerate. Kernel work is called out as an area of continued investment.
The post positions efficiency, alongside raw intelligence, as the axis on which frontier labs now compete. The claimed 54% output-token advantage over Claude Fable 5 shows how OpenAI intends to fight that battle. The engineering blog makes a plausible case that software optimization is becoming an important lever alongside hardware improvements in reducing the cost of serving frontier models. The figures remain OpenAI’s own production measurements. The autonomy on display operates within Codex, with engineers in the loop. Developers and enterprises benefit either way, as these under-the-hood improvements reach them as more capable models at lower prices across the cost-intelligence curve.
Over two days this month, two of the most credible people in the industry published framework manifestos on X.
Microsoft CEO Satya Nadella posted “The Reverse Information Paradox” on July 12.
And then Google DeepMind CEO Demis Hassabis posted “A Framework for Frontier AI and the Dawning of a New Age” on July 14.
When read side by side, each proposal reinforces the layer where its author’s company is already strongest.
Two manifestos, two boundary lines, one week
Nadella’s argument is about value capture. Enterprises pay for AI twice, he writes in the Reverse Information Paradox, once in tokens and again in the proprietary know-how they leak back into the model through prompts, corrections, and evals. His fix is to own the learning loop — meaning the data, the traces, the evals, the adapted weights, and the memory — then put a model-agnostic orchestration layer on top so any model stays cheap and swappable. Make the model a commodity and the value flows to the layers around it.
Hassabis is drawing a very different line. His concern is not who captures the value but who governs the risk. In his article, he calls for a standards body modeled on FINRA, industry-funded and subject to government oversight, that tests frontier models for cyber, bio, and deception before they ship. Labs would submit models up to 30 days before release, voluntarily at first, then as a hard gate for deploying in the US market.
Both frameworks route through the author’s own strength
Neither framework offers a neutral read of the field. Nadella’s advice to own your data and keep models swappable is correct, and it also routes enterprises straight to Azure and Foundry, where the orchestration, billing, deployment, and governance stay with Microsoft no matter which model wins. The overlap between that advice and the product stack Microsoft already sells is difficult to ignore.
Hassabis’s gate subtly emphasizes the importance of scale. Large incumbents can absorb testing costs, run sophisticated safety teams, and shape standards more easily than smaller challengers. Google DeepMind already operates its own internal Frontier Safety Framework, so an incumbent with an established safety apparatus would likely begin with a compliance advantage, especially if major labs influence the eventual protocol. That does not prove Hassabis planned it this way, but the incentive still leans toward the same outcome the proposal describes.
The value moves toward the layer each one governs
The pattern in both posts is that they have stopped competing solely on benchmark leadership and now compete over the systems that govern how models are used and released. Nadella wants the decisive layer to be the enterprise boundary, so the model provider cannot harvest the customer’s knowledge.
Hassabis wants it to be the frontier gate, so nobody deploys a top-tier model without an industry-run body signing off on it. Different layers, but the same instinct: put durable value where your company is already positioned.
The counter is that Hassabis proposes a wider net than self-interest alone would draw. His regime would cover every frontier model, regardless of national origin or openness. It would get independent experts and open-source representatives on the board and answer to the US government.
Those provisions complicate any claim that the body would simply serve Google. Read most fairly, both frameworks may solve real problems while also favoring the firms best equipped to supply the solution.
The model is no longer the only scarce asset. Value is moving toward whoever controls the data boundary, the deployment layer, and the rules of admission, and this month, two of the most influential in the AI ecosystem staked a public claim to control them.
Over the past nine months, Amazon, Microsoft, and Google have each introduced or renamed an enterprise agent platform. And all three have converged on the same core architecture. Runtime, memory, tool gateway, identity, observability, and governance now appear in Amazon Bedrock AgentCore, Microsoft Foundry, and the Gemini Enterprise Agent Platform, albeit under slightly different names. Each vendor presents these components as the foundation for production agents. What was a fragmented collection of libraries just 18 months ago is becoming a distinct platform layer.
To see where that shift may lead, consider the evolution of platform as a service. Between 2011 and 2016, developers built applications from virtual machines, load balancers, message queues, secret stores, and monitoring agents — each with its own API and operational model. Cloud Foundry and Heroku unified those pieces around an application contract. Developers could think less about machines and more about applications. The agent ecosystem is approaching a similar inflection point, but it still lacks an equivalent contract — and no open-source project has stepped forward to define one.
The plumbing a production agent still needs
Imagine a platform team putting a customer support agent into production this quarter. They choose a model provider, then a framework, and finally a place to store session state and long-term memory. They add a tool gateway so the agent can reach the ticketing system. An identity layer follows, so the agent acts on behalf of the person who asked, and a sandbox keeps generated code somewhere safe. Evaluation and tracing are considered last, once someone asks how quality regressions will surface before customers find them.
Each of those choices looks small on its own. When combined, they decide which cloud the workload lives in. Session state resides in a single provider’s managed store. The traces are logged in that same provider’s telemetry service, and the agent’s identity is derived from its directory. Moving that agent a year later means rebuilding the whole assembly, which is where enterprises were before PaaS gave applications a portable shape.
What Cloud Foundry got right before it lost the market
Cloud Foundry reduced application deployment to a single command, and the platform took responsibility for everything after it. Buildpacks detected the language and produced a runnable artifact. Service brokers provisioned a database or a message broker and bound the credentials into the application environment. Routing, logging, autoscaling, and health checks arrived as platform behavior, not as a queue of tickets aimed at an operations team.
What mattered was the contract, not the implementation. An application declared what it needed and stayed agnostic of where it ran. Buildpacks began life at Heroku back in 2011. Pivotal and Heroku started the Cloud Native Buildpacks project in January 2018, and the CNCF accepted it that October. A PaaS idea outlived the platform that produced it.
Cloud Foundry never became the dominant platform. Kubernetes did, and the Cloud Foundry community eventually rebuilt its abstraction on top of Kubernetes through Korifi. The design principles traveled anyway, and enterprises running that platform in 2016 had portability that most agent teams cannot buy today.
The same primitives across three clouds
Let me dissect the three platforms because the similarities are conveniently hidden behind the branding.
AgentCore reached general availability in October 2025 with seven composable services, namely runtime, gateway, memory, browser, code interpreter, identity, and observability. The runtime offers eight-hour execution windows with complete session isolation. The gateway connects to existing MCP servers and turns APIs and Lambda functions into agent-compatible tools, while observability exports through OpenTelemetry into CloudWatch.
Microsoft renamed Azure AI Foundry to Microsoft Foundry effective January 1, 2026. Foundry Agent Service covers the same ground. Microsoft documents hosted agents running in a session-isolated managed runtime, with Entra Agent ID handling identity. Managed memory spans session, user, and procedural scopes, and tracing is built on OpenTelemetry.
Google retired the Vertex AI name at Cloud Next 2026 and folded the platform into Gemini Enterprise Agent Platform. What was Agent Engine became Deployments, and Memory Bank, Sessions, Agent Registry, Policies, and Gateways sit alongside it under an agent-first information architecture.
The convergence is rational behavior rather than a conspiracy, since infrastructure companies build vertically integrated platforms because integration is where the margin lives. The consequence lands on customers rather than on the vendors. Identity, telemetry, and deployment all terminate within a single provider, which makes the operational layer beneath the agent the part that resists moving. Recently, I analyzed the runtime aspect of this problem in a piece about Google’s Agent Substrate.
The contract an agent platform would inherit
Any agent platform can be tested by one question: what would a Cloud Foundry-style contract look like if it had been written for agents rather than for web applications? The mapping is close enough to be instructive and different enough to be interesting.
PaaS abstraction
Agent platform equivalent
Where portability breaks today
Application source
Agent code, instructions and evaluation suite
Each framework defines its own package shape
Buildpack
Framework detection and agent packaging
No shared build contract across SDKs
Backing service
Model, memory, retrieval or tool provider
Providers are wired into agent logic
Service binding
Authenticated attachment of tools and data
Credentials are issued by the host cloud
Router
Agent endpoint, MCP or A2A interface
The protocols exist but not the lifecycle
Logs and metrics
Traces, tool calls, cost and quality scores
GenAI conventions remain in development
Release promotion
Evaluate, version and progressively deploy
Evaluation couples to one vendor’s harness
Platform policy
Agent identity, permissions and approvals
Identity ties to the provider’s directory
Real deployments will mix these rows rather than adopt them cleanly, and no team needs the complete set from day one.
Package the agent as one deployable unit
The thing developers ship has to be versionable, testable, and movable as a single artifact. The code, the instructions, the tool dependencies, the memory contract, the permissions, and the evaluation suite all travel together, or none of them do. AWS gets close with its harness export path. One command turns a configured harness into Strands code, and AWS says the export preserves the model, prompt, tools, memory wiring, and container environment. Customers can easily move from configuration to code without an architecture rewrite, which is the right instinct pointed at a single cloud.
Bind capabilities rather than embed providers
The Twelve-Factor App taught developers to treat databases and caches as attached resources reached through configuration. Models, memory stores, retrieval engines, browsers and tool gateways deserve the same treatment. An agent that names its model provider inside application logic has already given up portability, whatever the platform brochure claims.
Make operations part of the abstraction
Rather than just enabling deployment, PaaS proved its value by offering built-in capabilities like routing, logging, autoscaling, and rollback. The questions that matter for agents look different, and they should be the ones enterprises must ask. Platform teams want to know whether the agent completed the task and whether it chose the right tools. They also want to know whether it exceeded its authority, what the run cost was, and whether quality regressed after a model update.
How agents differ from applications
Agents are not web applications with a model attached, and a platform built on that assumption will fail in production. Three differences carry most of the weight here. Agent behavior is probabilistic, so two identical inputs can produce different tool calls. Agents act with delegated user authority, which turns a permissions bug into a real-world side effect rather than an error page. An agent’s dependencies can also change its behavior without any code deployment. A model update or a revised tool description alters what the agent decides to do.
The Twelve-Factor rule that processes should be stateless does not survive that. An agent platform needs disposable execution workers alongside durable, inspectable, and portable agent state. LangGraph already demonstrates the combination in open source, with checkpointing at every step, first-class human interrupts, and execution that resumes after a crash. The control plane around it is part of the commercial LangSmith product, covering deployment, evaluation, and observability. The fragmentation appears within a single project.
What the open protocols leave out
Most of the primitives a neutral platform would need already exist. The Model Context Protocol standardizes how agents access tools and data. A2A covers discovery and communication between independent agents. OpenTelemetry is defining GenAI conventions for agent spans, tool calls, and token usage, though most of those attributes remain marked as in development. OCI images stay available as the packaging escape hatch for anything a managed runtime cannot host.
The vendors have already conceded that a neutral layer matters. The Linux Foundation announced the Agentic AI Foundation in December 2025. Its founding projects were MCP from Anthropic, goose from Block, and AGENTS.md from OpenAI, and AWS, Google, and Microsoft joined as platinum members. Google also moved A2A into the Linux Foundation.
Protocols are not the same thing as a lifecycle platform. A foundation that governs how agents talk to tools says nothing about versioning an agent. It says nothing about promoting that agent through environments, or rolling it back when an evaluation regresses. Enterprises running diligence on agent platforms can work through three plain questions. The first is governance: whether the project is controlled by a neutral foundation or by the vendor selling the managed version. The second is packaging, where the same agent artifact should run on two different clouds without a rewrite. The third is state, where memory has to live somewhere the enterprise can export from. No open project answers all three today.
Where is this headed?
Kubernetes defined pods, deployments, and services, and those abstractions influenced how an entire industry thinks about running software. The equivalent agent abstractions have not been fixed yet. Whoever ends up owning the agent control plane will not merely own deployment. That owner defines what an agent is, which components it contains, and what a platform team is permitted to swap out.
In summary, hyperscalers are building robust platforms that effectively address operational questions. They also answer them inside a single cloud. If a neutral project assembles the same lifecycle atop the protocols already in place at the Linux Foundation, enterprises will regain the negotiating position that buildpacks and service brokers once gave them. An open, cloud-agnostic agent platform would benefit vendors as much as buyers, because a stable contract is what allowed the cloud-native ecosystem to grow beyond any single provider.
Google made GKE Agent Sandbox generally available in May 2026 and, in the same post, introduced a second project called Agent Substrate. These two announcements concede a point that Kubernetes veterans have been reluctant to call out.
That indirect admission is that the platform that won the container decade is not the right control plane for AI agents. Agent Sandbox provides agents with a secure environment to run untrusted code. Agent Substrate adds a scheduling layer that routes around the Kubernetes control plane because the API server was never designed for how agents behave.
Compare agents to processes in an operating system rather than services in a data center, and the mismatch is obvious. A modern OS runs thousands of processes that spend most of their life asleep. It wakes each one on an event, hands it a slice of a CPU, then pages its idle memory out to disk to make room for the next. Agents behave almost exactly like those processes. Kubernetes was originally created to manage a fixed set of long-running, replicated services. This fundamental design explains why much of the agent infrastructure now runs on Kubernetes rather than being integrated into it as one of the workloads, such as a Deployment or a StatefulSet.
What an agent actually is as a workload
An agent is a long-running, stateful session that stays idle for most of its life, wakes to execute a burst of code, then goes quiet again. The code it runs is generated by a model at runtime. The host has to treat it as untrusted by default. Each session needs a stable identity, the ability to pause and resume without losing memory, and hard isolation from its neighbors.
Think of the agent as a process in a time-sharing OS. Just as the scheduler suspends a sleeping process and restores it the moment a keystroke arrives, an agent runtime must hibernate an idle session and restore it with its working memory intact. The wake path is where the user is waiting, so every millisecond on it is felt.
Consider a coding agent that a developer leaves open across an afternoon. It runs for ten seconds when a prompt lands, then waits twenty minutes for the next one. Multiply that by every developer on a team, and you have thousands of sessions that are alive on paper and asleep in practice.
The hyperscalers have already moved in this direction. The session-aware, isolated runtime for agents has now become the fourth compute offering in addition to virtual machines, containers, and serverless.
Sessions that sleep for hours
Agent sessions are bursty in a way web services never are. Holding a full Pod for each idle session wastes the memory and CPU that the Pod reserves, which is why the emerging runtimes snapshot idle sessions out of compute entirely.
Code the platform did not write
Because a model writes the code an agent executes, the runtime cannot assume the workload is well-behaved. It must be able to run a process capable of performing any action, which moves isolation responsibilities from the container boundary to the kernel boundary.
State that has to survive a nap
An agent that loses its context each time it suspends becomes unusable. So, the runtime must save its volatile RAM and filesystem state during hibernation and restore them upon resuming.
Why the Kubernetes control plane sits in the wrong place
Kubernetes schedules work through a central API server and a scheduler designed for a modest number of long-lived Pods. That design assumes placement decisions are rare and durable. Agents violate the assumption by generating a constant stream of fine-grained scheduling events, making the control plane the bottleneck rather than the referee.
The scheduling policies are the first to be strained. Researchers studying agent scheduling have documented that the round-robin and random placement strategies common in Kubernetes clusters work well when requests are short and arrival rates are high, because a bad decision is amortized quickly. Agent requests run longer and arrive less often, so a poor routing choice lingers, amplifying tail latency for the user stuck behind it.
The second pressure point is the API server itself. Storing every agent, active or idle, as a Kubernetes object would mean millions of resources in a system never sized for that many. Agent Substrate’s own architecture notes are blunt about this, acknowledging that there is no clever way to make the standard control plane hold that many objects, so the runtime keeps most agents out of it. Routing takes a similar detour, with a dedicated networking layer that sends each request straight to the correct session and wakes it if it is asleep.
Kubernetes is a fine data center scheduler, and it stays useful for provisioning the machines underneath. It is the wrong scheduler for a workload that looks like a swarm of sleeping processes.
Agent Sandbox, a secure box for untrusted code
Agent Sandbox is the layer that answers the isolation problem. It is an open-source execution environment built on Kubernetes that gives each agent a hardened place to run model-generated code, and Google moved it to general availability after roughly 16x growth in GKE sandboxes in under five months.
The mental model is a jail rather than a container. A normal container shares the host kernel and trusts the workload to stay in its lane, whereas a sandbox assumes the workload is hostile and puts a real boundary around it. Agent Sandbox reaches that boundary through gVisor by default, adds a default-deny network policy, and exposes a pluggable interface so teams can swap in Kata Containers for full kernel isolation.
Customers such as LangChain and Lovable are already running millions of agents on it, which is what forced the performance work. The result is a runtime that treats security and speed as the same problem rather than opposing ones.
Warm pools for the cold-start problem
Spinning up a fresh sandbox per request would add seconds of latency, so Agent Sandbox keeps a warm pool of pre-provisioned replicas. Google reports the API can allocate 300 sandboxes per second per cluster, with 90 percent of allocations finishing in 200 milliseconds.
Pod snapshots for idle sessions
Idle agents are suspended via Pod snapshots and resumed on demand in seconds, freeing the underlying compute rather than paying to keep a sleeping session resident.
Kernel isolation as the default
The isolation is not an add-on for the paranoid. gVisor and network lockdown ship as the baseline, on the assumption that any agent might run something it should not.
Agent Substrate, a runtime for millions of mostly-idle agents
If Agent Sandbox is the secure box, Agent Substrate is the runtime that decides which agent runs where. It reuses the secure runtime and snapshotting from Agent Sandbox and pairs them with a small, focused control plane that sits alongside a Kubernetes cluster, taking the standard control plane off the critical path.
The trick is virtual memory overcommit applied to compute. An OS lets programs address far more memory than the machine physically holds by paging cold pages to disk. Agent Substrate does the same with sessions, multiplexing a large registry of stateful actors onto a much smaller pool of pre-warmed worker Pods and snapshotting the idle ones out to storage. The project reports 30x or more oversubscription with sub-second activation, because the worker Pods are already running when an event arrives and never wait on the Kubernetes scheduler.
The developer-facing shape consists of two custom resources, a WorkerPool that defines the ready compute, and an ActorTemplate that defines the agents. Substrate is framework-agnostic, running ADK, LangChain, Claude Code, or any OCI container as an actor, which is what lets it host full agent harnesses rather than single agents.
A control plane beside Kubernetes, not inside it
Substrate does not replace Kubernetes. It uses Pods and autoscaling for provisioning and layers its own scheduler on top for the agent-specific decisions Kubernetes handles poorly.
The path to production through kagent
Solo.io has already wired Substrate into kagent, its Kubernetes-native agent platform, exposing Agent Substrate as a selectable runtime so an OpenClaw-style harness can be scheduled as an actor onto a worker pool from a single UI.
Choosing where your agents run
The two projects are not competitors, and neither replaces the cluster underneath. The decision is about which layer owns which job, and most real deployments will use all three together.
Requirement
Recommended layer
Rationale
Running untrusted model-generated code safely
Agent Sandbox
Kernel isolation via gVisor, though it adds overhead
Millions of idle sessions on limited hardware
Agent Substrate
Actor multiplexing trades some cold-path latency for density
Provisioning machines and long-lived services
Kubernetes
Proven at scale, but not tuned for bursty agents
A production agent platform with a UI
kagent on Substrate
Packages the runtime, still early and evolving
In practice, most teams will not pick just one of these. A team running coding agents at scale is likely to sandbox the code, schedule it through Substrate, and let Kubernetes provision the nodes beneath both.
What this means to the Cloud Native ecosystem
Anyone who has managed a busy cluster will easily recognize this pattern. The agent is a process, the worker pool is a set of CPU cores, snapshotting an idle session is paging memory to disk, and oversubscription is the same bet virtual memory has always made. Kubernetes remains the underlying machine, and the layer that schedules agents is being rebuilt on top of it to better match how agents actually run.
The open question is who owns that layer. Agent Substrate is a first look rather than a finished product; kagent is early, and rival runtimes will arrive as the cost of idle agents becomes a line item that platform teams can no longer ignore. The next thing worth watching is whether this agent control plane consolidates around a single open project, the way container orchestration once settled on Kubernetes. The alternative is that every hyperscaler ships its own, and the fragmentation that agents were meant to escape returns one layer up.
OpenAI announced ChatGPT Work on July 9 and began rolling it out to Pro, Enterprise, and Edu users. It runs on the new GPT-5.6, opens a user’s local files, edits Google Workspace and Microsoft 365 documents, and carries a multi-step task through to a finished deliverable.
Reuters placed it directly against Anthropic’s Claude Cowork, and both target the same person — a non-coder who wants the power of a coding agent without the terminal. Counting Anthropic, Microsoft, Perplexity, and Amazon, five leading labs have now released an agent of this kind. The batch shows that the newest agents are organized more by their intended users than by their functions.
Based on the target user persona, four archetypes appear: the knowledge worker, the power user who self-hosts, the developer, and the enterprise. The personas often overlap since one individual can embody all three roles. Additionally, a product like Claude Code caters to both solo developers and platform teams.
Therefore, consider these deployment archetypes categorized by the main buyer, rather than strict separations. The archetype only represents what marketing promotes. Behind the scenes, each lab has almost consistently decided who owns the runtime, persists memory, manages credentials, and enforces policy.
Four archetypes, based on the user persona
Let’s analyze each of the four archetypes individually, as each differs in the level of control available to users.
The first archetype serves the knowledge worker. A vendor operates the runtime and sells the agent as a delegation to someone who lives in documents rather than code.
ChatGPT Work is the newest, alongside Claude Cowork, which now runs cloud sessions on web and mobile while keeping local-file access on the desktop; Microsoft’s Copilot Cowork, a cloud-hosted agent that executes long-running tasks inside the Microsoft 365 trust boundary; Perplexity Computer, which works across local files and Microsoft apps; and Amazon Quick, the successor to Q Business as that product closes to new customers at the end of July. The user grants access and supervises the result. In most cases, the vendor manages the runtime and persisted state, except for Perplexity’s local option.
A second archetype belongs to the power user who self-hosts. The provider controls the persistent agent process and chooses where to store the state and credentials, often on a Mac mini that has become a piece of personal infrastructure in its own right.
OpenClaw and Hermes are the reference examples, open source, and run on the operator’s own machine. Self-hosting involves managing the control plane rather than full local custody, since both options still allow access to a hosted model and the storage of credentials for external services.
Related reads:
“Microsoft has proved it can survive major changes in the tides of technology… Today, it faces another evolution in one of its core cash cows, as late-stage unicorns and AI labs alike push deeper into Office territory.”
The developer gets the third archetype, whose runtime spans the IDE, the terminal, the repository, and a cloud sandbox. Claude Code, OpenAI Codex, GitHub Copilot in agent mode, and the open-source OpenCode all live here, and Amazon’s developer agent is folding into its Kiro tool. Coding-agent execution is extending from the local IDE to vendor-managed sandboxes and asynchronous cloud workers, making this the most challenging archetype to categorize clearly.
The fourth archetype is built for enterprise workflows and integration with business processes. They run an open agent framework, such as LangGraph or CrewAI, on a managed, governed runtime. ADK on the Gemini Enterprise Agent Platform, Strands on the Bedrock AgentCore, Microsoft Agent Framework on Foundry Agent Service, and Claude Managed Agents belong to this category. OpenAI’s Agents SDK can be hosted on some of these runtimes, including AgentCore, which AWS lists as one of its supported frameworks. The vendor operates the infrastructure, and the customer configures identity, policy, and retention on top of it.
Customer defines identity, policy, and retention; platform brokers
Programmable platform
The line below the personas
The persona-based approach abstracts four things: where execution runs, where state is persisted, how authority is delegated, and where policy is enforced.
Anthropic describes its design as decoupling the brain from the hands. The harness that calls Claude runs separately from the sandbox where code executes, and a session, an append-only log of every model call, tool call, and result, connects the two. Because the sandbox is kept separate from the brain, the agent can start reasoning before any container exists, and the code it runs remains far from the developer’s credentials.
The same four planes show up at the other vendors. AgentCore Runtime gives each session a dedicated microVM with an isolated CPU, memory, and filesystem, and meters compute usage. Google can route governed traffic through its Agent Gateway, where Model Armor policies inspect configured ingress and egress flows, while Agent Identity and an Agent Registry track the fleet. Microsoft assigns each hosted agent a dedicated Entra Agent ID and runs it in a per-session sandbox whose filesystem survives idle periods.
Deploying agents on a managed runtime is more like leasing a workshop than purchasing a tool… a long-term tenant installs their own locks, maintains their records, and takes their tools when the lease ends.
Deploying agents on a managed runtime is more like leasing a workshop than purchasing a tool. The landlord manages the building and supplies the power, but a long-term tenant installs their own locks, maintains their records, and takes their tools when the lease ends. The personas often conceal this difference. Currently, the vendor typically operates the runtime, so the key question is how much control the customer can still exert and what they can take away across the four planes.
Where the line falls
What differentiates each offering is not the compute operator, since vendors handle nearly all of it. It is how much of those four planes a product leaves the customer to configure and export. A packaged experience hands nearly all four to the vendor and returns supervision and a finished outcome. A programmable platform operates the infrastructure but lets the customer define identity, policy, and retention and move the code elsewhere.
Copilot Cowork shows that the two axes are separate. It is a packaged knowledge-worker experience, yet it runs on a governed enterprise platform and inherits Microsoft’s identity, compliance, and audit controls.
The persona sells the product, but the four planes decide the lock-in.
A product can be packaged on the surface and programmable underneath, which is why the personas and the control planes have to be read as different questions. ChatGPT Work makes the same point from the developer side, since OpenAI’s new desktop app folds Chat, Work, and Codex into a single surface, though OpenAI has not detailed how far the runtime or credential store are shared beneath it. The persona sells the product, but the four planes decide the lock-in.
Usecase
Agent Type
Tradeoff
Delegate a knowledge task to an agent you supervise
A knowledge-worker agent such as ChatGPT Work, Copilot Cowork, or Amazon Quick
The vendor typically operates the runtime and persists state, and you configure little below the surface beyond access and approval
Keep state and credentials on hardware you control
A self-hosted agent such as OpenClaw or Hermes, in a local configuration
You control the persistent process, though model inference and some tools may still be remote
Ship code changes across the IDE, repo, and CI
A developer coding agent such as Claude Code, Codex, or Copilot
Execution spans your tools and a cloud sandbox, so ownership is split and worth mapping before you commit
Run many governed agents with audit and identity
An enterprise runtime platform such as AgentCore, Agent Platform, Foundry, or Managed Agents
The vendor operates the infrastructure while you define identity, policy, and retention, in exchange for coupling workflow logic to one cloud
Real deployments combine the rows rather than picking one. Teams on Foundry Agent Service commonly run open-source orchestration, such as LangGraph, for agent logic while leaning on the platform for governed execution, and Microsoft’s own hosted runtime now supports long-running personal agents like OpenClaw and Hermes with durable state. The boundary between experience and platform is not a thick, well-defined boundary, but a thin line a single system can cross, bridging rival camps.
The agent market is not being decided by open against closed. Open frameworks like Strands, ADK, and Microsoft Agent Framework are precisely what the governed runtimes are built to host. Vendors now manage the runtime across nearly all archetypes, shifting the competition to who can most effectively configure and export state, identity, and underlying policy.
If coding agents enter managed sandboxes alongside knowledge-worker agents, the developer archetype will be established on the platform side. The map will then transform into what the vendors are already outlining. When enterprise teams assess agents, their most important question should be how much of execution, state, identity, and policy they can configure and extract from the product. They should not focus on which persona it presents or whether its framework is open, as the framework no longer determines the product’s value or locking-in capabilities.
In April, Andrej Karpathy published a GitHub gist file called “LLM Wiki,” a brief text document designed to help one build a personal knowledge base using LLMs. It’s based on the premise that an AI agent will keep what it knows as linked Markdown files it can read and rewrite, because a language model does not get bored maintaining cross-references and can touch fifteen files in a single pass. It was only a few thousand words with no product attached.
Two months later, Google turned that instinct into a published standard called the Open Knowledge Format. The OKF packages organizational knowledge, metrics, tables, and runbooks as plain Markdown that any agent can read without a proprietary account. Google is careful to call it v0.1 — a starting point rather than a finished standard.
Garry Tan, the Y Combinator president, got there first in a different lane. His gstack, an MIT-licensed Claude Code setup that crossed 66,000 GitHub stars within weeks, comprises 23 specialist roles, each a Markdown file. No runtime; no code; just prose that runs across ten different coding agents.
Markdown has become the substrate agents read and write
Three approaches, three different needs, one common solution. Karpathy sought agent memory, Google aimed for enterprise context in BigQuery agents, and Tan wanted a way to summon an engineering team from a terminal. All three turned to the same basic resource: a folder of Markdown files versioned in git.
Developers had already established this practice. CLAUDE.md and AGENTS.md are present in millions of repositories as the initial files an agent loads. OKF and gstack are the evolved forms of this convention – one focused on what the agent knows, the other on how it behaves.
This is the Git and JSON playbook tied to the agent’s knowledge. The formats that survived are the ones you could start using without changing anything. You can simply cat the file, clone the repo, and any tool you already use can parse it. MCP remains important as the interface an agent connects to. Markdown is becoming the format that carries the content.
The lock-in moved from the model to the files
The significant factor to observe here is the competitive advantage, not technical specifics. For two years, the belief was that owning the best model meant controlling the developer.
This perspective is now shifting. Replacing Claude with GLM or Codex, gstack continues to operate because the core intelligence evolved, but the documentation did not.
The moat is shifting from the model to the Markdown a team owns and accumulates over time.
The moat is shifting from the model to the Markdown a team owns and accumulates over time. A company’s OKF bundle, including its runbooks, metric definitions, and architecture decisions, is, by design, portable across clouds, models, and frameworks.
That kind of portability is the reason vendor-neutral formats exist and why Google’s OKF deserves a closer look.
If no one develops consumers for it, it remains just a good idea that Google released on a slow Friday.
The area where I am most likely mistaken is durability. Declaring Markdown standards is easy, but making them reliable is difficult. OKF is merely a 0.1 draft with a reference implementation, not a full ecosystem. If no one develops consumers for it, it remains just a good idea that Google released on a slow Friday.
The direction remains determined by three separate bets targeting the same file format within a single quarter. Your next agent is likely to interpret its context from a Markdown folder, and the creator of that folder now possesses an advantage that the model vendor cannot easily replicate.
Over the past few months, four AI giants quietly rebuilt the same thing at once: AWS, Microsoft, Google, and Anthropic each shipped agent runtime updates that point to the same architectural shift.
Microsoft rebuilt its Foundry-hosted agents in April around per-session isolation. Anthropic’s Managed Agents virtualize the agent into a session, a harness, and a sandbox. AWS routes each session to its own microVM, and Google isolates agent code execution in a dedicated sandbox. The common pattern is not another model feature or developer tool. It is a move from request-level load balancing to session-aware execution.
This matters because enterprise agents are not ordinary API calls. They are long-running, stateful, tool-using processes that often run code influenced by user input. At its core, the agent runtime is becoming a control plane for state, identity, isolation, and lifecycle, and the unit it schedules has moved from the individual request to the agent session. The convergence across four platforms suggests session-aware execution is becoming a baseline requirement for production-grade agents.
The traditional cloud scaling model
To appreciate why this matters, it helps to start with what the traditional model got right. NGINX and HAProxy tiers typically sit in front of a pool of workers and route each incoming request to the next available backend. The state is deliberately kept elsewhere, in Redis or a database, so any worker can serve any request. That externalization of state is what enables elastic scaling, fault tolerance, and infrastructure replacement without disrupting the application. Sticky sessions were available, but enterprises treated them as an exception for stateful applications rather than the default.
The model rests on two assumptions: that requests do not depend on one another and that any backend can serve any request. For web APIs and microservices, those assumptions held for nearly two decades and enabled the scaling of the largest systems ever built. Agents challenge both at once, which is why conventional load balancing alone is no longer sufficient.
Why agents break the model
Agents pose two distinct challenges: preserving conversational state across turns and enforcing a security boundary strong enough to withstand untrusted code. Only the first can be addressed by routing alone.
Why can the agent state not be pooled?
Imagine an enterprise support agent processing a refund. It reads the order, calls a tool, and then waits for the model. The next turn asks a clarifying question. If that turn is routed to a different replica without access to the prior context, the agent loses the state it needs to finish the workflow.
The load balancer starts to behave less like a traffic distributor and more like a session router.
The industry already has a name for the workaround, session affinity, and it already concedes the cost. Load-balancing approaches increasingly rely on consistent hashing, using a conversation identifier to keep related steps in a warm state. In plain English, the load balancer starts to behave less like a traffic distributor and more like a session router.
Why the trust boundary forces isolation
The second challenge is harder because it turns on isolation rather than on the state alone. An agent can execute model-generated code shaped by user input, so the backend must be treated as a security boundary rather than a generic compute target. A shared kernel does not give untrusted, agent-generated code the tenant isolation that enterprise security teams require. Session affinity can preserve routing continuity, but it does not by itself provide per-tenant isolation for untrusted execution. For platform and security teams, that is the line where a tuned load balancer is no longer enough.
This is not a theoretical concern. When Asana disclosed a flaw in its MCP server in June 2025, the server had been live since May 1, a window of about five weeks. A tenant-isolation check failed, and around 1,000 organizations could see project data belonging to other customers. Based on the reported details, security researchers traced the issue to a server that validated the user but did not consistently enforce the agent and tenant context behind cached responses. No external attacker was involved, and data still crossed organizational boundaries. The incident shows why user identity, agent context, and session state must be consistently bound together.
The session as a unit of compute
Once that binding becomes a correctness and security requirement, the session becomes the unit that the platform schedules. The clearest evidence is in the lifecycle. A session now has a state it never had before. AWS documents that an AgentCore session remains Active while it processes work and sits Idle when it is provisioned but waiting. It reaches Terminated after a 15-minute idle timeout or an 8-hour maximum lifetime. That is the lifecycle of a long-running execution environment, not of a short-lived HTTP request.
The economics follow the same logic. When billing is tied to active sessions, concurrency, idle time, and agent sizing become the cost drivers rather than request volume. That puts a new line item in front of platform teams and finance. The better mental model is no longer a traditional load balancer. It is closer to a virtual actor runtime, where an addressable identity is instantiated on demand, kept active while it is needed, and deactivated when idle, with one live instance per key.
Four approaches to the same shift
All four platforms are moving away from treating stateful, untrusted agent work as ordinary load-balanced traffic. The key difference is the compute primitive each provider chooses for the isolated execution environment.
AWS AgentCore is the most opinionated of the four. Each session gets a dedicated Firecracker microVM with isolated compute, memory, and filesystem, and a request carrying the same runtimeSessionId is routed back to that microVM through a session header. When the session ends, the microVM is terminated and its memory sanitized.
The agents hosted on Microsoft Foundry implement a similar model through a different set of primitives. The platform creates a per-session VM-isolated sandbox on demand, runs it, and tears it down at session end, with no replica count and no warm pool to size. Each agent gets a dedicated Microsoft Entra identity, and sessions run a 15-minute idle timeout against a 30-day maximum lifetime.
Google Agent Engine is the most instructive hybrid. Its runtime keeps request scaling within the reasoning loop, a managed environment with configurable minimum and maximum instance counts, and a container_concurrency that defaults to 9. Even where Google retains request scaling for the loop, it separates untrusted code execution into an isolated Code Execution sandbox and externalizes conversation state to Sessions and Memory Bank. The provider that kept the load balancer in the loop still would not point it at stateful untrusted work.
Anthropic provides the clearest decomposition of the architecture. Managed Agents virtualize the agent into three components: a session that logs everything that happens, a harness that runs the loop and routes tool calls, and a sandbox where code runs. The harness becomes a near-stateless control plane, while the sandbox becomes a callable, rebuildable resource. The Cloudflare integration shows that the substrate can be decoupled. The agent loop runs on Anthropic while each tool call runs in a Cloudflare sandbox, which can be a full microVM or a lighter V8 isolate.
The four platforms converge on the routing and lifecycle model and diverge on the execution substrate, as the table shows.
Platform
Compute primitive per session
What it means for an enterprise buyer
AWS AgentCore
Dedicated Firecracker microVM, routed by session ID, 8-hour ceiling
The most opinionated answer, with isolation and routing fused at the microVM
Azure Foundry hosted agents
Per-session VM-isolated sandbox, dedicated Entra identity, 30-day max
Identity-led isolation suited to longer multi-day workloads
Google Agent Engine
Request-scaled loop plus a separate isolated code-execution sandbox
A hybrid that load-balances the reasoning and isolates the execution
Anthropic Managed Agents
Decoupled harness and sandbox, microVM or V8 isolate behind the loop
The most portable, with the substrate treated as a swappable layer
No single architecture is the right answer for every enterprise workload. A multi-day research agent fits Azure’s longer-lived session, a code-heavy agent needs microVM-grade isolation, and high-volume automation favors a lighter isolate. Many enterprise platforms will end up combining more than one of these patterns.
The binding the application still owns
The platform can isolate and route a session, but it does not know which human owns it. AgentCore explicitly states that it does not enforce session-to-user mappings, so the application backend must maintain the relationship between users and their session IDs and set per-user session limits. This is why the Asana incident is relevant to enterprise architecture.
Isolation was maintained while the user-to-session binding broke.
In that case, isolation was maintained while the user-to-session binding broke. The platform solves isolation and lifecycle, and it hands back identity mapping, authorization, and tenant context to the application. For an enterprise buyer, the key question is who owns that binding and how it is tested under concurrent, multi-tenant load.
How this differs from sticky sessions
A reasonable objection is that this appears to be sticky sessions combined with autoscaling on a microVM substrate. The distinction is worth drawing out. Traditional sticky sessions are usually performance optimizations, whereas in an agent runtime, session binding becomes a correctness and security requirement. Traditional load balancers route traffic to existing backends and do not own the lifecycle of the execution environment.
The new control plane provisions an environment when it first sees a session key, routes work to it, and tears it down after idle or lifetime limits. The primitive underneath is genuinely old, and its age is what makes the move legible. AWS open-sourced Firecracker at re:Invent in 2018 to pair VM-grade isolation with container speed. I covered it for The New Stack then, and ran more than a hundred microVMs on a single laptop to see that trade firsthand. The same microVM now powers Lambda and Fargate across trillions of executions a month. This is how cloud architecture tends to evolve, with existing primitives recombined around a new unit of abstraction.
The agent session is becoming the operational unit of enterprise AI infrastructure.
In summary, the important shift is not that cloud providers have found a better load balancer for agents. It is that the agent session is becoming the operational unit of enterprise AI infrastructure. Traditional load balancing routed independent requests to interchangeable workers. A session-aware runtime routes work to an isolated environment, preserves state across turns, and owns the lifecycle of the execution context. The dispatch layer did not disappear when agents arrived. It became session-aware, stateful, and lifecycle-aware, making it a control plane rather than a conventional load balancer.
Three practical questions will shape what comes next, and enterprises will need to evaluate each. The economics of per-session billing will decide how costs scale with concurrency. The open-source effort to rebuild this routing model on Kubernetes will determine its portability. The argument between microVMs and lighter isolates will decide the trade-off between isolation and density. For enterprises building agents, a runtime that owns isolation, routing, and lifecycle turns background plumbing into a deliberate architectural choice. It gives platform, security, and operations teams a foundation they can standardize on rather than rebuild for every workload.
OpenClaw made the always-on agent feel personal by making it live somewhere you could point at — a Mac mini on a shelf, drawing seven watts, running while you sleep. Peter Steinberger’s open-source project surpassed 300,000 GitHub stars by April and became one of the fastest-growing repositories on GitHub. The whole appeal was ownership: your hardware, your credentials, your lobster.
On Tuesday at I/O, Google launched Gemini Spark, and it makes the opposite bet. Spark is a 24/7 personal agent built on Gemini 3.5 Flash and connected to Google’s Antigravity agent stack. It runs in the background on virtual machines on Google Cloud. You never see the machine. Google plans to let you text and email the agent directly, so it works while your laptop is shut.
The split is about where the agent lives, not what it does
Strip away the branding, and Spark and OpenClaw do roughly the same job. Watch an inbox, draft the status update, browse the web, run the recurring task. Both are converging on MCP for tool connectivity, though the implementations differ in maturity. Both promise the assistant who does things rather than answers questions.
The substrate decides who holds your context, who sees your credentials, and who can change the terms later.
The difference is the substrate. OpenClaw runs on the metal you bought. Spark runs on metal Google rents to you and never names. That sounds like a deployment detail. It is actually the whole argument. The substrate decides who holds your context, who sees your credentials, and who can change the terms later.
Convenience usually wins this fight, and Google knows it
The self-hosted version asks for real work. Buy the Mac mini, keep it awake, install a daemon, set up Tailscale, and rotate the key when it expires. The reward is control. Your credentials and workflows can stay under your own hand, depending on how you wire up models and integrations. That control is not the same as safety. A misconfigured local agent with shell, browser, and inbox access is its own hazard, and Chinese regulators have already flagged exactly that risk with OpenClaw.
Spark asks for nothing. It is already inside Gmail, Docs, and Sheets, with no manual wiring, because Google owns both ends. That out-of-the-box reach is the structural advantage no third-party agent can copy. The history here is fairly settled. Dropbox beat the home NAS. Gmail beat the mail server. Managed nearly always beats self-hosted for the median user, because most people will trade control for not having to think about it.
OpenClaw is not losing. It is being sorted into the smaller, stickier half.
So the personal-agent layer splits in two. A hosted tier where Google, and soon OpenAI, own the runtime and the context. A self-hosted tier for developers who want the credentials on their own hardware and will pay for the setup time. OpenClaw is not losing. It is being sorted into the smaller, stickier half.
The privacy bargain here is not the one Dropbox asked for
This is where I would slow down before calling the race. Cloud storage won because the thing you handed over was inert. Files sat in a Dropbox folder, and nobody read them. A personal agent is of a different kind. To be useful, Spark needs broad standing access to your Gmail, Docs, Sheets, calendar, and live inbox. It does not just store your context. It reads it to act on it.
That changes the deal. Handing Google a folder of files is not the same as handing Google a system that processes your job, your relationships, and your calendar well enough to send mail on your behalf. The honest version of the worry is not that Google keeps your data. It is the unsettled gap between access, retention, and what trains the next model.
The self-hosted camp is small today, but that’s not because of nostalgia for running your own server. It is the instinct that an agent this intimate should answer to you, on hardware you can unplug. That instinct does not scale to everyone. It does not have to. It only has to hold the developers and the privacy-sensitive, and that is a durable floor.
The question for developers is not which agent is better. It is whether you are comfortable with Google holding the keys to the one that runs your life.
Login
