As enterprise leaders start deploying agentic workflows, they must establish the infrastructure to build and run them, one capable of fluidly routing a diverse set of workloads across the most efficient compute resources.

This requires the ability to manage heterogeneous infrastructure, utilizing high-performance accelerators for large-scale training and inference, and utilizing CPUs for the critical orchestration layer of agentic AI. As autonomous agents become more prevalent, CPUs are ideally suited for managing agent state, semantic routing, tool selection, and spinning up secure, isolated sandboxes to safely execute untrusted generated code.

The Google Axion advantage

Google Cloud, with its workload-optimized Compute Engine portfolio, which includes general-purpose and specialized offerings, shines in addressing this need.

Google Axion processors within this portfolio comprise a family of custom Arm processors engineered for performance, efficiency, and versatility, with a feature set that supports general-purpose workloads, CPU-based AI workloads, and other specialized tasks requiring Arm-native compatibility and direct hardware access.

Axion is Google’s first custom Arm-based server CPU, introduced in April 2024. It is designed specifically for hyperscale cloud and AI-era data center workloads. 

Axion also leverages more than a decade of Google’s custom silicon innovation. This enables Google to more readily incorporate customer feedback into chip designs and address the more general, though complex, needs of CPUs. 

Matching workload type to the processor

Bhumik Patel, Director of Software Ecosystem Development at Arm, says the key to all of this is to match the workload type as closely as possible to computing capacity. CPU-powered cloud instances are a practical option for certain AI workloads, particularly those with smaller datasets or less complex models. 

“Agentic tasks such as orchestrating, talking to APIs, and memory management are all ones CPUs are good at, so it’s a distributed and concurrent AI workload,” Patel tells The New Stack. Intelligent workload-processing apportionment makes agentic AI more cost-effective and efficient than running all workloads on a single compute type.

This efficiency is quantifiable. The Google Kubernetes Engine Agent Sandbox running on Google Axion N4A provides up to 30% better price performance than the next hyperscale cloud provider, says Google’s Mo Farhat, Axion Group Product Manager. The GKE Sandbox is an open-source Kubernetes-native primitive designed to execute untrusted AI-generated code safely. 

“Agentic tasks such as orchestrating, talking to APIs, and memory management are all ones CPUs are good at, so it’s a distributed and concurrent AI workload.”

Intelligent workload decoupling makes agentic AI significantly more cost-effective. Google Cloud’s fluid computing foundation enables engineering teams to reserve specialized accelerators strictly for heavy reasoning and generative workloads, while leveraging Axion CPUs for high-concurrency orchestration and context management.

Secure execution with the GKE Agent Sandbox

As agents begin to generate and execute dynamic code autonomously, security is non-negotiable. Running AI-generated code directly in a standard cluster poses severe security risks, as untrusted code could potentially access other apps or the underlying cluster node.

The Google Kubernetes Engine (GKE) Agent Sandbox resolves this by providing an isolated environment for safely executing untrusted code. Running on Axion-powered N4A instances, the sandbox provides up to 30% better price performance than comparable workloads on other hyperscalers.

The vertical stack isolates sensitive tasks at the kernel level with sub-second latency.

The vertical stack isolates sensitive tasks at the kernel level with sub-second latency.  GKE Agent Sandbox natively supports gVisor (an open-source application kernel developed by Google that acts as a secure sandbox for containers) and default-deny Kubernetes network policy. Agent Sandbox provides pluggable interfaces for open-source sandboxes, such as Kata Containers, enabling users to customize their kernel isolation. 

Powered by gVisor technologies with software support from Arm’s architecture, the sandboxes intercept and validate system calls before they reach the host kernel. These isolated execution environments enable deployment of autonomous systems at scale without sacrificing performance or operational agility.

To manage resources efficiently when agents sit idle, GKE Pod snapshots allow users to save and restore the exact process state of sandboxed environments. This functionality provides four major architectural benefits:

• Fast startup: Reduces sandbox startup time by restoring from a pre-warmed snapshot rather than initializing from scratch.
• Long-running agents: Pauses sandboxes that take a long time to run and resumes them later—or moves them across nodes—without losing progress.
• Stateful workloads: Persist an agent’s context, such as conversation history or intermediate calculations.
• Reproducibility: Captures a specific state to use as a baseline for spinning up multiple new sandboxes.

Getting started

As token generation, autonomous workflows, and continuous agent interactions grow exponentially, relying exclusively on accelerator-backed stacks for every task will become financially and architecturally unsustainable.

The combination of CPU and accelerator execution accounts for bursts in agent activity and unpredictable demand spikes by eliminating the inference tax. Google Cloud’s full-stack advantage enables organizations to deploy the right machine for the job. 

By using Google Axion and GKE Agent Sandbox, builders can optimize total cost of ownership and security while maintaining the performance required for AI agents.

Learn more about Google Axion.

The post Arm and Google offer a smarter option to run agentic AI workloads appeared first on The New Stack.

Google made GKE Agent Sandbox generally available in May 2026 and, in the same post, introduced a second project called Agent Substrate. These two announcements concede a point that Kubernetes veterans have been reluctant to call out.

That indirect admission is that the platform that won the container decade is not the right control plane for AI agents. Agent Sandbox provides agents with a secure environment to run untrusted code. Agent Substrate adds a scheduling layer that routes around the Kubernetes control plane because the API server was never designed for how agents behave.

Compare agents to processes in an operating system rather than services in a data center, and the mismatch is obvious. A modern OS runs thousands of processes that spend most of their life asleep. It wakes each one on an event, hands it a slice of a CPU, then pages its idle memory out to disk to make room for the next. Agents behave almost exactly like those processes. Kubernetes was originally created to manage a fixed set of long-running, replicated services. This fundamental design explains why much of the agent infrastructure now runs on Kubernetes rather than being integrated into it as one of the workloads, such as a Deployment or a StatefulSet.

What an agent actually is as a workload

An agent is a long-running, stateful session that stays idle for most of its life, wakes to execute a burst of code, then goes quiet again. The code it runs is generated by a model at runtime. The host has to treat it as untrusted by default. Each session needs a stable identity, the ability to pause and resume without losing memory, and hard isolation from its neighbors.

Think of the agent as a process in a time-sharing OS. Just as the scheduler suspends a sleeping process and restores it the moment a keystroke arrives, an agent runtime must hibernate an idle session and restore it with its working memory intact. The wake path is where the user is waiting, so every millisecond on it is felt.

Consider a coding agent that a developer leaves open across an afternoon. It runs for ten seconds when a prompt lands, then waits twenty minutes for the next one. Multiply that by every developer on a team, and you have thousands of sessions that are alive on paper and asleep in practice.

The hyperscalers have already moved in this direction. The session-aware, isolated runtime for agents has now become the fourth compute offering in addition to virtual machines, containers, and serverless.

Sessions that sleep for hours

Agent sessions are bursty in a way web services never are. Holding a full Pod for each idle session wastes the memory and CPU that the Pod reserves, which is why the emerging runtimes snapshot idle sessions out of compute entirely.

Code the platform did not write

Because a model writes the code an agent executes, the runtime cannot assume the workload is well-behaved. It must be able to run a process capable of performing any action, which moves isolation responsibilities from the container boundary to the kernel boundary.

State that has to survive a nap

An agent that loses its context each time it suspends becomes unusable. So, the runtime must save its volatile RAM and filesystem state during hibernation and restore them upon resuming.

Why the Kubernetes control plane sits in the wrong place

Kubernetes schedules work through a central API server and a scheduler designed for a modest number of long-lived Pods. That design assumes placement decisions are rare and durable. Agents violate the assumption by generating a constant stream of fine-grained scheduling events, making the control plane the bottleneck rather than the referee.

The scheduling policies are the first to be strained. Researchers studying agent scheduling have documented that the round-robin and random placement strategies common in Kubernetes clusters work well when requests are short and arrival rates are high, because a bad decision is amortized quickly. Agent requests run longer and arrive less often, so a poor routing choice lingers, amplifying tail latency for the user stuck behind it.

The second pressure point is the API server itself. Storing every agent, active or idle, as a Kubernetes object would mean millions of resources in a system never sized for that many. Agent Substrate’s own architecture notes are blunt about this, acknowledging that there is no clever way to make the standard control plane hold that many objects, so the runtime keeps most agents out of it. Routing takes a similar detour, with a dedicated networking layer that sends each request straight to the correct session and wakes it if it is asleep.

Kubernetes is a fine data center scheduler, and it stays useful for provisioning the machines underneath. It is the wrong scheduler for a workload that looks like a swarm of sleeping processes.

Agent Sandbox, a secure box for untrusted code

Agent Sandbox is the layer that answers the isolation problem. It is an open-source execution environment built on Kubernetes that gives each agent a hardened place to run model-generated code, and Google moved it to general availability after roughly 16x growth in GKE sandboxes in under five months.

The mental model is a jail rather than a container. A normal container shares the host kernel and trusts the workload to stay in its lane, whereas a sandbox assumes the workload is hostile and puts a real boundary around it. Agent Sandbox reaches that boundary through gVisor by default, adds a default-deny network policy, and exposes a pluggable interface so teams can swap in Kata Containers for full kernel isolation.

Customers such as LangChain and Lovable are already running millions of agents on it, which is what forced the performance work. The result is a runtime that treats security and speed as the same problem rather than opposing ones.

Warm pools for the cold-start problem

Spinning up a fresh sandbox per request would add seconds of latency, so Agent Sandbox keeps a warm pool of pre-provisioned replicas. Google reports the API can allocate 300 sandboxes per second per cluster, with 90 percent of allocations finishing in 200 milliseconds.

Pod snapshots for idle sessions

Idle agents are suspended via Pod snapshots and resumed on demand in seconds, freeing the underlying compute rather than paying to keep a sleeping session resident.

Kernel isolation as the default

The isolation is not an add-on for the paranoid. gVisor and network lockdown ship as the baseline, on the assumption that any agent might run something it should not.

Agent Substrate, a runtime for millions of mostly-idle agents

If Agent Sandbox is the secure box, Agent Substrate is the runtime that decides which agent runs where. It reuses the secure runtime and snapshotting from Agent Sandbox and pairs them with a small, focused control plane that sits alongside a Kubernetes cluster, taking the standard control plane off the critical path.

The trick is virtual memory overcommit applied to compute. An OS lets programs address far more memory than the machine physically holds by paging cold pages to disk. Agent Substrate does the same with sessions, multiplexing a large registry of stateful actors onto a much smaller pool of pre-warmed worker Pods and snapshotting the idle ones out to storage. The project reports 30x or more oversubscription with sub-second activation, because the worker Pods are already running when an event arrives and never wait on the Kubernetes scheduler.

The developer-facing shape consists of two custom resources, a WorkerPool that defines the ready compute, and an ActorTemplate that defines the agents. Substrate is framework-agnostic, running ADK, LangChain, Claude Code, or any OCI container as an actor, which is what lets it host full agent harnesses rather than single agents.

A control plane beside Kubernetes, not inside it

Substrate does not replace Kubernetes. It uses Pods and autoscaling for provisioning and layers its own scheduler on top for the agent-specific decisions Kubernetes handles poorly.

The path to production through kagent

Solo.io has already wired Substrate into kagent, its Kubernetes-native agent platform, exposing Agent Substrate as a selectable runtime so an OpenClaw-style harness can be scheduled as an actor onto a worker pool from a single UI.

Choosing where your agents run

The two projects are not competitors, and neither replaces the cluster underneath. The decision is about which layer owns which job, and most real deployments will use all three together.

Requirement	Recommended layer	Rationale
Running untrusted model-generated code safely	Agent Sandbox	Kernel isolation via gVisor, though it adds overhead
Millions of idle sessions on limited hardware	Agent Substrate	Actor multiplexing trades some cold-path latency for density
Provisioning machines and long-lived services	Kubernetes	Proven at scale, but not tuned for bursty agents
A production agent platform with a UI	kagent on Substrate	Packages the runtime, still early and evolving

In practice, most teams will not pick just one of these. A team running coding agents at scale is likely to sandbox the code, schedule it through Substrate, and let Kubernetes provision the nodes beneath both.

What this means to the Cloud Native ecosystem

Anyone who has managed a busy cluster will easily recognize this pattern. The agent is a process, the worker pool is a set of CPU cores, snapshotting an idle session is paging memory to disk, and oversubscription is the same bet virtual memory has always made. Kubernetes remains the underlying machine, and the layer that schedules agents is being rebuilt on top of it to better match how agents actually run.

The open question is who owns that layer. Agent Substrate is a first look rather than a finished product; kagent is early, and rival runtimes will arrive as the cost of idle agents becomes a line item that platform teams can no longer ignore. The next thing worth watching is whether this agent control plane consolidates around a single open project, the way container orchestration once settled on Kubernetes. The alternative is that every hyperscaler ships its own, and the fragmentation that agents were meant to escape returns one layer up.

The post Kubernetes won the container decade. Google’s Agent Substrate wants the next one. appeared first on The New Stack.

Microsoft recently took the wraps off Brain, the internal AI system that continuously monitors Azure’s health and, increasingly, acts on what it finds — declaring outages, pausing harmful rollouts, and notifying affected customers.

Azure CTO Mark Russinovich first wrote about the system in a blog post, “Meet Brain: The AI system behind Azure reliability,” the first in a planned multi-part series about the Azure team’s reliability and resiliency tooling.

To dive deeper, The New Stack sat down with Russinovich, who is also Azure’s deputy CISO and a technical fellow, to talk about how the Brain project came to be and how it evolved over time.

Brain, as Microsoft describes it, is Azure’s centralized AIOps system for cloud health. It operates as an intelligent layer on top of Azure Resource Graph (ARG), and together, the company says, the two form a real-time digital twin of Azure’s health.

A real-time digital twin

While Brain today uses many AI tools, the project is actually much older than the generative AI boom, and to get started, the team had to build a solid foundation first. “At the heart of this system is Azure Resource Graph, which started as ‘let’s create a digital twin of Azure, so we can understand the relationship between the different resources in Azure,’” Russinovich tells The New Stack.

That internal digital twin became a public service at the urging of what Russinovich calls whale customers, those “that have huge estates across many different tenants and subscriptions that wanted to do easy queries across the whole thing, like, ‘What Linux VMs do I have, and what versions of Linux are they on?'”

It was actually the root cause analysis on top of that graph that Brain really began with. “Many times you can just trace dependencies and say, well, these services all depend on this other service that has gone unhealthy, and so that I think was the genesis of having Brain go and start to have a lot of ML-driven algorithms to identify root cause on top of the graph,” Russinovich explains.

Around the same time, Microsoft kept encountering a measurement gap: A service team’s own health metrics indicated everything was fine, but customers saw failures. Russinovich says that could happen because the Azure team wasn’t “measuring what customers are experiencing, or because they’re aggregating at scopes that hide customer-specific problems.”

So Microsoft decided to standardize. “We decided, let’s go standardize on the way that we measure health,” Russinovich says, “and we came up with service level indicators, SLIs.”

Getting services across Azure to actually emit them through shared libraries that conformed to the schema was a complex task that took several years.

“It’s kind of a whole bunch of different things that happened in parallel that all have come together,” he says. “There’s just a tremendous amount of data engineering that goes into this, and trying to keep it as automated as possible.”

Three signals feed Brain

In his blog post, Russinovich writes that Azure’s reliability challenge isn’t a lack of tooling but a “comprehension problem,” with a hyperscale cloud now producing more signal than humans can read.

Azure runs hundreds of services across more than 80 regions, 500+ data centers, and 800,000+ kilometers of fiber and subsea cable. And yet, he writes, Microsoft still sometimes learns about a quietly degrading service from a customer before its own systems detect it.

Today, Brain pulls from three classes of signals. The standardized SLIs come first. Service teams also build and register their own domain-specific monitors, which run alongside telemetry-like deployments, support volume, and cross-service dependency signals. Third-party indicators make up the rest.

Brain produces the same four outputs for any subject, whether that’s a service, a region, a deployment unit, or a customer’s resources.

Based on this, Brain produces the same four outputs for any subject, whether that’s a service, a region, a deployment unit, or a customer’s resources. It reports the health state, how severe the issue is, who is impacted, and — crucially — why it reached that conclusion.

Those conclusions then drive alerts and remediations. Brain declares outages based on blast radius, Russinovich notes, and scopes customer notifications to the impacted subscriptions and regions. The system automatically routes incidents to the appropriate service team and sends deployment-gate signals to pause rollouts causing the issues.

Russinovich says the system is “very pluggable in terms of what signals go into it, and includes even things like customer support tickets that have been opened, and social media posts that mention Azure.” He says Brain is “primarily monitoring,” but “it also can take automated repair actions, too. So for some incidents, teams can specify if, when this happens, go try these things, and Brain kicks those off as well.”

He says the SLIs are “emitted at the scale unit level, so that we can do aggregations for overall health. We can pinpoint specific customers that are being impacted, because we know what customers map to which scale units, and that’s the way that the auto notification triggers off that.”

Why ML sets the thresholds

Microsoft’s original plan for turning SLIs into health determinations was the textbook one, asking every service team to define its own SLOs. It didn’t work.

Russinovich says the schema work itself was hard enough, but “even more challenging is coming up with an SLO that is actually a good SLO.” Teams sandbagged their thresholds, he says.

“[Everyone] wants to be very conservative because they don’t want to get paged or have customers told that things are unhealthy when they’re not, so they’re like, ‘You know what, my SLO is 5% of API queries can fail, and then let’s call it unhealthy,’ when actually that’s not a good way to determine health or regressions as rollouts happen, so we decided, ‘Let’s just stop asking them to define their SLOs.'”

“Everyone wants to be very conservative because they don’t want to get paged… so we decided, ‘Let’s just stop asking them to define their SLOs.'”

Instead, ML models now derive the thresholds from each service’s own behavior, per scale unit and per region.

“There’s a baseline for behavior of the service in this region versus that region,” Russinovich says, “and then we can see when there’s regressions.” The resulting SLOs are dynamically adjustable and automated, he explains.

Tying a regression back to the change that caused it is harder still, he says. Brain tracks rollouts of service updates, and “we’ve got ML algorithms too that can identify with confidence this rollout is causing a regression.” But, he says, the rollout that just reached a scale unit isn’t necessarily the culprit.

“A change doesn’t necessarily show up as a regression immediately. It can have latency; it can take hours to show up, or in some cases even days… there can be many, many deployments that have happened over the last day, and you’re like, which one was it?” That’s why, he says, “there’s a lot of ML going into symptom versus change mapping and automated detection.”

Agents that fix outages

The published post keeps its results vague. Detection precision “has improved significantly”; a “substantial majority” of Brain-integrated outages were auto-communicated to customers in the past year; and time-to-notification improved “materially” over manual notifications. In the interview, Russinovich puts numbers on some of it.

He says, “The thing that frustrates customers the most is when they’ve got to call us and tell us there’s an issue, because then they’re like, you guys don’t even know that there’s a problem. I have to tell you there’s a problem. If we can tell them, hey, there’s a problem, we know about it.” Auto-notification, he says, has driven “this reduction of like four to 6x in terms of customer support tickets open, because Brain is automatically notifying them, and they know that we’re on it.”

“The second you put a human in that loop, you can blow right past the 15 minutes.”

“Our time-to-mitigate goal is 15 minutes, from some problem to actually being resolved within 15 minutes,” Russinovich says. “The second you put a human in that loop, you can blow right past the 15 minutes.”

According to Russinovich, the company hits this 15-minute notification window for 80 to 90 percent of the services on Brain. Often, it’s also much shorter and closer to five minutes.

One caveat here is that not everything runs through Brain yet. Microsoft prioritized what it calls its critical services, the foundation the rest of Azure depends on, and Russinovich puts their coverage at “like 70 or 80% of them, and then the tail’s being worked on.”

He notes the rest aren’t flying blind. “It’s not like the services that aren’t on Brain don’t have health systems and alerting and everything. Brain improves things, even for those services.”

For the engineers who do get paged, Brain assembles the picture they used to piece together by hand.

“The incident gets populated initially with an automated collection of information that will say, here’s the graphs of availability on this SLI over these scale units over the last 24 hours, here’s the list of impacted customers, this is the scale unit, here’s the other information supporting this, and so already there you’re saving the engineer huge amounts of time just in going and information gathering and just presenting it right in front of them.”

Agents on top

In the Brain announcement, Russinovich writes that “agents need something to be agentic about.” A triage agent that doesn’t know the dependency graph can’t triage anything, he argues, and the health model is “the prerequisite, not the consequence, of agentic operations at this scale.”

At this point, Microsoft has started running agents on top of Brain. A system called Triangle, which Microsoft Research also described in a 2025 paper, gives each service team an LLM-based agent trained on its historical incidents and troubleshooting guides, with an orchestrator that routes ambiguous incidents among them.

“This Triangle system has agentic representatives for the services, where the Triangle orchestrator then fans it out and says, ‘Here’s the incident; raise your hand if you think it’s yours,’” Russinovich says. Without it, tickets would bounce from team to team — something Microsoft calls handoffs — which increases response times.

“We don’t have to write down every single rule prescriptively… let the agent do things based on its own judgment.”

It’s still early days for Triangle, though. “We’re still relatively early, so there’s only a small number of services onboarded to it, but already for them the handoffs are much faster and more direct than pre-Brain,” Russinovich says.

In the long term, he wants agents to replace the deterministic remediation rules that teams write today.

“We don’t have to write down every single rule prescriptively,” he says. “And have this tree of decision making, but rather let the agent do things based on its own judgment, which has a whole bunch of benefits, like the system keeps up to date automatically. Then it can also find paths to resolution that we might miss in the deterministic rules that we’ve got.” On agents that actually fix things, he says, “We still consider ourselves at the beginning of that.”

The post Meet Brain, the AI that decides when Azure is officially down appeared first on The New Stack.

Most engineering teams I talk to can ship an AI demo. The prototype works, stakeholders are impressed, and everyone agrees the use case has potential. Then the project hits a wall.

The reasons for this can vary, but new research shows that difficulties in collecting and parsing real-time data from multiple sources are often the problem. And it’s compounded by a growing skills shortage.

“Only 32% of organizations report having agentic AI running in production.”

According to Confluent’s 2026 Data Streaming Report, only 32% of organizations report having agentic AI running in production. At the same time, two-thirds of respondents cited data infrastructure and data quality as barriers to the success of agentic AI. The models work in controlled conditions, but production is a different story.

Why the demo-to-production gap is so wide

Demos tend to work because everything around them is controlled. The data is static and curated carefully to support exactly what the model will be asked to do. Production environments don’t always offer those luxuries.

In production, AI systems have to query data that lives across dozens of sources, including databases, event streams, application logs, and third-party feeds. Much of that data is poorly governed, and little of it is designed to be consumed by an AI agent in real time. Models that looked impressive in pilots return unreliable results because they’re working with stale, incomplete, or uncontextualized data.

The instinct is to tune the model, but the problem is more likely to be the data feeding it.

In the report, 72% of IT leaders cited insufficient infrastructure for real-time data processing as a barrier to scaling AI, up from 61% the year before. That increase suggests the problem isn’t going away; it’s getting more visible as teams move projects into production.

“The instinct is to tune the model, but the problem is more likely to be the data feeding it.”

AI systems need data that’s trustworthy, contextualized, and current, and those properties are hard to guarantee when data is sitting in siloes that weren’t built for continuous consumption. Batch pipelines almost always introduce latency, lack formal data contracts, and obfuscate lineage. The AI system ends up working with an inconsistent, partial snapshot of the business instead of what’s actually happening now.

The skills problem makes this harder

The report reveals another challenge: 71% of IT leaders cited a shortage of relevant expertise and skills as a barrier to AI adoption. 

The work of application development has shifted from encoding business logic to creating an information environment where automated systems can learn and generalize.  Building reliable AI applications requires developers to be stronger data engineers. They need to understand distributed systems, streaming architectures, data quality controls, and how to build pipelines that hold up under real-world conditions. They need to reason about data lineage, schema evolution, and what happens when an upstream source changes. And the QA patterns that work for deterministic software — where the same input yields the same output — don’t transfer to probabilistic systems.

Most developers haven’t had to think this way before. The discipline of getting the right data to the right system at the right time, in a governed and reusable way, has gone from a specialist concern to a requirement for anyone building production AI.

This affects how organizations should think about closing the demo-to-production gap. The investment in data engineering skills needs to keep pace with the investment in AI itself.

What production-ready AI actually requires

Organizations that make it out of the pilot stage treat data infrastructure as a first-class concern from the start. That means building real-time pipelines rather than batch processes. It means applying schema definitions, ownership metadata, and quality checks at the point of data production rather than in the data lake. And it means structuring data as reusable products that different teams and applications can build on, so the engineering work supporting one AI application can accelerate the next one, rather than starting from scratch.

The 2026 report found that 88% of IT leaders said data streaming platforms help address data infrastructure and quality issues for agentic AI. That’s because they address the specific reasons AI projects stall — real-time data delivery, upstream governance, and making data trustworthy enough to use at inference time.

The shift is already happening

For the first time, the report found that investments in data streaming outranked those in AI and machine learning, by 88% to 82%. Organizations that have tried to ship production AI are increasingly recognizing that the model isn’t the hardest part. 

“For the first time, the report found that investments in data streaming outranked those in AI and machine learning, by 88% to 82%.”

So if you’re stuck at the pilot stage, resist the urge to keep optimizing the model. A better question is whether the data feeding the model is fresh, accurate, and well-governed, and whether your pipelines were actually built for production AI or a demo that only had to work once.

The post The impressive AI demo is dead. Here’s what actually reaches production appeared first on The New Stack.

Agentic systems usually have two jobs: Build context, then use that context to produce an answer or action.

Many failures that look like LLM problems start in the context-building step. The answer the LLM gives is limited by the context it was given, or it finds through tool calls. If the agent model cannot find the right sources, then improving the generation model will not improve the overall system.

“Many failures that look like LLM problems start in the context-building step.”

A client, Specstory, wanted to give users the ability to ask questions from the agent’s history. For example, why a team chose Authlib for authentication and what alternatives they considered. The chatbot needs the right prior conversations, decisions, and tradeoffs from a large corpus of coding sessions. The model and system prompt help only after those chat turns have been retrieved and are in context.

If retrieval ranks implementation snippets above the discussion where the team weighed alternatives, the agent can still produce a confident answer. It may find code that imports Authlib and a few inline comments, then describe the decision based on implementation evidence rather than the actual trade-off discussion.

The same pattern showed up in an AnkiHub operator review in our private community. A request for help studying based on lecture slides only works if the agent’s tool calls retrieve the right flashcards. The hard part is not finding any related cards. A lecture on the function of the heart may match hundreds of cards. Ranking decides whether the core cards make it into context or whether the system has to raise top_k and flood the prompt.

The exact setup changes by product. The context-building step might use local search, semantic search, web or API calls, or database queries. It might be handled by an agent, a fixed workflow, or application code. The process stays the same: gather the right context, then generate from it.

For example, a coding agent runs rg, opens files, reads logs, and inspects tests before writing a patch. A research agent searches the web and internal notes before writing an answer. A study assistant searches deck facts and user context before suggesting what to learn next.

When context building fails, the symptoms look like generation failures.

Retrieval failures mimic generation bugs

Symptom	Retrieval cause
Hallucination	The answer source never made it into context.
Context rot	Low recall forces a high top_k, so noisy results fill the context window.
Latency	Weak retrieval leads to more tool calls, larger candidate sets, and larger context windows.

A better model helps with reasoning and writing, but it cannot give a better answer without the right context.

“A better model helps with reasoning and writing, but it cannot give a better answer without the right context.”

The Mixedbread OfficeQA-Pro Eval shows the same pattern at the benchmark scale. OfficeQA-Pro uses 89,000 pages of financial documents, dense tables, scanned PDFs, and questions that require reasoning across documents. Giving Codex better search tools reduced tool calls and improved answer quality.

Plain-text tools like grep and rg work (ish) on flat code files. They do not work well when context lives in PDFs, tables, chat histories, multi-modal inputs, web results, and permissioned data. In those cases, the agent needs a retrieval that can combine exact terms, meaning, metadata, permissions, and ranking quality.

Retrieval needs traces and evals

Once retrieval enters the architecture, the next question is whether it finds the right information.

For that, you need traces and evals. For each retrieval step, the minimum trace is the input, the outputs, and a way to label whether each output was relevant.

For a coding agent using rg the input is the command, the output is the returned snippets, and the label says which snippets helped, which were noise, and which relevant files were missing.

For product retrieval, the step might be BM25, semantic search, hybrid search with reranking, a generated SQL query, or something else. Capture the query or arguments, the returned documents or chunks, and whether those results were helpful.

Trace each retrieval step by itself, then evaluate the full context-building pass. The local trace answers “Did this query return useful material?” The full trace answers “Did the system collect everything the model needed before generation?” If it did, failures are a generation problem. If not, it’s a retrieval problem.

You cannot know where the failure started or what to fix without traces.

Different failures need different fixes

“Improve retrieval” is too broad to be useful, as different problems require different solutions. If a relevant document is missing, the trace should show where it disappeared: query building, retrieval, filtering, ranking, or final context assembly.

The failed step, plus what the trace shows, tells you what change to make.

Failed step	What the trace shows	Change to make
rg / grep	A conceptual query returns literal matches while missing relevant files.	Add semantic search over files or chunks, or generate better keyword queries before calling rg.
BM25	The query uses the right concept but different words from the source material.	Add semantic search, synonyms, or query expansion.
Semantic search	Exact names, error strings, document IDs, or domain terms are missing from the results.	Add a keyword or BM25 path, or boost exact term matches.
Hybrid retrieval	The relevant passage is ranked 7th, but the context only takes the top 5.	Add or tune a reranker, or raise candidate top_k before reranking.

The right fix depends on what the system was trying to retrieve. A decision-history question requires the decision, the alternatives, and the chats in which the team worked through them. A study question depends on the lecture material, deck metadata, semantic matches, and the user’s study context.

The architecture

Once you trace individual retrieval calls, the full architecture has a simple shape: fan out to context-building tools, then fan in to generate the final output.

The retrieval layer might be a search engine, a vector database, an SQL query, a local file tool, a web search API, or a custom service. The pattern stays the same: build candidate context, narrow it, rank it, assemble it, then generate from it.

Give agents human search controls

Semantic search compares embeddings (numerical representations of meaning). It helps when wording differs, but most retrieval intents also depend on structured constraints. A meeting search box can use semantic search over transcripts and notes, but a useful interface also lets someone filter by person, date, project, and source. 

A finance search may need the latest filing, a specific quarter, or an official source in addition to the closest semantic match. In e-commerce, the best semantic match for “32×30 cargo pants” may be an out-of-stock product. The system still has to decide whether to hide it, return it with a backorder note, or show it so the user can check later. That product decision is a retrieval decision because it changes which candidates reach the agent.

In a chat interface, those controls are in the tool schema, query planner, or app logic. If an agent runs the search, it needs arguments for the same constraints a human would set with filters, sliders, tabs, and sort menus.

A retrieval system usually needs several controls working together:

Control	What it does	Example
Exact match	Matches names, IDs, error strings, quoted phrases, tickers, or product codes.	Find EADDRINUSE, Authlib, or a specific SEC accession number.
Semantic match	Finds related content when the wording differs.	Find the meeting where the team discussed authentication tradeoffs.
Hard filters	Removes invalid results before ranking.	Limit by tenant, permissions, person, date range, size, or stock status.
Sorts	Orders candidates by a structured field.	Prefer the newest, latest filing, lowest price, highest rating, or recency.
Ranking	Scores candidates based on their likely usefulness for this request.	Combine semantic match, exact match, freshness, source quality, and use.
Reranking	Uses a slower model or scorer on a smaller candidate set.	Compare the query against the top 100 candidates before returning 10.

Here, a chunk means a small piece of source content, and a candidate is a chunk returned by the first search step. Ranking is the scoring step that orders those candidates. Context assembly then selects which chunks and structured fields to include in the model prompt.

Better ranking improves precision, which means a larger share of the returned chunks is useful. If the relevant chunks are near the top, the system can pass fewer chunks to the model, use fewer tokens, reduce latency, and expose the model to less noise. If the right chunk is ranked 40th and the context only includes the top 10, the system behaves as if the retrieval missed it.

People and agents use the same basic search path: ask for results, inspect what comes back, and decide what to use. A person can skim ten search results, compare titles, snippets, dates, domains, and URLs, and decide whether the result set looks right. They can open the third result, ignore the rest, and search again with a better query. An agent usually receives a bounded set of returned documents and reasons from the context. If the right source falls below the cutoff, the agent may answer from partial context. To avoid this, the system has to retrieve more candidates, run more searches, or pass more evidence into the model.

A missed document can change what the agent searches for next. Suppose someone asks why the team chose Authlib. If the first search misses the transcript where the team compared Authlib with alternatives, the agent may search the codebase instead. It finds imports, callback handlers, tests, and maybe a comment. Then it asks follow-up questions about OAuth configuration. The context starts to look complete, but it supports the wrong answer. It explains how Authlib was used and why it makes sense in the codebase, not why the team chose it.

“The context starts to look complete, but it supports the wrong answer.”

But ranking cannot repair every search problem. If the agent failed to request the latest filing, a reranker may faithfully select an older document with a closer wording match. If the tool has no date_range, person, source_type, size, or in_stock argument, the model has to impose hard constraints in the search text and hope that retrieval infers them. A hard filter gives the model less to infer, making semantic search more reliable.

Scale changes the retrieval problem

Search systems already have tools for this: indexes, filters, facets, sorts, caching, bounded reranking, and freshness jobs. Agent systems need the same discipline.

A human might search, adjust a date filter, scan the first page, then search again. One agent request can do that many times in seconds: rewrite the query, run keyword and semantic search, inspect thin results, issue follow-up searches, fetch sources for citations, and ask for more context before answering. With many concurrent users or agents, the retrieval layer can become a bottleneck.

Humans often wait through a slow search if the result is good. Agent systems often turn slow and uncertain search into more work. When ranking is weak, teams compensate by raising top_k, running keyword and semantic searches in parallel, adding reranking, fetching more source documents, and passing larger evidence bundles to the model. That can improve answers, but it moves the cost into tokens, latency, and retrieval load. A better ranking lets the system return fewer, better candidates, rather than making every request carry a larger pile of possible evidence.

With a small corpus, you can still search comprehensively quickly and cheaply, even with fully agentic approaches. That’s what I recommend when you’re starting and don’t have much data. Don’t add complexity until you need it. But with millions or billions of chunks, every extra retrieval call, candidate, ranking pass, and returned token adds up quickly.

Multi-stage retrieval is the production shape

Most production systems should split retrieval into stages, even when the UI is a chat box.

Stage	What happens	Trace question
Search argument construction	The app or agent turns the request and state into a query, filters, and sort.	Did it ask for the right content with the right constraints?
Candidate generation	The system finds plausible chunks from text, vectors, or structured data.	Did the right source enter the candidate set?
Filtering	Permissions and product constraints narrow what can be returned.	Was the source correctly excluded or wrongly lost?
Sorting	Structured fields order results when order matters.	Was the latest, cheapest, highest-rated, or current item surfaced?
Ranking	The system scores the candidates based on their usefulness for this request.	Was the source present but ranked too low?
Summary return	The system returns only the fields the agent needs.	Did the app receive usable evidence and provenance?
Context assembly	The app selects, formats, and budgets evidence for the model.	Did useful evidence get dropped before generation?
Evaluation	Humans or automated checks label whether the retrieval path worked.	Can the team turn the failure into a specific fix?

Each stage leaves a different repair path. If the agent chose the wrong filters, changing the embedding model will not help. If the latest document was available but the tool never sorted by date, the fix belongs in the search arguments or retrieval API. If the right source was present but below the cutoff, the fix belongs in the ranking. If the right source came back but was dropped before generation, the bug is in context assembly.

As retrieval becomes a core part of agent architecture, teams increasingly need infrastructure that can combine semantic search, exact matching, filtering, ranking, and large-scale retrieval in a single system. Depending on requirements, this may involve search and retrieval platforms such as Vespa, Elastic, or Coveo, each of which supports different approaches to ranking, retrieval, and operational scale. 

The important point is not the specific technology choice, but recognizing that retrieval quality has become a first-class engineering concern. As agent workloads grow, retrieval systems are increasingly determining the accuracy, cost, latency, and reliability of the overall application.

The post Why retrieval quality is becoming the defining challenge in AI agent architecture appeared first on The New Stack.

OpenAI announced ChatGPT Work on July 9 and began rolling it out to Pro, Enterprise, and Edu users. It runs on the new GPT-5.6, opens a user’s local files, edits Google Workspace and Microsoft 365 documents, and carries a multi-step task through to a finished deliverable.

Reuters placed it directly against Anthropic’s Claude Cowork, and both target the same person — a non-coder who wants the power of a coding agent without the terminal. Counting Anthropic, Microsoft, Perplexity, and Amazon, five leading labs have now released an agent of this kind. The batch shows that the newest agents are organized more by their intended users than by their functions.

Based on the target user persona, four archetypes appear: the knowledge worker, the power user who self-hosts, the developer, and the enterprise. The personas often overlap since one individual can embody all three roles. Additionally, a product like Claude Code caters to both solo developers and platform teams.

Therefore, consider these deployment archetypes categorized by the main buyer, rather than strict separations. The archetype only represents what marketing promotes. Behind the scenes, each lab has almost consistently decided who owns the runtime, persists memory, manages credentials, and enforces policy.

Four archetypes, based on the user persona

Let’s analyze each of the four archetypes individually, as each differs in the level of control available to users.

The first archetype serves the knowledge worker. A vendor operates the runtime and sells the agent as a delegation to someone who lives in documents rather than code.

ChatGPT Work is the newest, alongside Claude Cowork, which now runs cloud sessions on web and mobile while keeping local-file access on the desktop; Microsoft’s Copilot Cowork, a cloud-hosted agent that executes long-running tasks inside the Microsoft 365 trust boundary; Perplexity Computer, which works across local files and Microsoft apps; and Amazon Quick, the successor to Q Business as that product closes to new customers at the end of July. The user grants access and supervises the result. In most cases, the vendor manages the runtime and persisted state, except for Perplexity’s local option.

A second archetype belongs to the power user who self-hosts. The provider controls the persistent agent process and chooses where to store the state and credentials, often on a Mac mini that has become a piece of personal infrastructure in its own right.

OpenClaw and Hermes are the reference examples, open source, and run on the operator’s own machine. Self-hosting involves managing the control plane rather than full local custody, since both options still allow access to a hosted model and the storage of credentials for external services.

The developer gets the third archetype, whose runtime spans the IDE, the terminal, the repository, and a cloud sandbox. Claude Code, OpenAI Codex, GitHub Copilot in agent mode, and the open-source OpenCode all live here, and Amazon’s developer agent is folding into its Kiro tool. Coding-agent execution is extending from the local IDE to vendor-managed sandboxes and asynchronous cloud workers, making this the most challenging archetype to categorize clearly.

The fourth archetype is built for enterprise workflows and integration with business processes. They run an open agent framework, such as LangGraph or CrewAI, on a managed, governed runtime. ADK on the Gemini Enterprise Agent Platform, Strands on the Bedrock AgentCore, Microsoft Agent Framework on Foundry Agent Service, and Claude Managed Agents belong to this category. OpenAI’s Agents SDK can be hosted on some of these runtimes, including AgentCore, which AWS lists as one of its supported frameworks. The vendor operates the infrastructure, and the customer configures identity, policy, and retention on top of it.

Persona	Representative products	Runtime ownership	State and credentials	Platform type
Knowledge worker	ChatGPT Work, Claude Cowork, Copilot Cowork, Perplexity Computer, Amazon Quick	Vendor cloud for most, with per-folder local access on some	Vendor persists session state, user grants scoped credentials	Packaged experience
Power user, self-host	OpenClaw, Hermes	The operator’s own machine	Operator chooses where state and tokens live, though inference is often external	Packaged experience, self-operated
Developer	Claude Code, OpenAI Codex, GitHub Copilot, OpenCode	Split across the IDE, the laptop, and a cloud sandbox	Repo and local for now, drifting into hosted sandboxes	Swing, moving toward platform
Enterprise, workflow-driven	ADK on Agent Platform, Strands on AgentCore, MAF on Foundry, Claude Managed Agents	Managed vendor runtime, customer-configurable	Customer defines identity, policy, and retention; platform brokers	Programmable platform

The line below the personas

The persona-based approach abstracts four things: where execution runs, where state is persisted, how authority is delegated, and where policy is enforced.

Anthropic describes its design as decoupling the brain from the hands. The harness that calls Claude runs separately from the sandbox where code executes, and a session, an append-only log of every model call, tool call, and result, connects the two. Because the sandbox is kept separate from the brain, the agent can start reasoning before any container exists, and the code it runs remains far from the developer’s credentials.

The same four planes show up at the other vendors. AgentCore Runtime gives each session a dedicated microVM with an isolated CPU, memory, and filesystem, and meters compute usage. Google can route governed traffic through its Agent Gateway, where Model Armor policies inspect configured ingress and egress flows, while Agent Identity and an Agent Registry track the fleet. Microsoft assigns each hosted agent a dedicated Entra Agent ID and runs it in a per-session sandbox whose filesystem survives idle periods.

Deploying agents on a managed runtime is more like leasing a workshop than purchasing a tool… a long-term tenant installs their own locks, maintains their records, and takes their tools when the lease ends.

Deploying agents on a managed runtime is more like leasing a workshop than purchasing a tool. The landlord manages the building and supplies the power, but a long-term tenant installs their own locks, maintains their records, and takes their tools when the lease ends. The personas often conceal this difference. Currently, the vendor typically operates the runtime, so the key question is how much control the customer can still exert and what they can take away across the four planes.

Where the line falls

What differentiates each offering is not the compute operator, since vendors handle nearly all of it. It is how much of those four planes a product leaves the customer to configure and export. A packaged experience hands nearly all four to the vendor and returns supervision and a finished outcome. A programmable platform operates the infrastructure but lets the customer define identity, policy, and retention and move the code elsewhere.

Copilot Cowork shows that the two axes are separate. It is a packaged knowledge-worker experience, yet it runs on a governed enterprise platform and inherits Microsoft’s identity, compliance, and audit controls.

The persona sells the product, but the four planes decide the lock-in.

A product can be packaged on the surface and programmable underneath, which is why the personas and the control planes have to be read as different questions. ChatGPT Work makes the same point from the developer side, since OpenAI’s new desktop app folds Chat, Work, and Codex into a single surface, though OpenAI has not detailed how far the runtime or credential store are shared beneath it. The persona sells the product, but the four planes decide the lock-in.

Usecase	Agent Type	Tradeoff
Delegate a knowledge task to an agent you supervise	A knowledge-worker agent such as ChatGPT Work, Copilot Cowork, or Amazon Quick	The vendor typically operates the runtime and persists state, and you configure little below the surface beyond access and approval
Keep state and credentials on hardware you control	A self-hosted agent such as OpenClaw or Hermes, in a local configuration	You control the persistent process, though model inference and some tools may still be remote
Ship code changes across the IDE, repo, and CI	A developer coding agent such as Claude Code, Codex, or Copilot	Execution spans your tools and a cloud sandbox, so ownership is split and worth mapping before you commit
Run many governed agents with audit and identity	An enterprise runtime platform such as AgentCore, Agent Platform, Foundry, or Managed Agents	The vendor operates the infrastructure while you define identity, policy, and retention, in exchange for coupling workflow logic to one cloud

Real deployments combine the rows rather than picking one. Teams on Foundry Agent Service commonly run open-source orchestration, such as LangGraph, for agent logic while leaning on the platform for governed execution, and Microsoft’s own hosted runtime now supports long-running personal agents like OpenClaw and Hermes with durable state. The boundary between experience and platform is not a thick, well-defined boundary, but a thin line a single system can cross, bridging rival camps.

The agent market is not being decided by open against closed. Open frameworks like Strands, ADK, and Microsoft Agent Framework are precisely what the governed runtimes are built to host.

The agent market is not being decided by open against closed. Open frameworks like Strands, ADK, and Microsoft Agent Framework are precisely what the governed runtimes are built to host. Vendors now manage the runtime across nearly all archetypes, shifting the competition to who can most effectively configure and export state, identity, and underlying policy.

If coding agents enter managed sandboxes alongside knowledge-worker agents, the developer archetype will be established on the platform side. The map will then transform into what the vendors are already outlining. When enterprise teams assess agents, their most important question should be how much of execution, state, identity, and policy they can configure and extract from the product. They should not focus on which persona it presents or whether its framework is open, as the framework no longer determines the product’s value or locking-in capabilities.

The post OpenAI, Microsoft & Anthropic agree on who runs the agent. They disagree on what you can take back. appeared first on The New Stack.

Thomas Dohmke, who stepped down as GitHub’s CEO last year to become a founder again, is opening a preview of a distributed Git network on Wednesday that is designed to keep fleets of AI coding agents from overwhelming a single central server — and one that may soon compete directly with GitHub’s core service.

Entire, Dohmke’s post-GitHub startup, is launching a preview of this on Wednesday (but for now, it is behind a waitlist). With this, developers can mirror an existing GitHub repository onto Entire’s own infrastructure in one step.

“In the era of agents, centralized Git hosting has become a fundamental constraint, as the strain of billions of agents and developers hammering a central server shows up in the form of rate limits, high latency, or even outages,” says Dohmke in today’s announcement. “Today, we begin to return Git to its original promise, with a distributed, and soon fully decentralized and open-source network of interconnected nodes around the world. By doing so, we enable any developer or agent to host their code in-region, pushing, pulling, and cloning close to where they operate, fast and without bottlenecks, while still part of a global, collaborative network.”

The key here is that the code stays on GitHub, as Entire stresses, but coding agents can work with the Entire mirror and, as the company notes, “build without rate limits.”

Entire’s mirror is meant to absorb the constant flow of traffic that a fleet of agents can generate. That traffic, after all, is part of the reason GitHub is often buckling under pressure these days and startups like Entire have an opening.

Centralized Git hosting, Dohmke says in an interview with The New Stack, has become “a fundamental constraint” now that billions of agent and developer operations land on the same servers, showing up as rate limits, latency, and outages.

Given GitHub’s recent availability issues, it’s no surprise that startups are trying to get into this space. Entire is one — and it has the pedigree — but in June, Cursor also announced Origin, its own Git forge rebuilt for swarms of agents that are cloning and committing against a single repository in parallel.

Entire is starting with active regions in the United States, the European Union, and Australia, but the team says that now that is has spun up its first few regions, it will add more soon.

‘Git as a database’

To build its network, Entire rewrote the server part of git. GitHub, GitLab, and Bitbucket all wrap the server-side of the Git binary and build their infrastructure around it. Entire started from scratch.

“We see Git really as a database,” Dohmke says. The open source Git project has two halves, he explains: the client that an agent uses to talk to a repository, and the server a host runs to manage storage. Rather than build on that stock server, like most companies would do, “we made the decision of not going that route, and instead implemented our own Git backend.”

That only makes sense if Entire’s version has significantly better performance than the stock Git server, of course. Entire says its benchmarks have pushed the network to a sustained rate of 570,000 clones per hour, 586 pushes per second, and roughly 470 combined clone-and-push operations per second.

Pushing to a native Entire branch can run up to 25 times faster than pushing through to GitHub, Dohmke says.

Entire it will open-source both the git backend and the benchmark suite.

The foundation layer, now real

When Dohmke first described Entire’s plans to The New Stack in February, he described a three-layer platform that included a Git-compatible database at the bottom, a semantic reasoning layer in the middle, and an interface on top. Even then, he said that the database, unlike a centralized Git host, could be a globally distributed network of nodes.

But in February, Dohmke also said Entire wouldn’t necessarily end up competing with GitHub, and that code repositories would stay central to the pitch.

Pressed on whether that still holds now that Entire hosts its own copy of the GitHub repo, he calls the mirror complementary, in part because Entire can offer enterprises the ability to keep their code in a local region to fulfill local regulations. He also notes that GitHub has a huge ecosystem and an extended feature set.

“I think the question for the buyer really is, is it not better for me from an availability and reliability perspective, that I have both of these products, so if one of them is down — there’s always going to be single points of failure and human errors — then I have my mirror on the other side,” he says. “But we certainly will, in deals, compete for the dollar spent at a much smaller scale compared to the multi-billion-dollar business that is GitHub today.”

For now, that keeps the two complementary. Dohmke argues that GitHub remains the “source of truth,” or “cold storage,” while the working copy lives on Entire. But he also says that Entire will launch native repositories in the coming months, and those wouldn’t need GitHub underneath at all. All of this will be open-sourced as well.

Entire raised its $60 million seed round in February, when it had 15 employees. Felicis led it, with Microsoft’s venture arm among the backers. The company is now past 40 people and aiming for 60 by the end of the year.

Entire beyond Git: the semantic memory layer

Entire is building its middle layer — the semantic reasoning layer — in parallel with the Git platform.

The semantic layer now integrates with every major coding agent, including Claude Code, Codex, Cursor, Factory AI, and GitHub Copilot, and records each session, prompt, and tool call in the repository alongside the code.

Having this data is useful for agents, and it was the first core service the company launched. Now, it is also building more services on top of that history.

The company is adding Entire Blame, for example, which shows not just who last touched a line but the agent session and prompt behind it. There is also Entire Review, which fans out several agents for an intent-aware review, and the company is adding a code and semantic search feature that lets agents (and developers) search across code changes and the reasoning that produced them.

“Session logs are now the second most important artifact in software development, and they belong in the repository alongside the code,” Dohmke says.

The post Entire is building a Git network for agents appeared first on The New Stack.

Your organization constantly needs more information about system performance, usage, and data while in production — or better yet, before it heads to prod. The challenge of telemetry increases with the complexity of your stack and agentic sprawl. Because “it works in the testing environment” becomes moot in the face of non-deterministic agents.

After all, AI agents span multiple environments, and that leaves traditional log-metric-trace models insufficient to handle the volume of the agentic AI era. The situation can lead companies to think that the best option is to throw everything into the locked box of proprietary tooling, but that creates another problem: Information is siloed within each layer, fragmenting data and taking you further from realizing real AI ROI.

Unified context across fragmented workflows

The OpenTelemetry framework and the OpenSearch distributed search and analytics engine make for a powerful, open-source pairing that gives organizations of all sizes unified context across their fragmented workflows. In fact, OTel has crossed the 95% adoption threshold for new cloud-native instrumentation projects and has already become the default choice for Greenfield projects.

OpenSearch, sponsored by Amazon Web Services, is gaining traction with AI engineers, as it recognizes that observability and AI must be united. This year’s OpenSearch roadmap specifically focuses on making it the primary retrieval interface for AI agents and an essential piece of any retrieval-augmented generation and agentic AI stack. 

Join us on July 22

Just because open source doesn’t have a direct cost doesn’t mean it’s free. That’s why Dotan Horovits and Rekha Thottan of AWS are going to perform a live troubleshooting simulation using correlated logs, metrics, and traces, followed by a demo of how agentic traces flow through Otel pipelines. Also learn how the open-source evaluation framework Agent Health can provide a structured pre-production benchmark to flag unpredictable agentic behavior before release. 

Join us live on July 22 to learn along and ask questions to learn how your organization can adopt these open-source standards in the second half of this year — across agentic workloads and traditional infrastructure, at scale.

Register for the webinar here

The post Watch AWS engineers troubleshoot agentic AI with OpenTelemetry and OpenSearch appeared first on The New Stack.

Industrial machinery generates more alarms than technicians can triage. For each important alarm requiring follow-up, the technician pulls historical context,...

Industrial machinery generates more alarms than technicians can triage. For each important alarm requiring follow-up, the technician pulls historical context, determines the correct procedure, checks whether a specialist signal confirms the failure mode, and writes up a recommendation. This process remains consistent, and is well-suited for an AI agent. This post discusses a per-alarm…

Source

In April, Andrej Karpathy published a GitHub gist file called “LLM Wiki,” a brief text document designed to help one build a personal knowledge base using LLMs. It’s based on the premise that an AI agent will keep what it knows as linked Markdown files it can read and rewrite, because a language model does not get bored maintaining cross-references and can touch fifteen files in a single pass. It was only a few thousand words with no product attached.

Two months later, Google turned that instinct into a published standard called the Open Knowledge Format. The OKF packages organizational knowledge, metrics, tables, and runbooks as plain Markdown that any agent can read without a proprietary account. Google is careful to call it v0.1 — a starting point rather than a finished standard.

Garry Tan, the Y Combinator president, got there first in a different lane. His gstack, an MIT-licensed Claude Code setup that crossed 66,000 GitHub stars within weeks, comprises 23 specialist roles, each a Markdown file. No runtime; no code; just prose that runs across ten different coding agents.

Markdown has become the substrate agents read and write

Three approaches, three different needs, one common solution. Karpathy sought agent memory, Google aimed for enterprise context in BigQuery agents, and Tan wanted a way to summon an engineering team from a terminal. All three turned to the same basic resource: a folder of Markdown files versioned in git.

Developers had already established this practice. CLAUDE.md and AGENTS.md are present in millions of repositories as the initial files an agent loads. OKF and gstack are the evolved forms of this convention – one focused on what the agent knows, the other on how it behaves.

This is the Git and JSON playbook tied to the agent’s knowledge. The formats that survived are the ones you could start using without changing anything. You can simply cat the file, clone the repo, and any tool you already use can parse it. MCP remains important as the interface an agent connects to. Markdown is becoming the format that carries the content.

The lock-in moved from the model to the files

The significant factor to observe here is the competitive advantage, not technical specifics. For two years, the belief was that owning the best model meant controlling the developer.

This perspective is now shifting. Replacing Claude with GLM or Codex, gstack continues to operate because the core intelligence evolved, but the documentation did not.

The moat is shifting from the model to the Markdown a team owns and accumulates over time.

The moat is shifting from the model to the Markdown a team owns and accumulates over time. A company’s OKF bundle, including its runbooks, metric definitions, and architecture decisions, is, by design, portable across clouds, models, and frameworks.

That kind of portability is the reason vendor-neutral formats exist and why Google’s OKF deserves a closer look.

If no one develops consumers for it, it remains just a good idea that Google released on a slow Friday.

The area where I am most likely mistaken is durability. Declaring Markdown standards is easy, but making them reliable is difficult. OKF is merely a 0.1 draft with a reference implementation, not a full ecosystem. If no one develops consumers for it, it remains just a good idea that Google released on a slow Friday.

The direction remains determined by three separate bets targeting the same file format within a single quarter. Your next agent is likely to interpret its context from a Markdown folder, and the creator of that folder now possesses an advantage that the model vendor cannot easily replicate.

The post Andrej Karpathy, Google and Garry Tan agree Markdown is the answer, but they’re not solving the same problem appeared first on The New Stack.

Halfway through 2026, artificial intelligence has been at the center of every major story inside the world of software development and in just about every major story outside of it.

Last month, the Commerce Department ordered Anthropic to pull Fable 5 and Mythos 5 offline worldwide — only to lift the ban 18 days later. It wasn’t Anthropic’s only government clash; earlier in the year, the Pentagon fought the company over its refusal to give the military unrestricted model access.

Elsewhere, frontier labs planted their flag on Wall Street in May with new deployment arms and partnerships, while Anthropic and OpenAI both pursued IPOs at valuations above $800 billion. Underneath the valuations is an infrastructure buildout of chips, data centers, and deals meant to keep pace with model releases that land every few weeks.

Meanwhile, open-weight models are narrowing the gap between closed and downloadable, and the harness — the tools, memory, and orchestration around a model — matters more as agentic AI moves into the enterprise. Add tokenomics (the real cost of reasoning at scale) and non-technical execs vibe-coding their own tools, and you’ve got ten of the biggest AI moments of 2026 — so far.

Here are ten moments that have defined a pivotal first half of 2026 in the world of AI, as chosen by the editorial staff at The New Stack.

10. President Trump’s Executive Order on AI

President Donald Trump has been relatively friendly to AI, given his support from tech companies in Silicon Valley. On June 2, 2026, Trump signed an executive order aimed at hardening American systems against AI- related threats, while at the same time rejecting “overly burdensome regulation.” The order directs the Committee on National Systems Security to prioritize cyber defense and tasks the Treasury, the NSA, and CISA with establishing an AI security clearinghouse to coordinate vulnerability scanning and patching across critical infrastructure. The administration supports deregulation along with national security-driven AI oversight.

9. AI infrastructure buildout

Chipmakers and AI labs tightened ties in 2026 to keep pace with model releases. We also saw big moves, with Nvidia and SK Hynix striking a multi-year partnership spanning Vera Rubin supercomputers, Vera CPUs, and next-gen memory. Meanwhile, data center capacity has expanded globally. The buildout shows that compute, power, and hardware are now a bottleneck to AI’s growth.

8. The rise of the harness

“The harness is where the hard work is,” Harness CEO and founder Jyoti Bansal told The New Stack last month. As base models are performing closely on benchmarks, the harness becomes the differentiator. The harness is the tools, memory, orchestration, and guardrails wrapped around a model. The harness determines whether an agent stays on task, recovers from errors, and performs safely. The harness helps shift competitive advantage from model capability to system design.

7. Tokenomics

Spend is now the battleground for both AI producers and consumers. AI labs are restructuring pricing around compute consumption rather than flat subscription, and companies are looking for ways to cut their token spend. Last month, the Linux Foundation launched the Tokenomics Foundation with support from Google, Microsoft, IBM, JPMorgan Chase, KPMG, Oracle, and Salesforce. The organization is tasked with establishing open standards, benchmarks, and best practices across the entire AI token economy.

6. Agentic AI goes mainstream

A year ago, agents were largely proof-of-concept demos that were unreliable in production. In 2026, they became infrastructure. For instance, ChatGPT’s browsing agent, Claude’s tool use and multistep coding runs, and Google’s autonomous information agents now run continuously in the background rather than on command. Meanwhile, enterprises are adding agents into real workflows such as monitoring, code review, procurement, and customer support. However, this shift may carry security risks as agents gain access to data and systems.

5. The Pentagon goes to war with Anthropic

In February, Department of War Secretary Pete Hegseth summoned Anthropic CEO Dario Amodei to his office to demand that the military be granted unrestricted use of the company’s technology. But Amodei held his ground and refused to allow the military to use Anthropic technology for mass surveillance on citizens or for autonomous weapons. Following that, President Trump ordered federal agencies to phase out the use of Anthropic, and Hegseth designated the company a “supply chain risk,” which is a label previously used for foreign adversaries – basically blocking the company. Anthropic sued in federal court, claiming the government’s move was unfounded and retaliatory. The company received a preliminary injunction from a San Francisco court, holding that the government’s actions constituted unconstitutional First Amendment retaliation. 

Meanwhile, amid the initial fallout from Anthropic’s battle with the Pentagon, OpenAI made its own deal with the military.

4. AI titans plant their flags on Wall Street

Within 72 hours in May, Anthropic and OpenAI each launched enterprise deployment arms, announced major financial services partnerships, and shipped agent tooling targeting Wall Street workflows. The message was the same — the next phase of frontier AI is not about models. It’s about deployment. 

Anthropic’s new services firm — backed by Blackstone and Hellman & Friedman alongside General Atlantic, Apollo, Goldman Sachs, and Sequoia Capital — targets mid-sized enterprises that the large consulting and systems integration firms don’t prioritize. These include community banks, regional health systems, and mid-market manufacturers. Applied AI engineers from Anthropic embed directly with clients alongside the new firm’s own engineering staff, doing workflow discovery, building custom Claude-powered solutions, and supporting clients long-term.

OpenAI’s Deployment Company — “DeployCo” — operates one market segment up, targeting large enterprises with the same forward-deployed engineering model. Its acquisition of applied AI consulting firm Tomoro brings roughly 150 experienced Forward Deployed Engineers (FDEs) from day one, backed by more than $4 billion in initial investment and a partner roster that includes McKinsey, Bain & Company, and Capgemini.

Meanwhile, both companies are considering IPOs with valuations exceeding $800 billion.

And both companies are betting on the same thesis: that the deployment gap — the widening distance between what frontier AI can do and what enterprises have actually shipped — is the next major revenue opportunity. And both moved on it in the same week. 

3. Open-weight models are coming

Chinese labs continued to close the gap with Western frontier labs in 2026. Alibaba’s Qwen, Zai’s GLM and Moonshot’s Kimi, delivering open-weight releases that rivaled closed models on standard benchmarks. Zai’s GLM-5.2, released June 13, beat Anthropic’s Claude Opus 4.8 in some benchmarks – showing the highest marks for open-weight models. GLM-5,2 is also one-fifth the price of comparable closed models. 

“The industry is all focused on which lab has the smartest model, but that focus looks to the past,” David Mytton, CEO of Arcjet, told The New Stack. “GLM-5.2’s capability indicates that usage of open-source models is about to explode. “This all seems obvious in retrospect: models became more capable at the start of the year, agents started taking real actions (particularly since most of the work happens following the chat prompt), and legal restrictions on using frontier models are causing people to look elsewhere. This will cause all sorts of security issues because managing so many model capabilities will become difficult.”

Paul Sawers, contributing writer at The New Stack, tells us for this countdown: “Budget open-weight model panels are now matching frontier proprietary benchmarks at a fraction of the cost, undercutting the case for paying top dollar for a single closed model. Example of startup ditching Anthropic for DeepSeek.”

Indeed, the European AI agent startup Lindy AI migrated 100% of its production traffic from Anthropic to DeepSeek, citing millions of dollars in savings,

2. CEOs vibe coding their own tools

While software developers seem to have wholly adopted AI coding assistants, the business side of organizations — the C-suite and other executives high up in the chain of command — is adopting these tools to “vibe code” a variety of agents and productivity applications.

The trend runs from simple workflow automations to full production systems serving hundreds of users. The tools are Claude, Cursor, and, increasingly, the AI features embedded in the platforms these executives already run. The motivations range from impatience with IT queues to genuine curiosity about what the technology can do. And the results are more varied than the enthusiasm surrounding them might suggest.

Woodson Martin, CEO of OutSystems, took a more structured approach to his own vibe coding experiment. He built a personal mobile app wrapper on top of MCP services his team had created — and he built it twice in parallel, once using OutSystems’ own AI coding tool, Mentor, and once using Claude, connecting to the same backend both times.

“I was tired of explaining it to somebody who was supposed to build it for me,” Martin told The New Stack in April. “I was just like, ‘I’ll do this myself.’”

The app is a personal chief-of-staff system that consolidates customer account intelligence — buying signals, website activity, internal data — into a pre-meeting briefing he can pull up on his phone. It replaced what had been a 45-minute PowerPoint session plus multiple prep meetings from his sales team. 

1. The government cracks down (and later relents) on Anthropic Fable 5 and Mythos 5

The Fable/Mythos takedown illustrated how unpredictable AI policy has become. Anthropic launched Fable 5, its only Mythos-tier model, on Jun 9, 2026, with the fuller Mythos 5 reserved for a small set of trusted customers under what the company called Project Glasswing. The rollout lasted only three days.

On June 12, Commerce Secretary Howard Lutnick sent Anthropic CEO Dario Amodei a directive ordering the immediate worldwide suspension of both models for all foreign nationals, including Anthropic’s own non-citizen employees.

Apparently, the trigger was a jailbreak that Amazon researchers found, which could expose the models’ cybersecurity capabilities. This raised concerns at Commerce. Anthropic said it didn’t have a way to restrict access by nationality in real time, so it disabled Fable 5 and Mythos 5 globally. The Commerce Department partially opened Mythos 5 to select government-approved organizations in the following weeks, but the freeze didn’t lift until June 30 — after an 18-day period of limbo.

Meanwhile, Anthropic added extra cybersecurity safeguards to Fable 5 and began restoring global access on July 1.

Having mentioned Anthropic’s run-in with the Pentagon, Frederic Lardinois of The New Stack tells us: “While the two situations are not directly linked, it’s hard not to read the Fable controls as an extension of this existing animosity between Anthropic and the Trump administration — and in part, this seems personal as well.”

What’s next

What’s in store for the second half of 2026? We’ll be tracking how much AI-generated code actually makes it into production — and the tools designed to close that gap; the expanding autonomy of agentic AI and the guardrails keeping pace with it; the fast-moving regulatory environment around the frontier labs; and enterprise adoption of open-weight models.

We’re also watching AI’s spread among knowledge workers, the ever-longer stretches agents run unsupervised, and the hiring and performance of forward deployed engineers.

Whatever happens this year, The New Stack will cover it. And if you haven’t already, subscribe to The New Stack Daily newsletter for timely, thoughtful updates on how AI is reshaping software development.

The post 10 moments that defined AI’s turbulent first half of 2026 appeared first on The New Stack.

OpenClaw finally dropped its iOS and Android apps this week, meaning you can now ditch the Telegram and WhatsApp methods to talk directly to your personal AI agent. But what’s arguably more exciting is that the app isn’t actually running the AI on your phone. It’s just hooking up to an agent you’ve already got running somewhere else. Your phone now acts as a window into that agent, complete with voice, notifications, and camera access.

It’s a nice design choice, and exactly where personal AI agents are headed.

Phones become authenticated endpoints

The phone is basically becoming a really smart remote control for OpenClaw. Instead of cramming an increasingly powerful agent onto a phone with battery and memory constraints, developers are treating the phone as one more screen for an agent that lives elsewhere. The agent keeps working whether your phone is in your hand or charging in the other room.

Within this model, the phone approves actions, pings you with notifications, lets you talk to the agent, and shares your camera when the agent needs eyes on something.

Persistent runtimes replace mobile constraints

But OpenClaw isn’t the first to do this. Anthropic’s Claude Cowork with Dispatch follows a remarkably similar pattern. Users assign work from their phones, but execution occurs on a persistent desktop runtime. The mobile app acts as a companion for starting tasks, monitoring progress, and receiving results rather than becoming the agent itself.

OpenAI is moving in a similar direction as well. With Codex, developers increasingly interact with long-running coding agents that continue working independently and can be checked on from multiple clients, instead of treating the phone as the place where the agent runs.

Different companies, different products, but a similar architectural bet to keep the agent running in a persistent runtime and give people lightweight clients to interact with it.

When multiple teams independently converge on the same architectural pattern, it’s often an early signal that the industry has found a model that solves a real engineering problem.

The engineering problems are totally different now

This shift changes what developers spend their time thinking about. Building mobile apps used to mean worrying about battery life, memory limits, offline mode, and squeezing the best performance out of a phone. If the agent is running somewhere else, most of those concerns fade into the background.

Now, a new set of questions comes to mind, such as how a phone securely connects to a long-running agent? How do you manage permissions across multiple devices? What happens if every client disconnects but the agent keeps working?

Agent identity beyond login screens

There’s a downstream effect here. Once the phone is just one of several trusted endpoints talking to your agent, you need a much more robust approach to identity. You’re not logging a user into an app anymore. You’re authenticating devices into an ongoing relationship with a persistent agent.

As that agent gains the ability to read your files, send emails, call APIs, and control external tools, authentication becomes load-bearing infrastructure.

Distributed agents reshape developer tooling

Zooming out a bit and looking at the bigger picture highlights how personal AI agents increasingly resemble distributed systems rather than mobile apps. The intelligence lives in a persistent runtime while the phone is one authenticated endpoint among several.

For developers, the mobile app is only part of the job. They also have to build the components that keep an agent running, connect it to a user’s devices, and ensure those connections remain secure.

The agent keeps running independently, while the phone is simply another place to check in, approve actions, or start a conversation.

Looking at OpenClaw alongside Anthropic and OpenAI, it’s hard not to notice the same pattern. The agent keeps running independently, while the phone is simply another place to check in, approve actions or start a conversation. That architecture solves many practical problems, which may explain why several companies are heading in the same direction.

The post OpenClaw’s new app doesn’t run AI on your phone. That’s the whole point. appeared first on The New Stack.

AI has changed the web right before our eyes. With Google’s AI Overviews doing the heavy lifting, publications that once owned the first page of search results are being replaced by summaries. Readers get their answer without ever clicking through. Much of the traffic has simply stopped.

Cloudflare on Wednesday announced a slew of updates for publishers who are facing this new reality. From new crawler classifications and analytics dashboards to Answer Engine Optimization tools and an expansion of its Pay Per Crawl program, it’s clear the company is trying to become the economic pipes of the AI web.

The shift from ‘keep out’ to ‘let’s make a deal’

A year ago, Cloudflare’s pitch was practically defensive, asserting that website owners should be able to block AI crawlers. And while that still holds true, the company has pivoted to discussing building “rails” for an “agentic economy.” And it makes sense. If AI agents are already browsing the web, collecting content, and in some cases buying things, someone needs to handle the business end of how the sites they visit are compensated. Cloudflare thinks that someone should be Cloudflare.

Paying for value, not visits

Roughly a year ago, Cloudflare launched Pay Per Crawl, which let publishers set a price for AI companies to pay when they fetched a page. Now the company is pushing toward Pay Per Use, which means publishers get paid when their content actually appears in an AI-generated answer.

To backtrack, under the old model, an AI crawler pays to visit your site, whether or not it does anything useful with what it finds. Cloudflare says it’s already testing this with Ceramic.ai and You.com, each running slightly different versions of the concept.

Instead of charging for access — which is basically a toll booth — publishers are charging for value.

The economics here flip. Instead of charging for access — which is basically a toll booth — publishers are charging for value. That’s closer to how affiliate marketing or licensing deals work, and it’s a much harder problem to solve. It requires knowing which content contributed to which answer, which means an attribution infrastructure that doesn’t really exist at scale yet.

Crawlers need clearer labels

But here’s where things get more technical — and a bit political.

Cloudflare wants AI companies to stop lumping all their crawlers together. Right now, a single bot from a major AI company might be fetching pages for search indexing, model training, and agent tasks all at once. That makes it impossible for site owners to say yes to one use and no to another.

Starting September 15, Cloudflare plans to change the defaults for new and free-tier sites. AI search crawling stays on, but training and agent access get blocked on ad-supported pages unless the site owner opts in. Mixed-use crawlers that refuse to separate their traffic get blocked entirely.

The company is clearly taking a shot at Google here, noting that Google’s bundled approach gives it access to roughly twice as much content as AI-native competitors. Separating crawlers’ intent has become a prerequisite for any kind of functioning market between publishers and AI companies. Because if site owners can’t distinguish between “index my page for search” and “train your model on my writing,” they’ll increasingly just block everything.

Separating crawlers’ intent has become a prerequisite for any kind of functioning market between publishers and AI companies.

Optimizing for AI answers

Cloudflare is also rolling out a dashboard designed for business teams, called Attribution Business Insights. Think of it as the AI equivalent of knowing your Google Search Console numbers, except the “search engine” is now ChatGPT or Perplexity or whatever agent your reader happened to ask.

On top of that, Cloudflare is introducing Answer Engine Optimization (AEO). The idea is that ranking in Google is no longer enough; for publishers to succeed, they also need to understand how and where their content gets cited in AI-generated responses. That’s a different optimization problem than SEO, and right now almost nobody has good tooling for it.

Infrastructure as competitive advantage

Cloudflare already sits between websites and the internet. It sees the traffic and knows what changed on a page and what didn’t. It can tell a crawler to come back later because nothing’s new, which, by the way, it says would eliminate over 50% of current AI crawl traffic.

That puts Cloudflare in a unique position. It’s already part of the path between AI companies and the web. Now it wants to become the layer that manages access, attribution, and eventually payments between them. If AI agents become the primary way people find information online, Cloudflare is betting the next battle won’t be over who builds the smartest model but instead over who builds the infrastructure everyone else relies on.

If AI agents become the primary way people find information online, Cloudflare is betting the next battle won’t be over who builds the smartest model — it’ll be over who builds the infrastructure everyone else relies on.

The post Cloudflare wants to build the economic layer of the AI web appeared first on The New Stack.

Workday, the payroll and HR data platform, has been pursuing AI and agents for a while, but while other businesses may allow a little room for error, getting a payroll run in Workday 99% right is not exactly good enough.

Gabe Monroy, Workday’s chief technology officer, tells The New Stack that enterprise AI has to clear this bar before anyone will let it near their HR and finance data.

“There aren’t many systems that are more critical — or less forgiving — than ones that are dealing with people and money,” he says. There’s no tolerance for “well, it works most of the time,” Monroy says in an interview.

At its DevCon developer conference in early June, Workday laid out its plans to clear this correctness bar. The company introduced Agent-Ready Tools, a set of connectors that let agents act across the platform over the Model Context Protocol (MCP), a Developer Agent that lets people build apps and agents on Workday in plain language, and Agent Passport, which tests and verifies agents before they go into production and keeps monitoring them after, with Cisco as the first attestation partner.

Guardrails belong in the inference engine

Monroy spent most of his career in infrastructure and the developer space: at Deis, Microsoft, DigitalOcean, and, most recently, Google. At Google, he focused on building infrastructure for large AI labs to run inference at scale. For someone so focused on infrastructure, coming to Workday might seem a bit like an odd move, but Monroy argues that, at this point, LLM safety is — or at least should be — part of the core infrastructure for enterprises.

“The stakes are higher … in the world of people and money”

“The stakes are higher in the world of Workday and in the world of people and money, and that’s something that I was really excited about tackling at Workday specifically — and I do look at it as a core infrastructure,” he says. “A lot of what I’ve been doing in my recent past has been building infrastructure for large AI labs to do inferencing at scale, and what you pick up pretty quickly is that inferencing is probabilistic.

“[Inferencing] involves prefill and decode, and a whole bunch of really technical machinery in place to stream tokens out to end users, but what is nowhere in that stack today is the concept of native LLM-level enforced guardrails — guardrails that are part of the core inference.”

In his view, making it safe for enterprises to operate inference at scale has to be done at the inference engine layer. Agent gateways and similar add-ons that wrap the model from the outside sit at the wrong layer, he argues.

For a system of record like Workday, the system must strictly enforce guardrails on who the user is, what their budget authority is, and where they sit in the org chart. Those constraints, Monroy says, can be wired deep into the inference rather than checked after the fact. “These are things that we have the opportunity to wire deeply into the inferences at a very, very low level, in a way that produces much safer outcomes,” he says.

Workday’s recent acquisition of Pipedream partially plays into this. With Pipedream, an agent can reach out to third-party systems outside Workday to, for example, pull a policy document from Google Drive, and the platform can then verify that this specific agent has all necessary access rights to do so.

“Bring it to our shop”

All of those agents need to be managed, of course, and while many SaaS companies are currently building out their own agent platforms and agent orchestration services, seemingly all offering the same services, Monroy argues that orchestration should happen close to the data sources.

“If you’re trying to run an agent interaction that is integrating with people and money, that orchestration loop should probably happen closer to Workday, ideally on the Workday,” he says. “I do think there is something to be said for inference engines having proximity to the underlying system, because when it comes to a low-level inference engine and runtime perspective, there are some differentiated things you can do at the inference engine level that are only possible due to proximity to.”

“I do think there is something to be said for inference engines having proximity to the underlying system…”

He likened it to car repair. If somebody hands you a toolbox, you may or may not be able to fix your car’s problem. “Maybe you’ll get the outcome, but I tell you what, if you really want your car repair done right, bring it to our shop. We got the hydraulic lift, we got the trained engineers who know how to do this stuff, and we got the tools — but our tools are the best in the industry.”

For the part of an agent’s orchestration loop that has to run next to HR and finance data, “you really should be running that in our shop.”

For more general-purpose workflows, though, he believes there is room for general-purpose platforms. Not everything needs to run on Workday, after all, and Workday does offer MCP servers that third-party tools can tap into to access its data and tools as well.

To some degree, every system-of-record vendor is making some version of the proximity-and-context case right now. Owning the context for AI agents to work with is a major moat for these companies, after all, even as APIs and MCP servers allow agents to pull in data across vendors.

Meanwhile, there is also a group of other companies positioning themselves as neutral parties benefiting from the fact that these other vendors are now opening their platforms to third-party agents. It seems unlikely that, in the long run, enterprises will want to manage multiple agent platforms, but at this point, it is still far from clear in which direction this pendulum will swing.

Happy to lose the tooling fight

If proximity is where Workday thinks it can win, there is one area where the company doesn’t want to compete: developer tools.

“We’re not going to differentiate Workday on building better tools for developers,” Monroy says. “We’re going to differentiate on safety, on trust, on the inference engine.” It’s telling that the demos at DevCon ran on Claude Code, Cursor, OpenAI’s Codex, and Google’s Antigravity rather than on anything Workday-branded. “On the tooling front, I’m happy to have Claude Code and Codex and others,” he says.

It’s no secret that developer loyalty to AI tools doesn’t really exist at this point. Developers definitely don’t want a one-off tool from a single vendor, and Monroy notes that he isn’t going to ask them to adopt one. Instead, Workday wants to own skills, which he calls “the underlying lingua franca that we’re using to converge across the system.”

Workday is happy to be the system of record for its vertical and to let others fight over the tools.

The post “Bring it to our shop”: Workday’s pitch for keeping AI agents close to your most valuable data appeared first on The New Stack.

Okta has made its AI agent governance platform generally available for FedRAMP- and HIPAA-regulated environments, becoming what it claims is the first independent identity platform to extend AI agent lifecycle management inside the compliance boundaries federal agencies and healthcare organizations already trust.

The product, Okta for AI Agents – Core, elevates AI agents to first-class identities managed alongside human and machine workforces. This is a shift from the practice of treating agents as static service accounts or hardcoded API keys. The launch comes as federal agencies face mounting pressure from the recent executive order on AI innovation and security, which directs agencies to deploy AI agents and mandates that they secure them.

“The message to agencies is clear: Adopt AI aggressively, but secure it as you go,” writes Amy Johanek, Okta’s VP of Federal, in a blog post. “That puts identity at the center of the mission.”

“The fastest-growing class of NHI yet, and the hardest to see.”

Johanek also writes that AI agents are “the fastest-growing class of NHI [non-human identity] yet, and the hardest to see.” Anyone can spin one up, agents can spawn additional agents, and each connects across apps, APIs, SaaS tools, MCP servers, and data systems with little visibility, she says.

For organizations under mandates to harden systems and defend against AI-enabled criminal access, an unmanaged agent is not just an operational gap; it is more like an unguarded door, the company says.

“An unmanaged agent is not just an operational gap; it is more like an unguarded door.”

Johanek laid out four specific risks facing agencies running ungoverned agents: compliance violations when agents touch data outside authorized boundaries; compounding breach risk, where a single compromised credential doesn’t grant access to one system but to everything an agent can reach before a human can intervene; failed audits when agents run as orphaned accounts with no owner or evidence trail; and stalled AI adoption when delay becomes the only compliant option.

Moreover, the platform is organized around three governance questions: Where agents operate, what resources they can access, and what actions they’re authorized to take. Agents are registered in Okta’s Universal Directory inside an organization’s regulated cell, each assigned a unique identity and a named human owner, Johanek says. Every agent becomes a known, owned, first-class identity inside the environment, whether it came from a third-party platform or the organization’s own developers.

The platform replaces static credentials with scoped, short-lived tokens enforced at runtime. Least privilege is applied across authorization servers, third-party applications, and MCP servers. The governance layer mirrors existing federal workforce identity controls: access certifications, entitlement reviews, time-bound permissions, and a full audit logging stream that can be streamed to SIEM platforms for U.S. Government Accountability Office reporting requirements, Johanek says.

The offering also provides a kill switch

The offering also provides a kill switch. When an agent deviates from its intended mission or unexpectedly accesses sensitive data, security teams have a real-time mechanism to contain the risk before it escalates into a larger incident.

Johanek says she sees the offering as continuity rather than new infrastructure.  Agencies already trust Okta to manage human identities. Okta Identity Governance achieved FedRAMP High authorization earlier this year; bringing agents into that same identity fabric, she writes, is the natural next step, not a parallel system to build and defend.

However, there is one caveat: Okta for AI Agents – Core is not authorized in Okta for US Military cells.

The post Okta is the first to bring AI agent governance inside FedRAMP boundaries appeared first on The New Stack.

Over the past few months, four AI giants quietly rebuilt the same thing at once: AWS, Microsoft, Google, and Anthropic each shipped agent runtime updates that point to the same architectural shift.

Microsoft rebuilt its Foundry-hosted agents in April around per-session isolation. Anthropic’s Managed Agents virtualize the agent into a session, a harness, and a sandbox. AWS routes each session to its own microVM, and Google isolates agent code execution in a dedicated sandbox. The common pattern is not another model feature or developer tool. It is a move from request-level load balancing to session-aware execution.

This matters because enterprise agents are not ordinary API calls. They are long-running, stateful, tool-using processes that often run code influenced by user input. At its core, the agent runtime is becoming a control plane for state, identity, isolation, and lifecycle, and the unit it schedules has moved from the individual request to the agent session. The convergence across four platforms suggests session-aware execution is becoming a baseline requirement for production-grade agents.

The traditional cloud scaling model

To appreciate why this matters, it helps to start with what the traditional model got right. NGINX and HAProxy tiers typically sit in front of a pool of workers and route each incoming request to the next available backend. The state is deliberately kept elsewhere, in Redis or a database, so any worker can serve any request. That externalization of state is what enables elastic scaling, fault tolerance, and infrastructure replacement without disrupting the application. Sticky sessions were available, but enterprises treated them as an exception for stateful applications rather than the default.

The model rests on two assumptions: that requests do not depend on one another and that any backend can serve any request. For web APIs and microservices, those assumptions held for nearly two decades and enabled the scaling of the largest systems ever built. Agents challenge both at once, which is why conventional load balancing alone is no longer sufficient.

Why agents break the model

Agents pose two distinct challenges: preserving conversational state across turns and enforcing a security boundary strong enough to withstand untrusted code. Only the first can be addressed by routing alone.

Why can the agent state not be pooled?

Imagine an enterprise support agent processing a refund. It reads the order, calls a tool, and then waits for the model. The next turn asks a clarifying question. If that turn is routed to a different replica without access to the prior context, the agent loses the state it needs to finish the workflow.

The load balancer starts to behave less like a traffic distributor and more like a session router.

The industry already has a name for the workaround, session affinity, and it already concedes the cost. Load-balancing approaches increasingly rely on consistent hashing, using a conversation identifier to keep related steps in a warm state. In plain English, the load balancer starts to behave less like a traffic distributor and more like a session router.

Why the trust boundary forces isolation

The second challenge is harder because it turns on isolation rather than on the state alone. An agent can execute model-generated code shaped by user input, so the backend must be treated as a security boundary rather than a generic compute target. A shared kernel does not give untrusted, agent-generated code the tenant isolation that enterprise security teams require. Session affinity can preserve routing continuity, but it does not by itself provide per-tenant isolation for untrusted execution. For platform and security teams, that is the line where a tuned load balancer is no longer enough.

This is not a theoretical concern. When Asana disclosed a flaw in its MCP server in June 2025, the server had been live since May 1, a window of about five weeks. A tenant-isolation check failed, and around 1,000 organizations could see project data belonging to other customers. Based on the reported details, security researchers traced the issue to a server that validated the user but did not consistently enforce the agent and tenant context behind cached responses. No external attacker was involved, and data still crossed organizational boundaries. The incident shows why user identity, agent context, and session state must be consistently bound together.

The session as a unit of compute

Once that binding becomes a correctness and security requirement, the session becomes the unit that the platform schedules. The clearest evidence is in the lifecycle. A session now has a state it never had before. AWS documents that an AgentCore session remains Active while it processes work and sits Idle when it is provisioned but waiting. It reaches Terminated after a 15-minute idle timeout or an 8-hour maximum lifetime. That is the lifecycle of a long-running execution environment, not of a short-lived HTTP request.

The economics follow the same logic. When billing is tied to active sessions, concurrency, idle time, and agent sizing become the cost drivers rather than request volume. That puts a new line item in front of platform teams and finance. The better mental model is no longer a traditional load balancer. It is closer to a virtual actor runtime, where an addressable identity is instantiated on demand, kept active while it is needed, and deactivated when idle, with one live instance per key.

Four approaches to the same shift

All four platforms are moving away from treating stateful, untrusted agent work as ordinary load-balanced traffic. The key difference is the compute primitive each provider chooses for the isolated execution environment.

AWS AgentCore is the most opinionated of the four. Each session gets a dedicated Firecracker microVM with isolated compute, memory, and filesystem, and a request carrying the same runtimeSessionId is routed back to that microVM through a session header. When the session ends, the microVM is terminated and its memory sanitized.

The agents hosted on Microsoft Foundry implement a similar model through a different set of primitives. The platform creates a per-session VM-isolated sandbox on demand, runs it, and tears it down at session end, with no replica count and no warm pool to size. Each agent gets a dedicated Microsoft Entra identity, and sessions run a 15-minute idle timeout against a 30-day maximum lifetime.

Google Agent Engine is the most instructive hybrid. Its runtime keeps request scaling within the reasoning loop, a managed environment with configurable minimum and maximum instance counts, and a container_concurrency that defaults to 9. Even where Google retains request scaling for the loop, it separates untrusted code execution into an isolated Code Execution sandbox and externalizes conversation state to Sessions and Memory Bank. The provider that kept the load balancer in the loop still would not point it at stateful untrusted work.

Anthropic provides the clearest decomposition of the architecture. Managed Agents virtualize the agent into three components: a session that logs everything that happens, a harness that runs the loop and routes tool calls, and a sandbox where code runs. The harness becomes a near-stateless control plane, while the sandbox becomes a callable, rebuildable resource. The Cloudflare integration shows that the substrate can be decoupled. The agent loop runs on Anthropic while each tool call runs in a Cloudflare sandbox, which can be a full microVM or a lighter V8 isolate.

The four platforms converge on the routing and lifecycle model and diverge on the execution substrate, as the table shows.

Platform	Compute primitive per session	What it means for an enterprise buyer
AWS AgentCore	Dedicated Firecracker microVM, routed by session ID, 8-hour ceiling	The most opinionated answer, with isolation and routing fused at the microVM
Azure Foundry hosted agents	Per-session VM-isolated sandbox, dedicated Entra identity, 30-day max	Identity-led isolation suited to longer multi-day workloads
Google Agent Engine	Request-scaled loop plus a separate isolated code-execution sandbox	A hybrid that load-balances the reasoning and isolates the execution
Anthropic Managed Agents	Decoupled harness and sandbox, microVM or V8 isolate behind the loop	The most portable, with the substrate treated as a swappable layer

No single architecture is the right answer for every enterprise workload. A multi-day research agent fits Azure’s longer-lived session, a code-heavy agent needs microVM-grade isolation, and high-volume automation favors a lighter isolate. Many enterprise platforms will end up combining more than one of these patterns.

The binding the application still owns

The platform can isolate and route a session, but it does not know which human owns it. AgentCore explicitly states that it does not enforce session-to-user mappings, so the application backend must maintain the relationship between users and their session IDs and set per-user session limits. This is why the Asana incident is relevant to enterprise architecture.

Isolation was maintained while the user-to-session binding broke.

In that case, isolation was maintained while the user-to-session binding broke. The platform solves isolation and lifecycle, and it hands back identity mapping, authorization, and tenant context to the application. For an enterprise buyer, the key question is who owns that binding and how it is tested under concurrent, multi-tenant load.

How this differs from sticky sessions

A reasonable objection is that this appears to be sticky sessions combined with autoscaling on a microVM substrate. The distinction is worth drawing out. Traditional sticky sessions are usually performance optimizations, whereas in an agent runtime, session binding becomes a correctness and security requirement. Traditional load balancers route traffic to existing backends and do not own the lifecycle of the execution environment.

The new control plane provisions an environment when it first sees a session key, routes work to it, and tears it down after idle or lifetime limits. The primitive underneath is genuinely old, and its age is what makes the move legible. AWS open-sourced Firecracker at re:Invent in 2018 to pair VM-grade isolation with container speed. I covered it for The New Stack then, and ran more than a hundred microVMs on a single laptop to see that trade firsthand. The same microVM now powers Lambda and Fargate across trillions of executions a month. This is how cloud architecture tends to evolve, with existing primitives recombined around a new unit of abstraction.

The agent session is becoming the operational unit of enterprise AI infrastructure.

In summary, the important shift is not that cloud providers have found a better load balancer for agents. It is that the agent session is becoming the operational unit of enterprise AI infrastructure. Traditional load balancing routed independent requests to interchangeable workers. A session-aware runtime routes work to an isolated environment, preserves state across turns, and owns the lifecycle of the execution context. The dispatch layer did not disappear when agents arrived. It became session-aware, stateful, and lifecycle-aware, making it a control plane rather than a conventional load balancer.

Three practical questions will shape what comes next, and enterprises will need to evaluate each. The economics of per-session billing will decide how costs scale with concurrency. The open-source effort to rebuild this routing model on Kubernetes will determine its portability. The argument between microVMs and lighter isolates will decide the trade-off between isolation and density. For enterprises building agents, a runtime that owns isolation, routing, and lifecycle turns background plumbing into a deliberate architectural choice. It gives platform, security, and operations teams a foundation they can standardize on rather than rebuild for every workload.

The post AWS, Microsoft, and Google agree the session is the new unit of compute. They disagree on how to isolate it. appeared first on The New Stack.

Many agentic projects can sail through development just fine. Then they hit security review — and that’s where things can grind to a halt. Unclear identity models and overly broad permissions quickly become blockers.

You’ve probably seen this play out: A customer support agent is working well; it triages tickets and processes refunds, handles the whole workflow without a hitch. Then security asks a simple question: Under whose identity is this running? The answer stops the process cold: It’s a shared account with broad permission, no clear ownership, no audit trail, and no least-privilege controls in sight.

The root issue isn’t complicated. It’s undefined identity and poorly scoped permissions. And that challenge is accelerating fast. Research from the 2026 Tech Leader Study, conducted with Oxford Economics and IBM, shows surveyed enterprises expect to deploy an average of 1,661 AI agents, a 38% increase from today. Each new agent introduces another identity to secure, and without clear boundaries, the problem compounds quickly. 

As a result, many agentic systems focus on what agents can do without defining what they should do, or under whose authority. Agents also don’t hold a fixed set of permissions. They request access, call new tools, and assume roles as they work, so access paths compound in ways no one explicitly granted or reviewed. Without a verifiable identity, there’s no accountability, making least-privilege enforcement, traceability and incident response difficult.

“Each new agent introduces another identity to secure, and without clear boundaries, the problem compounds quickly.”

To address these gaps, this guide is written for developers, architects and DevOps engineers building agentic systems — and for the IT leaders responsible for approving them.

The four identity decisions every agentic system must make

Identity decisions can’t be treated as an afterthought. Identity shapes how agents authenticate, what they can access, and how their actions are controlled and audited over time. Get it wrong early, and you’re building on a shaky foundation.

Here are the four decisions that matter most:

Workload identity vs. shared service accounts 

Shared service accounts are easy, and that’s exactly what makes them dangerous. When multiple agents act under a single identity, it becomes hard to tell what happened or what went wrong after the fact. If an account is leaked or misused, everything it touched is exposed. 

“Shared service accounts are easy, and that’s exactly what makes them dangerous.”

Workload identity assigns each agent its own identity. Permissions stay scoped, and actions are attributable. It requires more setup but creates isolation and auditability.

Static API keys vs. short-lived credentials

Static API keys tend to stick around forever. They get hardcoded into apps, passed around between systems, and rarely rotated — which makes them a persistent vulnerability waiting to be exploited. 

Short-lived credentials work differently. They’re issued on demand, scoped to a specific task, and expire automatically. In practice, this often relies on identity federation (for example, using OIDC tokens) combined with systems that can issue dynamic credentials at runtime, rather than storing long-lived secrets in code or configuration

Direct credential handoff vs. brokered session access

Handing credentials directly to an agent is simple. It’s also opaque. You don’t have a natural point to evaluate policy or understand what’s happening in real time. 

Brokered access introduces a control point into the flow. Requests go through a broker, policies are evaluated in real time, and temporary credentials are issued per session. It adds infrastructure, but restores visibility and policy enforcement. 

Fragmented logging vs. full identity lineage

Most systems log what happened. Far fewer capture who initiated it or how an action propagated through a chain of agents and services. 

Full identity lineage connects every step. You can trace an operation from triggers to outcomes, which can make debugging faster and enable more credible incident response. The catch is that this requires consistent identity propagation and structured logging from the beginning—it’s hard to retrofit.

When these tradeoffs become real risks

These aren’t abstract architectural preferences. They show up as concrete vulnerabilities.  

Nightfall AI reports that organizations expose nearly 350 secrets per 100 employees each year, with 35% of exposed API keys still active. Combine that with persistent credentials and shared identities, and the potential blast radius grows fast. 

The pattern is consistent: shared accounts and long-lived keys are faster to build but harder to secure. Workload identity and short-lived credentials require more upfront investment but can deliver more security over time.

Debugging breaches by feel

Think about what happens when an agent running on a shared account with a long-lived key suddenly spikes its data access. Was it a bug? A breach? Routine behavior? Hard to say. Revoking the key might stop the issue, but it could also break a half-dozen unrelated workflows in the process. You’re now debugging by feel. 

Shortcuts reduce friction at the start and accumulate risk over time. 

Standardize identity at the platform layer

The answer isn’t to rebuild authentication, authorization and auditing from scratch for every agent you ship. That’s not scalable. 

Instead, standardize identity at the platform layer—centralized identity providers, policy engines and a credential broker to enforce secure defaults and make compliance straightforward rather than a constant negotiation. 

“Shortcuts reduce friction at the start and accumulate risk over time.”

Agentic AI works in production when identity is designed up front and enforced at runtime, rather than assumed from a prior login. When projects are treated as an afterthought, they stall. When it’s built in deliberately, agents can operate with the control that production environments demand. 

IBM supports this through an integrated identity-first approach that spans secrets management, secured access, and identity governance—helping organizations scale agentic systems securely without adding operational complexity. 

Learn how IBM approaches identity-first security for agentic systems

© Copyright IBM Corporation 2026. IBM and the IBM logo are trademarks of IBM Corp., registered in many jurisdictions worldwide. Examples presented are illustrative only. Actual results will vary based on client configurations and conditions; therefore, general expected results cannot be provided.

The post The AI agent identity problem nobody’s talking about appeared first on The New Stack.

Organizations are racing to apply AI across the enterprise, and infrastructure is one of the most compelling targets: automated provisioning, self-healing networks, and agents that deploy and manage servers without human intervention. The promise is real, but so is the risk. 

No matter the domain, AI agents are only as good as the data they’re given. Agents without a complete and accurate picture of the network and associated infrastructure will make confident mistakes. In infrastructure, those mistakes have brand and revenue-related consequences: exposed databases with PII, failed deployments, and outages that take the entire business offline. 

“Agents without a complete and accurate picture of the network and associated infrastructure will make confident mistakes.”

Most enterprise infrastructure is managed through a patchwork of siloed, fragmented tools: separate systems for IP address management, data center inventory, and device configuration. The list goes on.

Each system captures a slice of the picture, but none of them complete the full vision. HyperFRAME found that over 70% of industry leaders identified this as a core bottleneck. Agentic automation cannot solve this issue, but it will expose it through its failures.

Before you can trust an AI agent with your infrastructure, you need to give it something to trust: a single, unified model of what’s on your network, how it’s configured, and how it’s supposed to behave. According to NetBox Labs CEO and cofounder Kris Beevers, that’s an Infrastructure Intelligence platform.

What is infrastructure intelligence?

Whether run by AI or human agents, infrastructure is impossible to manage when critical systems contain unknowns. Infrastructure intelligence is the foundational blueprint of your infrastructure: a unified, continuously updated model that captures not just what exists, but what is intended, what has changed, and what needs attention. It is the prerequisite for automation at any scale.

“AI is raising the stakes for infrastructure management, and the challenge is no longer just documenting infrastructure; it’s also understanding it…”

“AI is raising the stakes for infrastructure management, and the challenge is no longer just documenting infrastructure; it’s also understanding it,” says Beevers. “A source of truth was enough for the last decade. But today, teams need context – a trusted, continuously updated understanding of infrastructure that helps them (and their AI agents) model, see, act, and govern with confidence. AI doesn’t eliminate the need for infrastructure data. It makes it more important than ever.” 

It starts with a system of record. More than just an inventory list: it is a living representation of the intended state (what everything is supposed to look like) and the operational state (what it actually looks like right now) of your network. The gap between these two states is drift, and that is where risk lives. Without a system that tracks both states simultaneously, your team is always reacting, chasing down misconfigurations, and manually reconciling tool outputs (hoping nothing critical slips through).

Full infrastructure context connects the intent and design to the operational state, providing drift detection, observability, and lifecycle management tools in a single continuous data thread. Instead of switching between five different tools to answer a single question about a specific device, your team and your agents have all the information they need in one place. What is the device supposed to be doing? What is it actually doing? When did it change, and who changed it? Full context means that these questions have immediate answers.

Guardrails close the loop. Both humans and AI agents can make well-intentioned errors, and in infrastructure, the blast radius of these errors can be severe. For this reason, your infrastructure must have well-defined audit trails, branching workflows, change management processes, and operational validation from the beginning, not as an afterthought after something goes wrong. 

Teams move from handholding every agent action to trusting the system to catch bad outcomes. Cautious early adoption quickly grows into confident, autonomous, scaled deployments.

The foundation for any automation journey

Agentic automation/Agentic NetOps is coming to infrastructure teams, whether they are ready or not.

No matter where a company is in its automation journey, Infrastructure Intelligence provides a strong foundation for everything else. Organizations that are early in their automation strategy have manual processes they want to automate — they need a clear picture of the environment to do this safely. Teams that are running agentic workflows across complex, multi-site networks share the same requirement: Infrastructure intelligence.

NetBox Labs, the commercial steward of the open-source NetBox, recently expanded its platform to ensure that every infrastructure management workflow can be addressed by agents. The announcements make infrastructure AI Agent-Native: Extending the NetBox MCP server across the entire NetBox Labs Platform and releasing an array of pre-built agent skills.

These agentic tools are designed to leverage the existing infrastructure intelligence from NetBox Labs’ systems, ensuring that all agentic network provisioning capabilities are combined with the required guardrails, validations, and protections to keep the network running smoothly. 

Adding agentic features across the entire NetBox Labs infrastructure intelligence platform gives agents unprecedented knowledge, skills, and power. Agents can access NetBox Data Exchange — the world’s largest database of infrastructure metadata. NetBox Assurance and Discovery helps teams and their agents identify and mitigate drift.

“Giving AI agents access to production infrastructure without guardrails is a recipe for outages.”

According to NetBox CEO and cofounder Kris Beevers, “The future isn’t just autonomous infrastructure. It’s a trustworthy infrastructure. We know that trust and governance are the foundation of AI-driven operations. Giving AI agents access to production infrastructure without guardrails is a recipe for outages. That’s why we’ve paired these AI agent native updates with new validation tools so teams can ask, “‘Is this change safe to deploy?’ and ‘What breaks if this fails?’”

The new validation tools help agents self-correct, validate changes, and meet compliance requirements, ensuring continuous compliance and pre-change safety within the System of Record.

AIOps teams that establish a foundation of infrastructure intelligence gain more than efficiency, visibility, and control. They gain the confidence to automate services in production without losing sleep over it. Agents stop guessing and operate from verified real-time data. Teams stop reacting and focus on building. And the organization does not see AI as a liability, but as a capability that can be expanded.

NetBox Lab’s new infrastructure intelligence platform is designed for both humans and agents, making it easier to manage infrastructure across every lifecycle stage — from design through end-of-life.

Whether you’re a NetBox open source user or NetBox Labs customer, you can celebrate NetBox turning 10 at its inaugural conference, NetBox Evolve, which will be in Florida at the Kennedy Space Center on October 13, 2026.

The post Agentic infrastructure operations begin with accurate, reliable infrastructure data appeared first on The New Stack.

This week, Sakana AI released Fugu, a multi-agent orchestration system designed to deliver frontier-model performance all while reducing the risks of relying on a single provider. 

The Japanese AI R&D company says Fugu performs as well as Anthropic’s Fable 5 and Mythos Preview on engineering, scientific, and reasoning benchmarks by breaking up tasks into subtasks and strategically routing them across a swappable pool of expert agents. But early reactions are mixed.

While Sakana positions Fugu’s “collective intelligence” as the blueprint for AI sovereignty, not all users report frontier-model-level performance. Others note fast burn rates and unnecessarily high prices. 

Many agree that, though interesting, Fugu likely won’t be the hero to AI sovereignty it hopes to be. 

Is this just another router? Not really. 

Sakana says Fugu’s internal routing logic is founded on its own research in learned model orchestration, specifically noting two papers, Trinity and the Conductor. 

Unlike multi-model routers, such as OpenRouter Fusion, that send a prompt to multiple models and then compare or combine the results, Fugu breaks down user prompts into subtasks and determines which subtask to send to which model. In this way, Fugu “dynamically orchestrates the world’s best models to tackle complex, multi-step tasks,” so Sakana says. 

From the outside, you just see what looks like one model, accessible via a single OpenAI-compatible API.

But what the AI company doesn’t tell you is how it decides which tasks get routed where; that information is proprietary. From the outside, you just see what looks like one model, accessible via a single OpenAI-compatible API.

“relying on a single company’s model for national infrastructure is a massive risk. As recent export controls have shown, access to top models can disappear overnight.”

Fugu doesn’t have to farm out every task, though. It’s a language model itself, specialized for model selection, delegation, verification, and synthesis internally, so it can also solve requests directly when its own response is sufficient.

A hero for AI sovereignty, it appears not

In an X post, Sakana CEO and co-founder David Ha writes, “relying on a single company’s model for national infrastructure is a massive risk. As recent export controls have shown, access to top models can disappear overnight.”

Human intelligence is fundamentally a collective intelligence. We solve complex problems by participating in a vast cultural network that builds upon ideas across generations.

I believe the strongest AI systems will become a collective intelligence, too.

Since we started Sakana… https://t.co/yulKqdei2c

— hardmaru (@hardmaru) June 22, 2026

That “massive risk” comment is likely a jab at what happened to Anthropic, when an export control directive forced the AI company to pull Fable 5 and Mythos 5 just three days after launch. 

See also: Fable 5 ban: 4 open models responded before Anthropic could restore access

Following this news, Sakana positions Fugu as the antidote to single-provider reliance. Because it relies on a pool of “entirely swappable agents,” the idea is that Fugu is less likely to leave users in a bind if one provider suddenly restricts access. It can simply route work to other models. 

The AI company considers this capability enough license to claim it’s “delivering the realistic, resilient blueprint required for AI sovereignty.” But some initial reactions call that hyperbolic: 

“This is just a highly advanced router/wrapper, not a fundamental leap like Mythos/Fable was,” argues one Redditor.

Though it’s likely not fair to call Fugu a simple multi-model router, its ultimate reliance on other models means it’s not the hero for AI sovereignty it aspires to be. After all, if more than one model provider restricts access at the same time, Fugu’s capabilities also take a hit. 

As another user writes on HackerNews: “As a developer outside the US I think it’s vital to have alternatives to OpenAI and Anthropic, but sadly this is not it,” calling out what they describe as the tool’s unfortunate price-to-burn-rate ratio, an “extremely slow” API, and poor quality in comparison to Fable:

“It’s nowhere remotely near usable as a day-to-day workhorse.”

Not all user reviews back up the benchmarks

Sakana points to coding, reasoning, science, and agent benchmarks to prove Fugu’s value, stating its tool consistently beats Gemini 3.1, Opus 4.8, and GPT 5.5.

It also highlights what it says is the success of its beta program, where almost 500 early users tested Fugu on lengthy, multi-step computational workflows.

In particular, it claims that one cybersecurity engineer confirmed Fugu successfully operated within parameters and avoided destructive actions, while other teams praised Fugu Ultra for besting GPT 5.5 in code review and maintaining an “unusually strong persona stability across long sessions.”

But moving from benchmarks and PR-ready examples to early community sentiment adds more color to the story. 

One user on HackerNews calls Fugu “quite strong” for a few agentic coding tasks, but notes they weren’t able to do many deep reviews before their quota ran out, adding: “For implementation I found it weaker, it made a few mistakes that I haven’t seen frontier models make in a long time.” 

A Redditor had a different experience. They, too, bemoan burn rate issues, but note: “It caught things Opus 4.8 ultra and codex 5.5xhigh clearly missed in a fairly large data ingestion / processing project.”

Some users question the price tag

Furu is generally available today in most regions (save the EU) in two tiers: a low-latency model that integrates with chatbots and tools like Codex for daily tasks and Fugu Ultra, the heavy-hitter that coordinates a deeper pool of experts for more complex, high-stakes tasks. (This is the one that’s supposed to rival Fable 5 and Mythos Preview.)

Subscription plans are available at $20, $100, and $200 monthly rates for both Fugu and Fugu Ultra. Pay-as-you-go pricing is also available, with Fugu billed at standard rates per underlying model, and Fugu Ultra running at $5 per million input tokens and $30 per million output tokens, with higher rates when context exceeds 272k.

Several early users on Reddit and HackerNews deem these price tags too high, especially when they’re experiencing what now feels like the soundtrack of new agent tools: burn rates that get away from you too fast. 

As one HackerNews user jabs: “I love when they put a black box in front of the other black boxes so I can get a questionably better black box for slower service and more money!” 

Is collective intelligence the future? 

On X, HA posits that large-scale, monolithic models have had their time in the sun and that solving more complex real-world challenges will require a different beast: collective intelligence. 

Moving forward, Sakana plans to incorporate new models in its agent pool, which could shore up that resilience Sakana is aiming for. But so far, users seem to question whether paying another company to sit between them and frontier models is really worth the spend.

The post Sakana Fugu is more than a router. But it’s not the blueprint for AI sovereignty, either. appeared first on The New Stack.

The Linux Foundation on Tuesday declared its intent to launch the Agent Name Service (ANS), an open standard that gives AI agents verifiable identities by tying them to the internet’s domain name system (DNS).

The idea behind the ANS has actually been around for a while. It began as a research paper published in May 2025 by the OWASP GenAI Security Project, written by a group of application-security researchers. Its authors include Ken Huang, the CEO of security consultancy DistributedApps.ai and a co-author of the widely cited OWASP Top 10 for LLM Applications that chronicles the top security risks related to LLMs, and Akram Sheriff, an AI security engineer at Cisco.

ANS is a bit of a redesign of the original idea, which has gone through a few iterations since it was published. The 2025 original described ANS as a “universal directory” — basically a central registry with naming borrowed from DNS. A second version, published as an individual draft at the Internet Engineering Task Force in April, takes this a step further and ties each agent instead to a real domain its operator already controls.

How it would work

The design essentially copies how websites already prove who they are today. An operator demonstrates control of a domain like example.com through ACME, the automated protocol behind Let’s Encrypt, and a registration authority issues the agent a pair of certificates. Every change to the agent’s status, from registration to renewal to revocation, is written to an append-only log. A client checking an agent can choose how much assurance it wants, from a basic certificate check to a tier that also consults the log.

It’s worth noting that the ANS system separates identity from discovery and hands the job of finding agents to other services built on top.

The DNS industry and AI agents

Discovery is actually handled by DNS-AID, a separate discovery standard the foundation took in on May 27. It lets agents publish their endpoints as DNS records so other agents can find them. DNS-AID was originally built by Infoblox, and GoDaddy, which is also involved in ANS, is among its backers.

Agent identity and discovery projects based on DNS aren’t limited to these two Linux Foundation projects, though. Including those two, there are now at least four similar proposals. There is DNSid, for example, a durable-identity scheme from the registry operator Identity Digital, and AID, a minimal discovery draft that came out of the developer community.

Vineeth Sai Narajala, a co-author of ANS now with OWASP, says in the announcement, “we didn’t need to reinvent the wheel, we needed to extend the foundational trust of the internet to a new generation of autonomous technology.”

Not reinventing the wheel also means basing this system on the registrars and certificate authorities that come with it and the trust hierarchy they built, which security researchers have long considered fragile.

Maybe it’s no surprise that many agent identity and discovery solutions are coming out of the domain industry. GoDaddy, after all, registers domains, Identity Digital operates top-level domains, and Infoblox, which backs ANS, sells DNS infrastructure. For all of them, DNS-linked agent identity and discovery extends a (profitable) business they already run.

What about A2A and co.?

As is so often the case, the Linux Foundation is playing host to several alternative systems. Google’s A2A protocol, for example, gives agents a signed “Agent Card” they can publish at a known web address, with an agent registry on its roadmap. Cisco’s AGNTCY ships an agent directory and its own cryptographic identity service. Outside the foundation, Microsoft’s Entra Agent ID and Okta for AI Agents, both generally available since the spring, treat an agent as an identity managed inside the corporate directory, with short-lived tokens that tie each action back to the person who authorized it.

And while Cisco is backing both ANS and AGNTCY, some names are missing here, including major players like Google, Anthropic, Microsoft, and Amazon. Given their outsized role in the agent ecosystem, it’ll be interesting to see if they’ll join in this effort or decide on their own standards (insert obligatory xkcd comic here).

The post Can DNS become the basis for AI agent identity? appeared first on The New Stack.