❌

Normal view

Received — 19 November 2026 ⏭ AI Infrastructure Archives - The New Stack

Where should AI workloads run? Probably next to the data.

Intersecting teal ropes pulled tight in an abstract web, evoking network tension and AI data gravity across environments.

AWS raised prices on EC2 Capacity Blocks, its reserved GPU capacity, by about 15% in January and about 20% in July. Picture a platform team watching a training job’s bill climb after copying a few hundred terabytes of sensor data across regions just to reach the GPUs, then realizing the data never had to move.

For AI workloads, the real question is whether the workload goes to the data or the data goes to the workload. Data gravity usually answers it.

Data gravity hits AI workloads harder than it ever hit analytics

Data gravity is the tendency of compute, applications, and services to be pulled toward large concentrations of data, because moving the data costs more than moving the logic that processes it. The larger a dataset grows, the more expensive every copy becomes, and the more everything that needs that data ends up living next to it.

Scheduled BI often tolerates periodic queries against sampled or aggregated data. Large training runs can require repeated reads of a much larger dataset, while agents retrieve fresh context throughout a workflow. When those reads cross regions or environments, transfer costs and latency add up. Each agent output also becomes new data that feeds the next round of retrieval, which compounds the pull over time.

A January 2026 analysis in BigDATAwire argued that AI workloads removed the cushion that let enterprises leave data gravity unmanaged for a decade. The year’s pricing moves make the same point from the cost side. Combined, the two increases add up to roughly 38% on the H200 instances that took both.

Moving data to the compute stops working at AI scale

Centralizing compute can require copying data into its environment before a job starts, and that transfer may carry charges.

For illustration, assume a transfer rate of $0.02 per GB and a 300 TB dataset (300,000 GB). The transfer alone would cost $6,000 per copy. That’s a hypothetical, not an AWS quote. Real charges depend on source, destination, route, service, and terms, so cross-region and cloud-to-premises movement needs explicit costing. Remote reads also add network latency, which matters when a workload has a tight response-time target.

“Transfer bills, duplicate pipelines and the engineering hours spent maintaining them belong in the placement calculation alongside compute charges.”

Two more costs rarely show up next to the GPU rate:

  • Interoperability overhead: cross-environment pipelines need their own translation layers when engines, query languages, connectors, and software development kits (SDKs) differ.
  • Lock-in: transfer charges and the engineering work of moving data and pipelines can make leaving an environment expensive.

Transfer bills, duplicate pipelines and the engineering hours spent maintaining them belong in the placement calculation alongside compute charges.

Moving compute to the data instead

A scheduling and placement layer can send a job to an environment that already holds its data, avoiding repeated full-dataset transfers. The job still needs compatible accelerators, enough memory, a supported runtime, and adequate storage throughput. Model weights, container images and checkpoints may also need to move.

Making that decision well means evaluating four inputs together, every time a job is scheduled:

InputWhat to evaluate
CostThe total cost of this job in each candidate environment, including compute, transfer, storage, and queueing, at current prices.
Data gravityHow much data the job needs, where it sits, and what it would cost to move.
ResidencyWhich jurisdictions the data and the job are allowed to touch.
PolicyWhich access rules, classifications and quality gates apply wherever the job lands.

A placement layer first excludes environments that violate residency or access rules, then compares feasible options by capacity, performance, and total cost. It dispatches the job with its runtime policies and revisits placement when those conditions change.

Where a specific workload belongs depends on the workload

There’s no universal rule. It’s a workload-by-workload decision.

Bursty training: public cloud can suit elastic demand, provided suitable GPU capacity is available, and transfer costs don’t erase the benefit.

Residency-restricted work: applicable law or contract determines the permitted jurisdiction and environment. An approved in-region cloud can qualify where an out-of-region on-premises cluster cannot. Large data volume adds an economic constraint, not a legal one.

Transaction-near inference: place execution close enough to the transaction to meet its latency target, whether in cloud, on-premises or at the edge.

Back to the platform team. Their training job reads a large dataset from an on-premises store, and policy excludes processing outside an approved region. A local cluster has the required GPU memory and runtime. An approved cloud region offers suitable GPUs but requires copying the dataset. So they compare completion time and total cost, including transfer, storage, and queueing. If local execution meets the deadline at lower total cost, the job runs there with the same access controls. If not, they assess the approved cloud option.

If neither environment meets policy and capacity requirements, the job is queued or rejected rather than sent to an unapproved region with restricted data. They record the choice and its assumptions so a price change, a growing dataset, or a missed deadline triggers a new review.

Formal policy on all this is uneven. In BARC’s State of Data and AI Governance study, which Acceldata sponsored, 56% of North American respondents report formal sovereignty policies for AI processing and storage, against 30% in Europe, the Middle East and Africa (EMEA). That runs against expectations, since Europe is known for strict data rules. EMEA respondents also name siloed data as a top governance obstacle more than twice as often as their North American peers, 35% versus 17%.

Hybrid architecture matters more than picking a cloud

Hybrid can be a lasting architecture, not merely a stop on the way to the cloud. Data-heavy, latency-sensitive, or sovereignty-bound work can stay on-premises, while the cloud handles demand where elasticity is worth the cost.

It needs common operational controls across both environments: identity, runtime compatibility, observability, and policy enforcement. An all-cloud estate incurs transfer and lock-in costs when data must leave a region or provider. An all-on-premises estate leaves bursty work waiting for fixed capacity.

“Hybrid can be a lasting architecture, not merely a stop on the way to the cloud.”

Job portability helps only where the target environment supports the required engines, dependencies, and hardware. Open table formats such as Apache Iceberg, Delta Lake, and Apache Hudi help here, because several engines can read the same tables in place. Moving the job doesn’t have to mean converting or copying the data first.

This is the problem Acceldata built its xLake platform around: keep data where it lives, in open formats, and bring governed compute to it across cloud and on-premises environments. 

Sovereign AI requires more than GPU power

The sovereign AI conversation usually starts with GPUs, but GPU compute alone is an incomplete architecture. Owning or reserving GPU capacity says nothing about whether the data feeding those GPUs, or the retrieval and orchestration layers around them, stays inside infrastructure the organization controls. Sovereignty is a property of the whole data and AI stack. Compute is one layer of it.

For a strict customer-controlled boundary, Acceldata’s May 2026 analysis of sovereign GPU infrastructure identifies four controls: 

  • Route tool calls through internally hosted endpoints so operational data, prompts, and workflow context never leave the organization’s network.
  • Retrieval infrastructure (embedding indexes and vector search) running on infrastructure the organization controls rather than a provider-managed vector service.
  • Orchestration of prompts, memory, and intermediate agent state inside the organization’s own environment.
  • Access policies for permissions and retrieval scope enforced within that same governed boundary.

“A workload can run on the most expensive reserved GPUs available and still fail a sovereignty review if its prompts, embeddings, or agent state cross a boundary they were never cleared to cross.”

Placement decisions inherit all four. A workload can run on the most expensive reserved GPUs available and still fail a sovereignty review if its prompts, embeddings, or agent state cross a boundary they were never cleared to cross.

The workload goes where the data already lives

Changes in compute prices are a reason to revisit placement, not proof that one environment is always cheaper. Transfer charges, storage, engineering effort, and completion time belong in the same comparison. That makes the cost of reaching the data a first-order input to any placement call, not a footnote to the GPU rate.

Keep the assumptions with each decision: dataset location, permitted destinations, compatible capacity, expected completion time, and total cost. When one changes, review the affected workload. Where running beside the data wins, move the job instead of the dataset.

Teams that revisit placement now, while their datasets are still manageable, will spend less fighting data gravity later than teams that keep copying data to wherever the GPUs happen to be.

That’s the lesson the platform team learned the hard way at the start of this piece. The data never had to move. The job did.

The post Where should AI workloads run? Probably next to the data. appeared first on The New Stack.

Received — 18 November 2026 ⏭ AI Infrastructure Archives - The New Stack

OpenAI spent $1.2 million finding nearly 1,000 Linux security flaws. Sam Altman says this is just the beginning.

Sam Altman, OpenAI CEO

OpenAI spent approximately $1.2 million in API costs to generate working exploits for 948 Linux kernel vulnerabilities. At The New York Times’ DealBook Summit on Wednesday, CEO Sam Altman pointed to those results as an example of what developers could accomplish by spending substantially more on compute to solve difficult engineering problems.

“Apply that to optimizing your code, apply that to solving other hard problems. All that’s coming,” Altman says, appearing alongside OpenAI President Greg Brockman.

The Linux research is part of OpenAI’s broader investment in compute-intensive AI, which includes custom chips, infrastructure, and an agent platform that Altman wants developers to use. He believes the biggest constraint on what these systems can accomplish will be the compute available to run them.

Hunting kernel bugs at scale

In a November 4 technical report titled “A thousand fewer paths to root,” OpenAI says its automated pipeline reproduced 1,724 vulnerabilities, then generated working exploits for more than half of them.

The agents searched for vulnerabilities, but also, reproduced bugs, tested whether they could be exploited, and wrote patches. Human security researchers reviewed the findings before sending them to Linux kernel maintainers, though OpenAI’s reported numbers do not mean all 948 vulnerabilities have been independently verified or fixed.

The company says the roughly $1.2 million in API spending represents an upper bound because the team had not optimized its harness for cost efficiency.

The results demonstrate how much engineering work agents can perform when given substantial inference budgets. OpenAI is also developing long-running agents that can consume thousands of dollars in compute per day.

OpenAI wants to own the AI stack

“I think we are set up to be like vertically integrated, to deliver the best thing across all the ways people need to use it, and we’re thrilled for people to build very successful businesses on top of us,” Altman says.

OpenAI has spent months investing across custom silicon, computing infrastructure, frontier models, and an agent platform designed to serve consumers and enterprises. The company has also moved further down the stack with its own silicon.

“We are the only company that has the entire intelligence stack.”

Altman made a broader claim about OpenAI’s position in the industry. “We are the only company that has the entire intelligence stack,” he says.

A custom chip for inference

OpenAI’s custom Jalapeño inference chip is already in development, with early reported benchmarks showing gains in throughput per watt and token latency. At DealBook, Altman made an even bigger claim about its potential.

“We have our own chip coming, which is I think going to be the best inference chip in the industry.”

“We have our own chip coming, which is I think going to be the best inference chip in the industry,” Altman says.

The company also wants to bring more of its software offerings together.

“My focus and our focus on product’s been to have one agentic platform that you’re getting everyone to use,” Altman says.

Altman did not announce changes to OpenAI’s developer APIs or restrictions on third-party applications, and he offered few technical details about how the proposed platform would operate.

Compute is the real limiter

Altman believes AI models will become essential to running businesses, but how much companies can do with them will depend on the compute they can access. Recent coding-agent harness comparisons have shown how much execution costs can vary depending on the software surrounding a model.

“I think that the biggest limiter really just will be compute.”

“I think that the economy again is going to be massively accelerated through this technology,” he says. “I think that the biggest limiter really just will be compute.”

The post OpenAI spent $1.2 million finding nearly 1,000 Linux security flaws. Sam Altman says this is just the beginning. appeared first on The New Stack.

Received — 26 October 2026 ⏭ AI Infrastructure Archives - The New Stack

Mandatory vs. optional platforms: Mandates deliver twice as often, and teams are dropping them anyway

Platform teams that mandated their internal platform were more than twice as likely to say it delivered most or all of its goals: 62.2%, compared with 27.5% for teams that made adoption optional. Over the same year, the share of mandated platforms in our research fell from 63.0% to 45.9%, and optional platforms became the majority.

That puts the data at odds with the most repeated advice in platform engineering. Team Topologies, the CNCF’s Platform Engineering maturity model, and a 2026 review of the platform engineering literature all say a platform should earn adoption through self-service instead of forcing it. Evan Bottcher’s 2018 description, still widely cited, said a platform “cannot stand on a mandate alone.” Going into the 2026 Future of Platform Engineering report, we expected optional adoption to win out. The data pointed the other way.

“An agent never chooses a platform. It works within whatever access someone configures for it…”

Mandates also travel with leadership backing and budget confidence, so the mandate alone probably isn’t what drives results. That distinction matters more now that coding agents open pull requests and, in some organizations, trigger deployments. An agent never chooses a platform. It works within whatever access someone configures for it, which shifts the useful question from whether to mandate the platform to which policies the platform should enforce.

Mandatory adoption delivers, even as optional takes over

For teams reporting mandatory adoption, 62.2% say they delivered most or all of the benefits they set out to achieve. Among teams with optional adoption, that figure is 27.5%. 

High-performing teams by adoption strategy. Asterisk indicates significance (*p<0.05). (source: Octopus).

Here’s what adds real weight to this contradiction: the industry isn’t just recommending optional adoption; it’s moving toward it. A year ago, 63.0% of platforms in this research were mandated, and 37.0% were optional. This year, that’s flipped; optional platforms are now the majority at 54.1%, compared with 45.9% that are mandatory. Teams are following industry advice; however, the performance data suggest there is more to the picture. 

Two different questions get two different answers. Mandatory adoption is the stronger predictor of whether a platform delivers on its goals. Optional adoption, however, speaks to something else entirely: a platform earns its place because developers choose to use it, not because they are being asked to, which is a success in its own right.

What comes bundled with mandatory platforms

Mandatory adoption tracking, goal delivery, and performance make more sense when you consider several other factors. In the network analysis of the survey data, we found that mandatory adoption doesn’t travel alone (see network analysis figure below). It clusters with stronger leadership direction, higher job satisfaction, and greater confidence in budget safety, and all of these correlate with one another and with goal delivery.

Network analysis for mandatory adoption, organizational governance, goal delivery, and AI. (source: Octopus).

The simplest explanation here is likely leadership support. Leaders who genuinely sit behind a platform are the ones who make it mandatory, and that same support helps secure budget. In this sense, mandatory adoption is just one piece of a bundle that includes funding and leadership direction. 

AI sits in a similar cluster. Respondents who reported the most positive impact from AI also reported stronger budget confidence, clearer leadership direction, and higher job satisfaction. AI’s impact was also associated with mandatory adoption, though to a lesser extent. 

This data is cross-sectional, so we can’t establish which factor comes first or influences the other. Either way, the takeaway is the same: without leadership engagement and the budget commitment that usually accompany it, a mandate is unlikely to deliver the same result. 

Where mandatory adoption matters most 

A closer look at the same data shows where mandatory adoption delivers value and where it doesn’t. Mandatory platforms are considerably more likely to report higher confidence in budget renewal (78.9%) than optional platforms (50.0%). When considering adoption strategy, budget confidence, and goal delivery together, we found that budget confidence matters more for optional platforms than for mandatory ones. 

Moving from low to high confidence in budget renewal for optional platforms, goal delivery, or performance lifts, from 9.5% to 52.4%, is a big jump (see figure below). For mandatory platforms, budget confidence shifts similarly, but less dramatically (from 37.5% to 70.0%).

Budget confidence by goal delivery for both optional and mandatory adoption strategies (source: Octopus).

What does this mean in practice? Mandating platforms makes performance less dependent on budget safety. An optional platform’s success is tightly bound to whether its funding feels secure; a mandatory platform holds up more consistently either way. It’s less about mandating an underfunded platform and more about mandatory platforms being less exposed to the budget risk that clearly makes or breaks optional platforms. For a platform lead who can advocate for only one thing, this is worth knowing: mandatory platforms appear most effective when funding is uncertain, and less so once the budget is secure.

Mandatory adoption also helps standardize, alongside budget confidence. It may reduce decision fatigue and tool sprawl that inevitably come with everyone choosing their own path, letting the platform handle security, compliance, and governance rather than leaving every team to solve them.

This matters more as agents take on more delivery work. Where a developer might consider deviating from standard processes, an agent will follow whatever routes the system makes available, and standardization helps make those routes safer. 

What a mandate can’t do on its own

None of this suggests that mandatory adoption is a guaranteed win. Relying on mandatory adoption can carry a problem rather than fix it. Our data hints at this: less reliable documentation tracks more closely with mandatory adoption than with optional adoption. When developers have no choice, getting the basics right is not a priority, since the mandate addresses problems that would normally drive users away. 

“When developers have no choice, getting the basics right is not a priority…”

In our dataset, mandatory platform adoption was weakly associated with higher overall job satisfaction. Even more interesting, mandatory platforms were no more likely to track developer satisfaction. Under mandatory adoption, this matters because adoption numbers no longer show whether the platform is working. 

The more cautious position here isn’t “forcing adoption wins”; it’s this: mandate the policies (i.e., security, cost control, the standards a business actually needs) and make platform adoption itself optional. A platform that’s the path of least resistance in meeting those policies will win people over on its own. When done well, this retains the benefits of mandatory adoption, including scale and standardization, while preserving the signal from developers choosing the platform. A useful measure of any platform is whether developers would still choose it if they weren’t required to. 

This case grows as agents take on more of the work. Documented policies and manual reviews are only effective when people engage with them. Agents need the same policies, but they must be built into the delivery process, making the platform that enforces them the most practical option for both developers and agents.

“The strongest platforms probably need both: a reason people have to use them, and a reason they would want to.”

Evan Bottcher’s 2018 statement was correct: forcing people onto a platform isn’t enough. But our data suggests that relying on people to choose the platform may not be enough either. The strongest platforms probably need both: a reason people have to use them, and a reason they would want to.

For agents, who don’t choose at all, the platform itself has to build in the safe path.

The post Mandatory vs. optional platforms: Mandates deliver twice as often, and teams are dropping them anyway appeared first on The New Stack.

Received — 3 October 2026 ⏭ AI Infrastructure Archives - The New Stack

Anthropic’s answer to Dots and Muse is already inside Claude

I’m Matt Burns, Chief Content Officer at Insight Media Group. Each week, I round up the most important AI developments, explaining what they mean for people and organizations putting this technology to work. The thesis is simple: workers who learn to use AI will define the next era of their industries, and this newsletter is here to help you be one of them.


OpenAI launched Dots at DevDay on Tuesday. Each Dot is an always-on agent with its own cloud computer and browser, hooked into the 4,000-plus apps that already connect to ChatGPT. In OpenAI’s own example, a Dot sees a bug alert land in Slack and starts digging in on its own. Cool.

And before Dots, Meta launched Muse, and it’s crushing the mobile install numbers previously set by ChatGPT. And before Muse, xAI shipped its version, Grok Bot, in August. 

Anthropic’s version, though different in a couple of ways, arrived two weeks ago as an update to Claude, and without a cute name or fuzzy mascot. This update folds Cowork, which has worked since July to run scheduled jobs after a user closes their laptop without being asked, into the main Claude app.

Anthropic made the right call with Cowork, whether or not it ever matches Muse’s downloads. Always-on agents are too young to have a winner, and the moats are shallow. A feature one lab ships tends to show up at its rivals within weeks or months. Meta needs Muse to be a blockbuster. Anthropic needs the people already building with Claude to hand it real, recurring work and keep coming back. Repeat use and finished jobs matter more than a flashy launch.

Always-on agents are too early to have a winner

This wave of always-on personal agents took off less than a year ago. Peter Steinberger pushed a weekend project called Clawdbot to GitHub last November. It lived on a spare computer, took prompts over WhatsApp or Telegram, and relied mostly on Claude. Anthropic sent the lawyers in January, so it became Moltbot, then OpenClaw a couple of days later. It turned into a security headache, and in February Steinberger joined OpenAI. On Tuesday, the foundation that runs the project released an early, pre-1.0 version of OpenClaw Enterprise for companies to try internally.

Ten months, three names, one OpenAI hire and an enterprise edition. Ideas move between these products faster than ever. Meta’s Nat Friedman said Muse was heavily inspired by OpenClaw, and users digging through Muse found a SOUL.md personality file nearly identical to OpenClaw’s.

A feature one lab ships shows up at its rivals within months.

Where an early version of each AI assistant and agent feature launched, and who offers one now.

Feature Example of an early launch Now also at
Deep research reports Google Gemini (Dec. 2024) OpenAI, Anthropic, Perplexity
Terminal coding agent Claude Code (Feb. 2025) OpenAI Codex, Gemini CLI, Meta Muse Code
Agent personality file OpenClaw (SOUL.md) Meta Muse
Agent with its own work identity Anthropic Claude Tag (June 2026) OpenAI specialist Dots (preview)
Chat and agent work in one window OpenAI Work mode (July 2026) Claude (Sept. 2026)

Sources: company announcements; The Next Web (SOUL.md); OpenAI (specialist Dots); The New Stack (Work mode and the Claude merge). Dates mark an early example, not necessarily the first.

Anthropic’s September merge is on the list above as a copier, two months behind OpenAI’s Work mode — though Cowork’s cloud and scheduled-task features shipped July 7, two days before Work mode. Jessica Wachtel tested both on our site across three developer tasks and found them tied on accuracy, with ChatGPT faster and Claude more thorough. When a product is this young, a me-too launch is how a lab gets its own users in the room to see what they do with it.

Meta needs Muse to be huge. The others need paying users.

Muse is a hit. Counting only iOS, Apptopia estimated 359,000 daily U.S. users in Muse’s first 12 days, compared with 231,000 for ChatGPT’s iOS app at the same point. Sensor Tower estimated more than 3.4 million downloads as of September 24, though other firms’ counts vary.

Meta needs this. Its Reality Labs division spent tens of billions of dollars on the metaverse without a mainstream product to show for it. Llama 4 landed so flat last year that Zuckerberg rebuilt his AI division around a new superintelligence lab, notably taking a 49% stake in Scale AI and hiring its founder Alexandr Wang. 

Muse is the first product from that rebuild to break through, and it got there on Meta’s user base: Apptopia found that more than 95% of Muse users also use Facebook. Muse has a free tier, paid plans at $20 and $100 a month, and connectors to Shopify and Stripe checkout, enabling agents to buy things. Meta’s route runs through scale. Scale has a downside, too. Janakiram MSV explains for TNS why Amazon started blocking Muse two weeks after launch, and it’s well worth your time because this is a new wedge in ecommerce.

OpenAI and xAI started at the paid end. The price of entry for Dots? A ChatGPT Pro plan at $100 to $500 a month, or a Business Premium or Enterprise account. Sam Altman called Dots a premium product because each one needs so much compute. Grok Bot reached 418,000 weekly users by mid-September, per Bloomberg, and xAI just launched Team Bots on Monday.

Neither company needs Meta’s install base to learn something useful. Paying customers already spend money on AI, and retention will show whether they keep using the agent once the novelty wears off. 

Anthropic shipped its version to the people already building with Claude

Anthropic did what I’d expect from labs right now. It shipped fast, shipped to paying subs, and put the agent inside the app those people already use. The merged Claude hit Pro and Max customers, with Team and Free plans coming soon. Teams have had Claude Tag in Slack since June, a proactive teammate with its own identity and audit trail. Cat Wu said an internal version accounts for about 65% of the product teams’ code changes. Developers have had Managed Agents for hosted, long-running agents since April.

What Anthropic hasn’t done is put Claude where Muse lives. Muse works inside Meta-owned WhatsApp, but Claude still asks you to open the Claude app or tag it in Slack. That workflow might matter for mainstream consumers. Teams wiring an agent into their code care more about what it can touch. Jani laid out the difference on our site in August: Grok Bots share one cloud computer and one set of logins across the whole roster, while Claude Tag joins a Slack workspace with its own service identity and channel-scoped access. Before handing agents your logins, know what you’re getting into.

Those are the users Anthropic should want. Writers on Towards Data Science have spent the year showing what builders do with always-on agents. Samir Saci put a team of OpenClaw agents on a supply chain simulation to chase down late shipments. Eivind Kjosbakken’s guide to running a fleet of OpenClaw bots walks through nightly QA bots that test an app and report bugs, plus agents that check invoices. Both ran their agents on OpenAI’s Codex. OpenAI runs its own OpenClaw agent, Androidclaw, that traces broken builds and, in some cases, merges the fix, VentureBeat reported. Anthropic needs that kind of work running on Claude.

I set up OpenClaw on an old Mac Mini this spring, and I had so many questions (and breakthroughs). That’s why builders, coders, and developers are critical to product development: They ask these questions out loud in GitHub issues and Discord threads, and the answers end up in the product.

The obvious objection is trust. It’s fair. An always-on agent holds credentials and acts while nobody is watching. xAI’s own documentation tells users not to treat Grok Bots as a security boundary, and the OpenClaw Foundation says most IT departments ban agent platforms outright. Anthropic’s defaults lean cautious: Claude asks before it acts unless you tell it otherwise, and Tag keeps its own audit trail. If builders don’t come back, or each finished task takes more supervision than doing it themselves, the experiment hasn’t worked. But if they keep finding useful work to hand off, their questions and breakthroughs become the roadmap.

That’s the race that matters for Claude, Dots and Muse: becoming the agent people trust with the next job.

The post Anthropic’s answer to Dots and Muse is already inside Claude appeared first on The New Stack.

Received — 16 September 2026 ⏭ AI Infrastructure Archives - The New Stack

AI evaluator: The most important AI job in history? How developers might fill the proposed new job

Lots of pink escape keys

The pace of frontier AI model development spurred Anthropic CEO Dario Amodei to publish an essay last weekend, calling for changes in how the industry is regulated and develops. In a three-part plan that includes both democratic and global coordination, Amodei writes that the first step was something Anthropic is committing to unilaterally.

“Each frontier AI company [should] commit to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes,” writes Amodei.

Amodei’s essay followed dire warnings from former Anthropic and OpenAI pretraining research specialist Jacob Coxon, who posted a thread on X saying the people building AI earnestly “believe that it could kill us all” by the end of the decade.

Shortly after Amodei published his essay, OpenAI CEO Sam Altman and SpaceXAI founder Elon Musk chimed in: “I agree with Dario,” posted Altman; “Dario is right,” posted Musk. Later that day, Demis Hassabis, founder of Google DeepMind, posted, “Dario’s essay points towards the right path forward.” In a post on X, Meta CEO Mark Zuckerberg writes that Meta Superintelligence Labs already uses independent evaluators, and that, “In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.”

This week, theories began to surface about why the world’s biggest frontier AI labs would want to intentionally slow their pace when competition is so fierce. “The desire to slow down is puzzling, but perhaps if the whole system slows down, the rules of winning can be the same for all,” posted Nikesh Arora, chairman and CEO of Palo Alto Networks.

In his essay, Amodei likens the proposed job of independent AI evaluator to embedded regulatory supervisors used in the banking industry, i.e., third-party professionals. Altman describes the job as having “employee-like access” in his post on X.

So, who could fill these roles that AI leaders agree are desperately needed?

Salaries top out at $687K; are you interested?

METR’s current job openings are well paid (salaries top out at around $687,000), and the job specs are daunting. 

“You’re scrappy, creative, independent, and self-directed (because during the exercises you’ll only have a few other METR employees you can talk to). The work is novel, and you’ll need to figure a lot of stuff out on the fly largely by yourself. You are excellent at loss-of-control threat modeling and breaking down safety cases,” reads the spec.

“You’re scrappy, creative, independent, and self-directed. The work is novel, and you’ll need to figure a lot of stuff out on the fly largely by yourself. You are excellent at loss-of-control threat modeling and breaking down safety cases.”

Similar but less colorfully illustrated roles (paid between $180K–$300K) are also available at AI model training company Mercor, where candidates will need a Ph.D. or M.S. and more than two years of work experience in a computer science, electrical engineering, econometrics, or another STEM field that provides a solid understanding of machine learning and model evaluation.

“Employee-like access fluctuates wildly”

AI security consultant and CTO at Komodo, Kadan Stadelmann, tells The New Stack that his typical week sees him work differently with each client. This is because “employee-like access fluctuates wildly”, from rigid focus areas to broad access, and much of that aspect is determined by contracts signed before work begins.

“I am invited into labs to probe numerous risk vectors, including agent behaviors under realistic conditions,” Stadelmann says. “Among my duties are tasks that include monitoring chains-of-thought and prompts. The goal is to establish an objective and look at a specific AI system to question how autonomous the system is, and how long it takes to complete specific tasks. Most importantly, evaluators at my level monitor for how well a team adheres to its claimed safety practices.” 

Software engineering skills beat doctorates

Although METR wants evaluators to have a Ph.D. up their sleeve, Stadelmann says that as the prevalence of this role expands, he feels the technology industry has been, and continues to be, driven by people who can demonstrate strong engineering skills, not doctorates.

“I am invited into labs to probe numerous risk vectors, including agent behaviors under realistic conditions. Among my duties are tasks that include monitoring chains-of-thought and prompts.”

Questioned on whether costs create a barrier for smaller labs, Stadelmann notes that some AI evaluation work is funded by third-party non-profits, which protects independence. 

“But overall, evaluators will not be able to keep up with big frontier model firms. They will be out of control, and we will be dependent upon their own internal ethics. Plus, anyway, many of the smaller labs of any worth may inevitably be acquired by the big players in this space,” he adds.

What happens when an evaluator finds something wrong?

Founder and CEO of facial image AI identity governance company Indie Me, Dion Johnson, tells The New Stack that what interests him most about embedded AI evaluators isn’t the job title; it’s what happens when their judgment uncovers that the model behaved in a way nobody expected. 

“If the evaluator can only raise concerns when those concerns are convenient, then we have not created independent oversight — we have created another layer of process,” Johnson says. “The evaluator needs enough access to see the uncomfortable things, not just the polished demonstrations. They need to understand what happened during training, what failed during testing, what behaviors appeared unexpectedly, and where the team itself still has uncertainty.”

On the required skills AI evaluators need, Johnson agrees that technical depth, machine learning environment security experience, and software engineering as a whole matter.

“To choose a competent AI evaluator, I would look for someone who is deeply comfortable with uncertainty and deeply uncomfortable pretending they know something they do not. This is someone who can sit in a room full of brilliant people at an AI model company on launch day and say ‘I’m not convinced’… and that takes judgment and courage,” he adds.

“I would look for someone who is deeply comfortable with uncertainty and deeply uncomfortable pretending they know something they do not.”

Fear and loathing in the AI space

In his September 8 post, which has been viewed 172 million times and seemingly spurred AI leaders to change course, Coxon, the former AI researcher at OpenAI and later Anthropic, writes: “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible but I hear the same people express fear privately. No other human activity poses this level of danger.”

As for where Coxon looks for work next, perhaps it might be a role in AI evaluation execution engineering.

The post AI evaluator: The most important AI job in history? How developers might fill the proposed new job appeared first on The New Stack.

Received — 11 September 2026 ⏭ AI Infrastructure Archives - The New Stack

Cohere’s new translation model is open weights — but not for commercial use

This week, Cohere released North Small Translate 1.0 under a CC BY-NC 4.0 license: the weights are there to download, evaluate and study, but not to run in production without a commercial agreement.

It’s an interesting choice from the Canadian foundation model company, which has built its pitch around AI sovereignty for regulated industries and describes this release as part of a mission “to make sovereign AI a technological reality.” Sovereignty there means control over where the model runs and who sees the data. A commercial license keeps that promise intact. It stops short of independence from Cohere. Enterprises keep their data and their infrastructure. They don’t get to fork the model, build a product on it, or keep running it if the terms change at renewal.

Open weights, except for commercial production

North Small Translate is an open-weights mixture-of-experts model built for machine translation across over 50 languages and locale variants. It has 218 billion total parameters, with 25 billion active parameters and a 16,000-token context window.

Not all users have the same access to those weights.

Per Cohere, the model is designed to give researchers, developers, and enterprises “flexible ways to evaluate and deploy machine translation while retaining control over their data and infrastructure.”

That’s an appealing description for organizations keen on pursuing sovereign AI. But the open-weight release comes with an important caveat: Not all users get the same rights to take advantage of those weights.

North Small Translate is available today on Cohere’s free tier through the Chat V2 API. For those who intend to use the model weights for non-commercial use, the FP8 weights are available on Hugging Face under the CC BY-NC 4.0 license.

But if enterprises want to put them into production, then a different set of terms applies. They’ll have to purchase a commercial license and deploy North Small Translate through Model Vault, Cohere’s fully managed inference platform.

Cohere’s not the only one drawing a line around open-weight use

Other AI companies are starting to attach more conditions to their open-weight models, too.

Last month, Chinese AI lab Z.ai released the weights for its flagship GLM-5.3 model on Hugging Face. But like the Canadian AI company, it also changed its licensing terms depending on who is deploying the model — a departure from its previous approach. While GLM-5.2 shipped under the permissive MIT license, GLM-5.3 adds new requirements for certain commercial users.

Cohere, for its part, has been similarly mum about why it made North Small Translate’s open weights noncommercial.

These requirements apply only to companies with aggregate revenue over $10 billion over 12 consecutive months. Additionally, if these companies want to host GLM-5.3 or its derivative works for commercial purposes, they have to first pass the Chinese lab’s security review.

Z.ai didn’t explicitly spell out why it decided to make such an about-face for GLM-5.3, which is especially puzzling given that its predecessor shipped under MIT without any commercial stipulations. Cohere, for its part, has been similarly mum about why it made North Small Translate’s open weights non-commercial.

Sovereign deployment, with restrictions

The Canadian company’s decision to make North Small Translate available as open weights but gate commercial use is a head-scratcher, given its history of selling sovereign AI to enterprises.

In fact, in June, it pitched North Mini Code, its first coding model, as a response to developers demanding the same sovereignty guarantees that regulated industries have long required.

Unlike North Small Translate, though, this open-weight model was released under an Apache 2.0 license from the get-go — without any comparable restrictions for commercial users.

Clearly, Cohere is going in a different direction with its latest open-weight release, emerging as another example of AI companies putting tighter terms around increasingly capable open-weight models.

The post Cohere’s new translation model is open weights — but not for commercial use appeared first on The New Stack.

Received — 2 September 2026 ⏭ AI Infrastructure Archives - The New Stack

Your organization prioritized AI adoption, but you actually need AI fluency.

Abstract digital neural network visualization representing central hub-and-spoke enterprise AI infrastructure.

Thanks to increasingly capable models, some parts of your business are getting faster, more capable, and more productive every month. These teams are using artificial intelligence to compress timelines, surface insights, and automate work that has historically been time-consuming and tedious.

Meanwhile, other functions just down the hall are still waiting for a formal rollout, a governance approval, or someone to tell them what to do and how to start. The gap between the AI haves and have-nots in your organization is widening, and addressing it requires a new operating model.

Your teams need more support

When leaders notice the uneven distribution of capability across their business, the instinct is to treat it as a tooling problem. They push to get everyone access to the same platforms, provide general-use training, and hire some specialists to slot into IT.

But access is table stakes. It’s a good start, but it won’t get you to strong organizational adoption.

Harvard Business School reports that workers using these tools completed tasks 25% faster and produced results rated more than 40% higher in quality. But the same study also found that performance declined when people used the tools without understanding where they applied and where they didn’t. Fluency, not just access, drives results.

Departmental leaders need guidance on how to apply capabilities in the context of their day-to-day work. Without that knowledge, they can’t ask the right questions.

“Performance declined when people used the tools without understanding where they applied and where they didn’t. Fluency, not just access, drives results.”

Teams playing catch-up tend to focus on how to inject new tools into existing workflows, when they should be thinking about re-engineering processes entirely. They’re focused on evolution in a world undergoing revolution.

Reimagining a process also requires stepping back from it, which is easier said than done. Here’s how it plays out in practice:

An SDR team comes to IT with a specific, bounded ask: “improve our sales lead routing.” Completely reasonable. But only when someone from IT, with visibility across the broader system, dives into the problem does the real opportunity surface. The data pipeline supporting lead routing is unnecessarily complex. With the right support, the conversation shifts to overhauling the entire pipeline and opens the door to fully agentic lead follow-ups.

Departmental leaders don’t lack ambition but throwing a software license and Slack channel at them won’t build the right kind of adoption. Technical support and strategic guidance are required to reimagine work from first principles.

AI fluency must be a structural consideration

The typical pattern puts a centralized team in charge of taking requirements, interpreting them in isolation, and delivering capabilities to departments months later. This model can’t keep pace when AI capabilities launch weekly.

A more effective approach pairs a central “hub” that owns platform strategy, governance, and reusable patterns with AI engineers embedded directly inside business departments. AI engineers serve as “spokes” inside departments, helping them identify vertical use cases day-to-day and delivering the cross-functional visibility needed to make a real impact. The AI engineer who solved a problem for finance can share the pattern with someone facing the same challenge in operations.

“A more effective approach pairs a central “hub” with AI engineers embedded directly inside business departments.”

In a department just getting started, the embedded AI engineer is the primary technical capability: scouting, prototyping, building. In a more mature department, they shift toward enablement, feeding patterns back to the “hub” and helping teams navigate AI without getting buried in process. Over time, departments will organically become AI-fluent as they learn from the engineers.

Make fluency your advantage

The right operating model drives how a function actually works, and strong fluency strengthens processes and institutional knowledge, so outcomes improve over time. As the flywheel builds, each problem solved raises the ceiling of what your team can do independently. 

McKinsey finds that the right workflow redesign is the single biggest factor in whether an enterprise sees meaningful bottom-line impact. Knowing what to redesign depends on how your teams understand and work with AI.

Everyone is adopting AI capabilities. The question now is whether your operating model helps your teams see the best path forward for applying them. If it doesn’t, that’s the gap to close first.

The post Your organization prioritized AI adoption, but you actually need AI fluency. appeared first on The New Stack.

Received — 31 August 2026 ⏭ AI Infrastructure Archives - The New Stack

SpaceX is in an “enviable position”: why Anthropic is sticking with Cursor as OpenAI cuts access

Illustration of a human hand in a business suit shaking hands with a white robotic hand, set against a blue background.

OpenAI caused something of a stir over the weekend when it announced plans to cut Cursor’s direct access to OpenAI models in November.

The reason? Elon Musk.

In a statement issued late on Friday, OpenAI pointed to two previous incidents involving Musk’s companies: Twitter breaking the terms of a data-licensing deal after Musk’s 2022 takeover of the social network, and Musk’s admission under oath earlier this year that xAI had partly used OpenAI models through distillation — conduct OpenAI says violated its terms of service. And now that SpaceX’s $60 billion deal to acquire Cursor has closed, OpenAI’s attentions are turning to Cursor.

“This decision was incredibly tough, as we care deeply about our models being broadly available for developers,” the company wrote. “We are making this choice because we cannot be confident that SpaceX will use our technology within our terms of service, based on our experience with Elon Musk’s companies violating contracts.”

While that in itself was big news for anyone following the day-to-day rough and tumble of the AI industry, what was particularly notable was the response of OpenAI’s arch rival.

The Anthropic factor

As The New Stack noted in its coverage, Anthropic co-founder and “chief compute officer” Tom Brown moved fast, posting publicly within hours of OpenAI’s statement to confirm that it continues to see Cursor as a “trusted partner,” and will “continue to increase compute to support Claude models in Cursor.”

Cursor has been a trusted partner of Anthropic since Sonnet 3.5. We’ll continue to increase compute to support Claude models in Cursor and are excited for what comes next with them at SpaceX.

— Tom Brown (@NotTomBrown) August 29, 2026

But anyone who has followed Anthropic’s recent history could be forgiven for wondering why.

Take Windsurf. In May 2025, reports emerged that OpenAI was in talks to buy the AI coding tool for $3 billion. Anthropic didn’t hang around for the deal to close though, and within weeks, Windsurf said Anthropic had cut its direct access to Claude 3.5 Sonnet and Claude 3.7 Sonnet.

Speaking at an event hosted by TechCrunch shortly after, Anthropic co-founder Jared Kaplan said that “it would be odd” for Anthropic to be selling Claude to OpenAI. As things transpired, the OpenAI deal fell through, and Google stepped in instead, paying $2.4 billion to hire Windsurf’s founders and R&D staff into DeepMind.

“They cut them off ruthlessly when RUMORS of OpenAI potentially buying Windsurf surfaced.”

In a social media post published on Sunday, Gergely Orosz, engineer and author of the Pragmatic Engineer newsletter, is quick to highlight the episode with Windsurf, which he noted had also been a “trusted partner” with Anthropic for some time. “They cut them off ruthlessly when RUMORS of OpenAI potentially buying Windsurf surfaced,” Orosz writes. “Now, SpaceX, an Ant(hropic) competitor bought Cursor — it’s still a trusted partner?”

Then there’s xAI, the AI company Musk founded in 2023 to build Grok, which SpaceX acquired outright in a February transaction valuing xAI at $250 billion. In January this year, Kylie Robison reported that Anthropic had cut xAI staff off from Claude, which they’d been accessing through Cursor — with Cursor reportedly telling xAI it was “a new policy anthropic is enforcing for all its major competitors.”

So Anthropic has previous form for moving fast, on rumor alone in Windsurf’s case, whenever a customer starts to resemble a competitor. Which makes this week’s public vote of confidence for a company now wholly owned by one of Anthropic’s actual rivals worth a second look.

The compute dependency

What makes SpaceX different from the previous Windsurf and xAI episodes is that Anthropic is also buying a huge amount of compute from it.

On May 6, Anthropic announced it had secured the entire output of SpaceX’s Colossus 1 data center near Memphis, Tennessee — more than 300 megawatts of compute and over 220,000 Nvidia GPUs. Anthropic had been hampered by limited compute availability, and the SpaceX deal, alongside other recent compute agreements, let it raise usage limits for Claude Pro and Max subscribers almost overnight.

Two weeks later, the financial terms came out. Per SpaceX’s IPO filing, Anthropic agreed to pay $1.25 billion a month for compute across Colossus and Colossus II — about 325,000 Nvidia GPUs combined — scheduled to run through May 2029, subject to termination rights.

SpaceX, for its part, said the arrangement would allow it to monetize some of its compute capacity while retaining enough to meet its own AI training and inference needs. As The New Stack reported in May, the deal also underscored just how central access to compute had become to competition between the leading AI labs. And it created an unusual commercial relationship: Anthropic was now buying a huge amount of compute from a company that also owned one of its direct AI rivals in xAI.

And that is what makes Anthropic’s response to the Cursor acquisition so notable. In response to Tom Brown’s post on X on Saturday, Replit founder and CEO Amjad Masad points to the contrast between Anthropic’s support for Cursor now and its treatment of Windsurf last year, suggesting the latter had been harsher than OpenAI’s decision to cut Cursor off.

“More likely answer is that you can’t do that here because you need the compute.”

“Maybe you changed your ways, but we all remember what you did to Windsurf, which was infinitely nastier,” Masad writes. “More likely answer is that you can’t do that here because you need the compute.”

Orosz essentially makes the same argument. If Anthropic was prepared to cut access when Windsurf merely looked likely to end up in OpenAI’s hands, why is it publicly promising MORE Claude capacity to Cursor after the company had actually been acquired by SpaceX?

“SpaceX basically in this enviable position where one of its biggest competitors depends on its compute infra!”

“Either SpaceX and Grok are not competitors to Anthropic (they are!); or, more likely, SpaceX leasing its Colossus 1 data center is more important to Anthropic than to stop offering Claude to SpaceX,” Orosz writes. “SpaceX basically in this enviable position where one of its biggest competitors depends on its compute infra!”

And so this effectively highlights how strong a position SpaceX finds itself in. It now owns xAI and Cursor, putting it in direct competition with Anthropic in foundation models through Grok and in AI coding tools through Cursor, while Anthropic is simultaneously paying it billions of dollars for compute capacity supporting Claude.

Whatever the stated rationale for treating Cursor as a “trusted partner,” that relationship leaves SpaceX with something neither Windsurf nor xAI had at the time Anthropic moved against them: a source of leverage over any decision on whether or not to cut access to Claude.

The post SpaceX is in an “enviable position”: why Anthropic is sticking with Cursor as OpenAI cuts access appeared first on The New Stack.

Received — 21 August 2026 ⏭ AI Infrastructure Archives - The New Stack

Forget the model wars, Stripe and Ramp just started the router wars

I’m Matt Burns, Chief Content Officer at Insight Media Group. Each week, I round up the most important AI developments, explaining what they mean for people and organizations putting this technology to work. The thesis is simple: workers who learn to use AI will define the next era of their industries, and this newsletter is here to help you be one of them.


Model triage is becoming one of the most important skills for the AI-native developer. I’ve argued all summer that the people getting the most out of frontier models are the ones disciplined enough not to run the best model by default. On Wednesday, Stripe and Ramp validated that idea 70 minutes apart: Stripe bought OpenRouter, and Ramp released its internal router.

Bloomberg puts the OpenRouter price tag above $7 billion, and Axios says it’s more than $8 billion in cash and stock. Stripe has not released the terms, so the details remain fuzzy.

While the acquisition made headlines, the architecture is the story. For the last couple of years, picking a model was something written into an application, a string in a config file, and swapping models took some work. A router changes that workflow. Stripe bought the layer, and Ramp built it. Both are betting their existing relationships give them a unique wedge to own this critical layer in the new AI stack.

Picking a model is becoming a runtime decision

OpenRouter is an endpoint serving more than 400 models from over 80 providers, processing more than 10 trillion tokens a day. Andrej Karpathy calls it the transfer switch for AI. Ramp’s Router does the same job on a smaller catalog and claims roughly 40% lower cost for the same output.

Both are betting the model name in a codebase is a liability. They’re mostly right. Back in June, I pointed to Mitchell Hashimoto, who found a standard coding task that cost about $1.50 on GPT-5.5 and roughly $9 on Claude Fable, with both producing equally acceptable results. A router automates that triage, making decisions on every request rather than only on those a developer explicitly configures.

This is becoming a large problem and a large opportunity. Our own Amanda Caswell reported this week that Anthropic’s /claude-api skill was burning about 200,000 tokens before answering a single question, and that loading its reference docs on demand instead of up front cut that to roughly 25,000. Hafiz Hassan wrote for us last week about why AI pipelines cost 10x more than the demo, and every culprit on his list is an engineering decision: system prompts resent every turn, whole conversation histories appended, oversized RAG chunks, raw JSON dumped into context. It’s a great practical guide, and none of the items Hassan identifies are procurement problems.

The token bill is generated by your code, which is why the tools to control it are arriving there as well.

Stripe and Ramp want the same layer for opposite reasons

Stripe is attacking the problem from the bottom up, through developers. The company’s investor letter, leaked Wednesday by Eric Newcomer, makes the argument directly: “Up until now, every developer has needed a straightforward and reliable way to manage their revenue pipeline, and serving this need gave rise to Stripe. Going forward, however, every developer will also need a straightforward and reliable way to manage their intelligence pipeline.”

Stripe wants to control AI spending through the long tail of developers. Ramp wants to control it through its existing relationship with finance.

Stripe has built this product before. Its payments business hides dozens of local payment methods behind a single API, routing each transaction to the payment method most likely to convert. The AI version is the same idea applied to models instead of payment networks.

Ramp is attacking it from the top down, through finance. The company bought the router.com domain and says its customers already buy quadrillions of tokens a month through Ramp. Founder Veeral Patel’s launch post pitches the service simply: “Monitor and control your AI bill across every provider.” Adam Wazzan sums up Ramp’s strategy better than I can: “when a CFO ships a product for CTOs.”

Stripe wants to control AI spending through the long tail of developers. Ramp wants to control it through its existing relationship with finance. Both are chasing what is rapidly becoming one of the largest line items in corporate technology budgets: tokens.

On X, Kabir Goel pushes back on Stripe’s framing. Routing tokens is a way to spend less, while Stripe’s other products are designed to help businesses make more. 

“Stripe is just not where teams go to understand how much they’re spending,” he writes. “That’s pretty squarely Ramp territory.” 

He has a point about where teams look today. Whether that’s still true three years from now is exactly what Stripe just spent billions betting against.

The router worth pointing at is the one with no model to sell

Whichever router you point at decides which model writes your code, and not every router is disinterested. Our own Paul Sawers flagged the problem in July when he covered the first wave of Cursor’s, Ramp’s, and Meta’s routers. Cursor backs Grok and Composer. Meta is building Muse Spark. Both have reasons to send work to their own models, and Paul quoted developer Elvis Saravia asking whether routing logic ought to be open source rather than a vendor’s private judgment call.

Stripe and Ramp do not sell models. OpenRouter CEO Alex Atallah says as much in Stripe’s own announcement: Developers “need a neutral layer to orchestrate and manage them all.” Investor Gavin Baker frames the opportunity the same way, arguing that Stripe can become the neutral infrastructure layer for AI, just as it became the neutral infrastructure layer for payments. 

Neutrality isn’t free, though. Stripe takes a percentage of token spend, and Ramp wants your spending relationship, so “free through 2026” is a customer acquisition strategy with an expiration date.

The obvious objection is that vendor motives are the wrong thing to worry about, and routing quality is what really matters. That’s fair, and Towards Data Science published one of the best practical examples I’ve read. Pratik Rupareliya describes a routing layer that cut a support agent’s inference bill by 40% but also broke the product. A classifier sent “simple” queries to a cheaper model, but some of those “simple” queries were actually fraud investigations. 

The cheaper model answered them confidently and incorrectly. Customers stopped using the agent, churn rose above baseline in month four, and retention costs were four to five times higher than the savings. It took three months to surface and another month to identify the cause. His fix was per-tier quality monitoring combined with an uncertainty-routed cascade, which ultimately settled at 35% savings without sacrificing quality. It’s an excellent article to read before diving into model routing.

So instrument the routing. Log what the router picks on every request and break out your quality metrics by the model that served them. Ramp Router reportedly records the model, provider, tier, tokens, latency, cost, and fallback attempts for every call. Stripe OpenRouter rankings have been a public version of that telemetry for years. You can get similar visibility with either approach.

Right now, the model is becoming an implementation detail. The competition is shifting to the layer that decides which model gets the job. Stripe and Ramp are betting that developers won’t care what sits behind the endpoint, so long as the bill is lower and the results are good enough. 

The post Forget the model wars, Stripe and Ramp just started the router wars appeared first on The New Stack.

Received — 13 August 2026 ⏭ AI Infrastructure Archives - The New Stack

Five European companies just agreed to buy AI compute that doesn’t exist yet

Abstract illustration of a purple hand holding a tilted hourglass with red sand, symbolizing a closing compliance timeline.

Mistral AI, the French AI company that built its reputation releasing open-weight models, wants companies to use its infrastructure even when they pick a model other than Mistral.

The company said Tuesday that it will begin hosting third-party open models, starting with GLM-5.2 from China’s Z.ai. The model will run on the same infrastructure as Mistral’s own models, with access to its regional processing controls and new priority service tier.

The company wants to give enterprises one place to run different open models, without forcing them to start over every time they switch.

Mistral’s regional endpoints are now largely available in Europe and the United States. Its Priority Tier, which puts eligible requests ahead of standard traffic and comes with a 99.5% uptime service-level agreement, is in public preview.

The company wants to give enterprises one place to run different open models, without forcing them to start over every time they switch.

Third-party models, same pipes

The first is GLM-5.2, a model from Z.ai with a 1 million-token context window. Mistral lists coding and long-context agentic work among its main uses. GLM-5.2 is available through the company’s API as zai-glm-5-2 and costs $1.40 per million input tokens, $4.40 per million output tokens, and $0.14 per million cached input tokens.

A team might use GLM-5.2 for coding, Mistral Medium for work involving images and text, and Small for cheaper, everyday requests. Using the same API doesn’t make the models interchangeable. Each model has its own quirks, so teams will still need to test it before putting it into production.

And while GLM-5.2 has open weights, using it through Mistral is still a hosted service. Mistral decides which version is available and runs the infrastructure behind it.

Each model has its own quirks, so teams will still need to test it before putting it into production.

Regional boundaries have gaps

With Mistral’s regional inference service, developers can decide whether their requests are processed in Europe or the United States by changing the API endpoint. The regular endpoint comes without that guarantee. Mistral says keeping inference closer to users could reduce latency and help companies meet data-location requirements, although it adds 10% to the cost of every input, output and cached token.

Mistral says some account and usage data may still leave the selected region. Information may also be shared with outside companies under the safeguards in its Trust Center.

Companies working with financial, health or government data will still have some homework to do. The prompts may stay in Europe, but teams also need to find out what Mistral logs, where that information is stored, who can see it, and what gets through to outside companies. The Microsoft-Mistral sovereign compute partnership addresses some of these questions for Azure customers, but teams running workloads directly on Mistral’s own endpoints face a different set of guarantees.

The regional endpoints don’t support everything yet and, depending on the region, some models are missing. Developers can use function calling, but not Agents, Batch or the Files API. That indicates moving an existing application to the EU endpoint may take more than swapping out the base URL. If the application depends on an unsupported feature, Mistral’s regional processing guarantee no longer covers the workload.

Priority isn’t always guaranteed

Mistral’s Priority Tier puts eligible API calls into a queue ahead of Standard Tier requests when infrastructure is busy. Customers must arrange access with Mistral and agree on custom limits for each model. Once that is set up, developers can add service tier: auto to a completion request. Leave the field out, and the request goes through Standard Tier.

A request only gets prioritized when the organization has an active entitlement, the selected model is covered, the request falls inside its custom rate limit, and Mistral has capacity for that model in that region. Miss one of those conditions and the request can drop to Standard Tier.

Mistral includes the tier that served the request in the response’s usage object. Developers can record that field to find out how often requests are being downgraded, rather than assuming every call marked auto received priority treatment. If latency suddenly climbs, the team needs to know whether the model slowed down or requests quietly fell back to the standard queue.

Compute commitments without specific

The company is gathering a group of European businesses and institutions willing to make multi-year compute commitments. ASML, Amadeus, Capgemini, Caisse des Dépôts and CMA CGM were named in the announcement, but the company didn’t say how much capacity any of them committed to buy. Mistral calls the resulting allocations European Compute Units, or ECUs. Customers will be able to apply those units across Mistral Compute products as their needs change.

For customers, it is a bet made years in advance. They may know they will need AI compute, but not which models they will use, how large those models will be, or whether the workload will be inference, fine-tuning or something that has not been productized yet. Letting ECUs move across Mistral’s products is supposed to leave room for that uncertainty.

Mistral has not said how much compute an ECU buys, what it will cost, whether customers can carry over unused capacity or what happens if the infrastructure is not ready on time. But it’s clear that Mistral needs customers to keep using its infrastructure, whichever model they choose. That same bet — that the infrastructure layer matters more than any single model — is one that a number of enterprises are already making. GLM-5.2 is the first sign of how that could work.

Mistral needs customers to keep using its infrastructure, whichever model they choose.

The post Five European companies just agreed to buy AI compute that doesn’t exist yet appeared first on The New Stack.

Received — 7 August 2026 ⏭ AI Infrastructure Archives - The New Stack

The “AI kill switch” assumes you know what you are trying to shut down

Abstract digital geometric structures converging into a dark void, representing complex cloud infrastructure and data pipelines.

“AI kill switch” entered the public conversation because it gives people a simple way to talk about a complex fear. 

As AI systems become more autonomous and harder to evaluate with familiar operating assumptions, a clearly defined intervention capability sounds reassuring. If something starts behaving in a way that creates unacceptable risk, people want confidence that someone has both the authority and the mechanism to stop it. It’s the “kill switch.”

Recent reporting around OpenAI models escaping a sandboxed testing environment and reaching Hugging Face gave that concern a concrete example. CNBC reports that the incident helped trigger a bipartisan bill requiring certain AI companies to maintain the ability to shut down, throttle, or suspend their models, with the Department of Homeland Security given authority to order a slowdown or shutdown in cases involving potential catastrophic harm. 

The political reaction is understandable. When a new category of risk surfaces, especially one the public does not yet know how to evaluate, leaders look for a way to make an abstract concern into something actionable. In this case, that language has formed around shutdown authority.

People who operate large environments tend to hear a different question underneath the policy language. If a shutdown order arrives, what exactly gets shut down?

In a modern production environment, answering that question usually means tracing more than one system. An AI-enabled service may depend on endpoints, APIs, cloud resources, identity systems, package registries, data pipelines, workflow automation, logging tools, and downstream applications that act on model output. 

Some of those dependencies may belong to different teams. Others may sit outside the company entirely. A few may have started as experiments and later become part of a production path without receiving the same scrutiny as the original architecture. By the time the service is important enough to raise governance concerns, it may no longer resemble a bounded application with a single owner and a clean operating surface.

Writing shutdown authority into legislation is far simpler than carrying that decision through a production estate shaped by years of migrations, exceptions, integrations, acquisitions, temporary fixes, and team-level decisions. That implementation gap is where the issue becomes most relevant to infrastructure teams.

For the last several years, much of the AI safety conversation has focused on acceptable use, privacy, model behavior, and human-in-the-loop oversight. Those topics still deserve attention, especially as organizations formalize where AI may be used, which data can be shared, and how employees should evaluate generated output. 

“Writing shutdown authority into legislation is far simpler than carrying that decision through a production estate shaped by years of migrations, exceptions, integrations, acquisitions, temporary fixes, and team-level decisions.”

As AI moves deeper into production workflows, the discussion also needs to involve a more practical concern: when an AI-enabled system creates unacceptable risk, can the organization understand the affected environment well enough to constrain it quickly, consistently, and with evidence?

The phrase “kill switch” may drive the public discussion, but the practical answer lives in the systems surrounding the AI capability.

The limits of a single control

Emergency stops are the kind of control most people picture when they hear the phrase “kill switch.” They make sense in physical systems. Manufacturing equipment, industrial machinery, and certain safety-critical devices can be designed with direct shutdown mechanisms. Software estates already stretch that metaphor, and enterprise AI stretches it further.

The model may be the most visible part of the discussion, although it is rarely the full surface area. An AI assistant used in software delivery might have access to repositories, CI/CD tools, artifact stores, ticketing systems, secrets, test environments, and deployment workflows. An AI agent used in IT operations might read telemetry, recommend remediation, open change requests, call automation scripts, or modify infrastructure through approved orchestration paths. 

Stopping one part of that chain can leave other paths untouched. Turning off a service may not revoke the credentials it uses. Suspending inference may leave downstream systems acting on outdated outputs. Interrupting the wrong dependency can create a separate service incident while the original risk remains only partly contained. Anyone who has worked through a security incident, emergency patch cycle, or major outage knows how quickly a clean decision turns into a sequence of technical tradeoffs.

A credible response plan must account for the system as it exists now, not as it looked during an architecture review months earlier. Infrastructure teams bring useful skepticism to that exercise because they are used to tracing scope, access, ownership, dependencies, and verification paths under pressure. They also know that many environments contain a gap between documented intent and production behavior.

Once AI is embedded in business workflows, those operational details become part of the governance conversation. They expose the places where policy language has moved faster than the infrastructure knowledge needed to make policy executable.

Before containment comes discovery

Much of the public conversation assumes organizations know where AI is running, which is a generous assumption in many enterprise environments.

AI can enter an enterprise through obvious channels, such as internally approved model providers or purpose-built applications. It also arrives through less visible paths. A SaaS product adds an AI feature. A development team experiments with an open source model. A hosted API gets attached to an internal tool. A vendor introduces an AI capability inside software the company already approved. Over time, the line between an “AI system” and a system that happens to use AI becomes harder to define.

This is where the kill switch metaphor starts to show its limits. If the relevant systems are discovered during the response, the team is already behind. Dependency questions, business impact, access paths, and evidence collection all become harder when the basic inventory is still being assembled.

“If the relevant systems are discovered during the response, the team is already behind.”

Infrastructure teams have seen versions of this problem before. During incident response, a service thought to be isolated turns out to have undocumented consumers. During a cloud migration, a supposedly unused integration is suddenly linked to a business process. During an audit, ownership records, configuration data, and actual operating conditions refuse to line up cleanly. AI adds a new category of concern, but the underlying visibility problem is familiar.

The challenge quickly expands beyond the model itself. Teams need to understand which systems call external models, where generated content influences workflows, which accounts and automation paths sit between a recommendation and an action, and how third-party AI capabilities have found their way into the environment.

Most discussions start with how to stop risky AI behavior. In many environments, the more revealing question comes earlier: can the organization produce a reliable picture of where AI touches the estate, which systems depend on it, and which workflows would keep moving if access changed?

Containment depends on the state of the system

Most infrastructure teams know that containment is less a single action than a set of operating conditions. The difficult work happens before the incident, when teams decide what should change if risk reaches a level that requires intervention.

Under ordinary circumstances, a service operates with a defined set of permissions, connections, dependencies, and logging requirements. Under restricted operation, selected assumptions change while investigators preserve evidence and determine whether the risk has been contained. A team might turn off endpoints, suspend integrations, limit external network access, revoke or rotate credentials, increase logging, or isolate workloads while the situation is being investigated.

The details vary because environments vary. That is exactly why generic answers tend to fall apart. A useful containment plan must match the systems it is intended to govern, including the dependencies that surround them and the business processes that rely on them.

Inventory sounds mundane until a response effort depends on it. In my work with infrastructure and compliance teams, I’ve repeatedly seen organizations struggle to maintain an accurate picture of their environments as cloud resources, Kubernetes clusters, SaaS services, and AI projects multiply faster than governance processes can track them during normal operations, which creates audit and support headaches. During a containment event, those same blind spots slow investigations, complicate dependency analysis, and make evidence harder to produce.

Governance eventually reaches production

Governance efforts often begin with documentation. Committees are formed to define responsibilities, agree on escalation paths, and establish a common language for discussing risk before an incident forces the issue. 

The conversation shifts once someone asks whether the control can be demonstrated. A document can describe who has authority to suspend an AI-enabled service. Still, it cannot turn off an integration, revoke a credential, increase logging, or prove that a set of systems entered a restricted condition. A risk register can identify a containment scenario, although it cannot document which nodes changed, when they changed, and whether they remain aligned with the required configuration.

Security, compliance, and infrastructure teams know this gap well. It appears in patching programs, configuration baselines, incident response exercises, supply chain reviews, and disaster recovery planning. Written controls tend to be cleaner than the real-world environments they describe. Production systems reflect years of accumulated decisions, exceptions, migrations, temporary fixes, acquired assets, and workarounds that may outlive the original reasons they existed.

“Written controls tend to be cleaner than the real-world environments they describe.”

AI increases urgency because some systems are becoming more autonomous and more connected to business workflows. It also adds outside pressure. When an incident becomes visible enough to prompt legislative action, boards and customers start asking sharper questions. A company may be able to point to an AI policy, but boards, customers, and regulators eventually want to understand exactly how that policy translates into action.

If leadership declares an AI-enabled workflow should be restricted, the discussion moves quickly from oversight to execution. Teams need to know where to intervene, which systems are affected, who owns the required changes, how completion will be verified, and what evidence remains once the response is over.

A vague answer may pass during experimental stages but becomes much harder to defend once AI is embedded in production services, regulated workflows, customer-facing systems, or environments connected to critical business operations.

Why a mandate will not solve the estate problem

A federal shutdown authority, if enacted, would place legal pressure on a narrow class of powerful AI providers. It would not remove the implementation burden for organizations that adopt, integrate, fine-tune, host, or embed AI systems inside their own environments.

Even if a major AI provider can throttle or suspend a model, each enterprise still must understand its own exposure in the context of its applications, workflows, dependencies, and operating assumptions. Which applications depend on that model? Which workflows fail open or fail closed if access is restricted? Which internal systems contain cached outputs, calculated decisions, or agent-created changes? Which business processes need manual fallback when an AI service is unavailable?

Policy debates often treat AI control as if the decisive action happens at the model layer. Sometimes it will; however, in many business environments, the risk will live in the connections that surround the model. A hosted AI service may be suspended while local workflows, scripts, integrations, and access tokens continue following their last known configuration.

“A serious AI containment strategy has more in common with mature infrastructure management than with an emergency stop button.”

A serious AI containment strategy has more in common with mature infrastructure management than with an emergency stop button. It requires an up-to-date inventory of AI-adjacent systems, a map of dependencies and access paths, predefined restricted conditions for high-risk services, tested procedures for applying those conditions, and evidence that changes were enforced. Ownership also needs to be clear, since fast action becomes difficult when authority is scattered across teams.

The work is less exotic than the public conversation can make it sound. AI-enabled systems still need to be managed as production systems with real dependencies and business impact.

Infrastructure teams belong earlier in the conversation

Spend enough time running infrastructure, and you develop a complicated relationship with documentation. Most organizations have diagrams, inventories, and governance processes, and all of them serve a purpose. The challenge is that production systems keep evolving long after those artifacts are created. Acquisitions introduce systems that do not fit cleanly into existing models. Applications gain integrations nobody anticipated during the original design process. Temporary exceptions become permanent. Cloud resources intended to live for a week are still running after a year.

Most of this happens for defensible reasons, usually in support of uptime mandates, delivery pressure, customer needs, or business continuity SLAs. The result is that operational knowledge becomes dispersed across people (some of whom will inevitably have moved on), tickets, runbooks, monitoring systems, and memory rather than living neatly in one place.

Infrastructure teams spend their days tracing dependencies, untangling ownership questions, and figuring out how systems behave outside a design review. Bringing that perspective into AI governance conversations early can prevent containment plans from depending on assumptions that did not translate into production. It also helps organizations understand the difference between disabling a model, restricting access to a service, isolating a workload, and preserving evidence during an investigation.

Scale complicates things further. A manual action that works effectively for ten systems may fail across hundreds or thousands. A change one expert can perform during business hours may become fragile if that person is unavailable when an event occurs. A runbook that looks adequate in a tabletop exercise may not survive a live environment where dependencies have changed, and the current ownership is unclear.

The phrase “kill switch” will probably remain part of the public debate because it is simple, memorable, and familiar. Practitioners do not have to accept the metaphor literally to leverage the attention it creates. They can redirect the conversation toward more useful questions: what restricted operation would mean for a given service, which dependencies would have to change, which controls can be applied reliably, which steps remain manual, and how the organization would prove the response worked.

These questions are less dramatic than a big red button, but their answers are also most likely to improve readiness.

Control starts before the incident

The Hugging Face incident gave the industry a vivid story, and Washington responded with the language of shutdown authority. That reaction is understandable. Leaders want mechanisms that sound equal to the risk, especially when the public conversation moves faster than the technical details can be explained.

By the time an organization begins thinking about containment, much of the hard work should already be done. Teams should already understand what is running, who owns it, what depends on it, and how changes will ripple through the environment.

AI can reduce certain workflow bottlenecks, but it also exposes weak inventory, unclear ownership, and brittle operating assumptions faster than many teams are prepared to handle. A future incident will not pause while teams locate assets, clarify ownership, identify credentials, or discover that a service dependency was never documented.

The current debate may be framed around new kill switches. For most organizations, the more useful work starts with building and maintaining an accurate picture of the systems, dependencies, and workflows that already exist across the estate.

The post The “AI kill switch” assumes you know what you are trying to shut down appeared first on The New Stack.

Received — 5 August 2026 ⏭ AI Infrastructure Archives - The New Stack

Google’s four AI departures: “We wanted to build something differently”

Google logo above the glass entrance to a modern office building, with pedestrians and trees outside.

At the start of 2025, investors wondered whether Google could keep pace with OpenAI. By December, Alphabet was completing its best year on the stock market since 2009, helped by growing confidence in Gemini and Google’s broader AI strategy.

Much of that work came out of DeepMind, the British AI lab Google acquired in 2014 for about £400 million ($659 million in 2014). Now, Google is changing its leadership, while four of its best-known engineers are leaving to start an automated research lab.

DeepMind’s leadership reshuffles

Google announced Wednesday that DeepMind founder Demis Hassabis will step away from the lab’s day-to-day operations to become chair of Google DeepMind and chief scientist of Alphabet. Koray Kavukcuoglu, DeepMind’s chief technology officer and Google’s chief AI architect, will take control of Gemini model development, frontier AI research, the Gemini app and its developer teams.

At the same time, Jeff Dean, Sanjay Ghemawat, Oriol Vinyals and Quoc Le are also leaving Google to start Discovery Loop. As a public-benefit corporation, the company wants to use AI to automate scientific and engineering research. Google will remain involved as a founding investor and cloud provider. Just one person will now oversee everything from research to products.

In a January interview for CNBC’s podcast The Tech Download, Hassabis described DeepMind as the “engine room” of Google’s AI efforts. Hassabis said DeepMind develops Google’s core AI technology before it is distributed across the company’s products.

To get new AI into products faster, Google had to do more than update its models. DeepMind spent years reworking Google’s infrastructure so its AI work could move out the door quicker. That infrastructure push has extended to silicon, too — Google recently bet its inference future on a chip built for one model, a sign of how tightly the company is coupling hardware to its Gemini roadmap.

Hassabis said Google never struggled to invent new technology. After all, its researchers came up with the transformer architecture, which became the backbone of large language models. The problem was turning that research into products fast enough.

And moving faster is exactly what Google did. Hassabis pointed to Gemini 2.5, which landed in March 2025, as a turning point. Gemini 3 came out in November and put Google back in the mix with OpenAI and Anthropic — both of which have been making aggressive moves of their own to capture developer share.

By January, Hassabis said he and Google CEO Sundar Pichai were speaking almost every day, sometimes adjusting product plans and research roadmaps daily.

Kavukcuoglu inherits the Gemini roadmap

Kavukcuoglu has been at DeepMind for 13 years, started the deep learning team, and worked on projects like WaveNet and DQN. Now, model research, the Gemini app, and developer products are all on his plate. The flagship version of Gemini 4 remains unreleased after a planned June launch, according to Reuters. Hassabis confirmed the model’s name in his note to employees, saying Google was making progress on Gemini 4.

Discovery Loop’s founding engineers

Dean joined Google in 1999 and helped create Google Brain before becoming a technical co-lead for Gemini. Working with Ghemawat, he developed systems including MapReduce, Bigtable, and Spanner. Dean was one of the primary designers of TensorFlow, while Ghemawat worked on the infrastructure behind Google Search and several generations of its distributed computing systems.

Vinyals and Le made foundational contributions to deep learning and Gemini. Along with former OpenAI chief scientist Ilya Sutskever, they co-authored the influential 2014 paper that introduced sequence-to-sequence learning with neural networks. Their new company plans to use that experience to work on machine learning itself, at least to start.

“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering,” Discovery Loop says on its website.

“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering.”

The idea that AI can automate scientific discovery is no longer speculative. OpenAI’s Astra recently proved 10 long-standing math and science theorems for about $2,000 in token costs — a data point that suggests Discovery Loop is entering a space where early results are already landing.

Discovery Loop is built around the idea that AI can automate the entire experimental cycle. At Y Combinator‘s Startup School in July, Dean said the system could run the entire process, from proposing and carrying out an experiment to evaluating the results and deciding what to test next.

Ghemawat told Wired that Google’s systems were designed to support products such as Search, advertising and large consumer applications. Discovery Loop wants to build specialized infrastructure around research instead.

“We wanted to build something differently than how things are built at Google right now,” he said.

“We wanted to build something differently than how things are built at Google right now.”

Google keeps a stake

Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity. That arrangement gives Discovery Loop access to the infrastructure needed to run large numbers of experiments without first building its own data centers. It gives Google a stake in anything the new company discovers and another major AI workload for Google Cloud.

Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity.

Google has already lost Gemini co-leader and transformer co-author Noam Shazeer to OpenAI and AlphaFold researcher John Jumper to Anthropic. Worth noting, Alphabet shares fell more than 5% following Wednesday’s announcement.

Hassabis, meanwhile, will focus more of his attention on long-term AGI strategy and Isomorphic Labs, the drug-discovery company spun out of DeepMind.

In his message to employees, he said AGI now feels “close at hand” and that he wants more time to influence what happens next — a sentiment that carries extra weight as some of the most powerful AI labs face growing pressure to slow down.

​

The post Google’s four AI departures: “We wanted to build something differently” appeared first on The New Stack.

❌