❌

Normal view

CoreWeave connects custom training loops to model serving

Abstract glowing digital light threads and data particles sweeping across a black background, representing AI training loops.

Production traffic shows failures that benchmarks or pre-deployment evaluations can’t fully reveal. It tells you which edge cases matter and what users need. Turning that signal into the next model version requires an upfront decision: does the model need to change, or is the problem in retrieval, context, prompts, tools, or serving?

Changing weights won’t fix a failure elsewhere in the system. When the model is the right place to intervene, production evidence can improve both quality and economics. A smaller model might reproduce a more expensive model’s behavior; better examples might improve consistency; an outcome-based reward might suit the task. Refining a capable model can also avoid another migration to a new model generation.

Choose a technique from the failure mode

Model distillation fits when a stronger model already produces the desired behavior and you want a smaller, more economical model to reproduce it. Supervised fine-tuning (SFT) fits when high-quality examples can teach more consistent behavior. Reinforcement learning (RL) fits when you can reliably evaluate an outcome but cannot readily specify the ideal response in advance.

“Changing weights won’t fix a failure elsewhere in the system. When the model is the right place to intervene, production evidence can improve both quality and economics.”

You may use all three approaches over a model’s lifecycle. Choose from the evidence in your workload, even when that means using a technique that takes more setup.

Production traces have to become training data. Post-training needs compute and orchestration, and the resulting model has to be evaluated and returned to inference. That is the work of closing the loop. In RL, inference generates rollouts during training, and new weights must return to serving throughout the run.

At Fully Connected 2026, we introduced CoreWeave Model Distillation, CoreWeave RL Rollouts on CoreWeave Inference, and a programmable training API. These workflows connect in CoreWeave Forge. The programmable API is in limited preview, with an official launch planned for later in 2026.

Keep the data fresh when the base model stays

CoreWeave Model Distillation provides a managed workflow. CoreWeave Agent Lens captures inference traffic as the application runs. The workflow turns it into training data, relabels weak outputs, tests base models and hyperparameters, deploys candidates to Serverless Inference, and compares them with the production model.

If the model already has the capabilities your task needs, fresh production data can improve its fit without changing the serving profile you know. Compare the new version against the model already running before you move traffic.

Method used this workflow to revisit a model it had already deployed. In 2024, it used OpenPipe to train Llama 3.1 8B to navigate bank interactive voice response (IVR) systems, using GPT-4o outputs. In August 2026, Method brought in fresh production data, relabeled it with GPT-5.6 Sol, and trained another version. The candidate won 57.5% of head-to-head evaluations against the deployed version.

The base model had not changed. Fresh production data and Model Distillation in Forge made another iteration possible sooner. The 57.5% result is the share of head-to-head evaluations the new version won.

Control the training recipe in Python

For teams that want to design their own learning process, the programmable training API for Serverless Training separates training logic from infrastructure operations. You write a Python loop; CoreWeave manages the compute and underlying training infrastructure.

Here’s one way to combine RL updates with periodic supervised updates on verified demonstrations. The example creates a learner and serving adapter, uses an Adam optimizer with a learning rate of 1e-5, and publishes updated weights to the sampler after each step.

Python
learner = await training.create_learner(model=model, lora=lora)
sampler = await serving.create_adapter(name="sampler")
optimizer = AdamConfig(learning_rate=1e-5)

for step in range(num_steps):
    trajectories = await run_agent(sampler, tasks)
    update = learner.batch()
    update.forward_backward(
        build_rl_batch(trajectories), loss="importance_sampling"
    )
    update.optim_step(optimizer=optimizer)

    if (step + 1) % sft_interval == 0:
        update.forward_backward(next_demo_batch(), loss="cross_entropy")
        update.optim_step(optimizer=optimizer)

    update.publish_weights(to=sampler)
    await update.result()

await (await learner.save_state("final")).result()
await learner.close()

The rollout and batch helpers are illustrative. Your agent logic, reward calculations, and data preparation belong in those helpers. The loop uses importance-sampling loss for RL and cross-entropy loss for demonstrations, then waits for the update result. It saves the final learner state and closes the learner.

You choose how often demonstrations enter the loop, how rewards are calculated, and how the recipe evolves. SFT, RL, and distillation share the training primitives, while the platform runs the distributed systems underneath.

Willow began with supervised learning, then added RL scorers to reduce filler words, improve readability, and remove self-corrected phrases. The scorers were combined into a reward function. Lawrence Liu, Willow’s chief technology officer and co-founder, reported better text styling, emoji handling, and responsiveness to commands than in the deployed model.

Remove checkpoint reload delays from RL

RL serving needs enough rollout generation, alignment with the policy being trained, and timely weight updates. The trainer, reward logic, and environment can stay where they are while CoreWeave Inference handles this part of the loop.

CoreWeave RL Rollouts is built on Nvidia Dynamo with vLLM support. When the trainer writes a checkpoint, the service hot-loads it into a live deployment while preserving in-flight requests and endpoint availability. It returns training metadata: token log probabilities (logprobs), token IDs, and expert routing for mixture-of-experts (MoE) models.

“When the trainer writes a checkpoint, the service hot-loads it into a live deployment while preserving in-flight requests and endpoint availability. Serving uses the trainer’s current policy weights, removing staleness and redeployment delays that would otherwise hold up the next batch of rollouts.”

Checkpoints can be full snapshots or incremental deltas, reducing synchronization after the first upload. Serving uses the trainer’s current policy weights, removing staleness and redeployment delays that would otherwise hold up the next batch of rollouts.

Measure quality and infrastructure separately

We validated the workflow with Nvidia and you.com by post-training Nvidia Nemotron 3.5 Lightning for web search. The run combined SFT and RL with you.com’s search tools inside the training loop.

BrowseComp accuracy increased from a 36.97% baseline to 45.45%, a reported 22.94% relative improvement. Average tool calls fell 30.24%. Separately, hot-loading checkpoints into a live deployment was approximately 15 times faster than traditional redeployment cycles.

Better search tooling, post-training refinement, and harness improvements worked together to improve accuracy and reduce tool calls. Hot-loading cut the time spent waiting for checkpoint synchronization. If you’re trying to make a smaller or more efficient model competitive with models an order of magnitude larger, that extra training time matters too.

Keep the model connected to its experiment

CoreWeave Forge connects production inference, training, and serving again. Model Distillation supplies a managed path. The programmable API lets researchers control the training loop without managing its infrastructure. RL Rollouts supports teams keeping their own trainer while using CoreWeave for rollout inference.

Models produced by serverless training return as tagged artifacts that can move into Serverless Inference without a separate registry handoff. Experiments remain logged beside them, keeping the run, model, and eventual deployment connected.

“Choose a failure you can reproduce, use it to guide the training method, and compare the candidate with the model serving users.”

Choose a failure you can reproduce, use it to guide the training method, and compare the candidate with the model serving users. Keep the production examples, evaluation, and resulting artifact together so your team can use them again.

The CoreWeave Forge blog and announcement press release provide context on the capabilities introduced at Fully Connected 2026. CoreWeave also plans further engineering explanations of Model Distillation, the programmable training API, and RL Rollouts.

The post CoreWeave connects custom training loops to model serving appeared first on The New Stack.

CoreWeave tests AI integrations on its cloud before you migrate

Futuristic digital visualization of global trade data.

Moving an AI workload to another cloud means bringing more than a model and its data. Your serving layer, observability dashboards, security controls, and storage systems have to work together on the new infrastructure. For platform teams, proving those connections under real traffic can become the slowest and riskiest part of the move.

As a stack grows across vendors and clouds, each project adds more integration questions. A compatible specification is a starting point. It doesn’t guarantee a tool will behave as expected in production, or that the runbooks and security workflows built around it will carry over.

Make production validation visible

An integration needs to work on the infrastructure where you plan to run it, under production traffic. The tools should also be ones your team already uses or has asked for.

“A compatible specification is a starting point. It doesn’t guarantee a tool will behave as expected in production, or that the runbooks and security workflows built around it will carry over.”

We built the CoreWeave Partner Network to make that validation visible. It brings relationships with CrowdStrike, VAST Data, Reflection, IBM, and other technology vendors into a curated network of integrations tested under real-world production conditions on CoreWeave Cloud.

The network covers four parts of the AI stack, from training and inference to data movement, security, and the next iteration:

  • Infrastructure: compute, storage, networking, servers, and interconnects, with partners including VAST Data, Dell, AMD, IBM, BackBlaze, and Pulumi.
  • Data services: storing, processing, moving, and accessing AI data throughout its lifecycle, with ClickHouse, Encord, LanceDB, and Datology.
  • Independent software vendors (ISVs): orchestration, observability, and model development, including Grafana, Datadog, Union, SkyPilot, IBM, Pinecone, Redis, LlamaIndex, and Clockwork. A dedicated security segment includes CrowdStrike, Zscaler, and Cranium.
  • Models, agents, and inference: foundational software and services for building, training, serving, and scaling applications, with Reflection, RadixArk, Inferact, OpenRouter, and Red Hat.

Together, these integrations support running and observing models, curating data, improving behavior, and evaluating the next release. Every partner must meet the same requirement: an integration that works on CoreWeave infrastructure under production conditions. Partners enter the network because customers run them or ask for them.

Read the workload evidence

IBM trained its Granite models on CoreWeave around one of the first deployments of Nvidia GB200 NVL72 clusters. Cohere built North, its agentic AI platform, on one of the industry’s first production GB200 NVL72 clusters and reported training three times faster than on Nvidia Hopper.

Mistral AI trains its models entirely on CoreWeave and cites training 2.5 times faster on Nvidia GB200 NVL72 systems. Decart uses Nvidia HGX B200 at production scale through its CoreWeave collaboration, on a platform that includes Weights & Biases.

The underlying relationships predate the network announcement. CoreWeave and Weights & Biases have run production integrations with Encord, Pinecone, and other partners since the early wave of open-source large language models (LLMs) and frontier-model training.

Jon Jones, CoreWeave’s Chief Revenue Officer, describes the change as making existing partner work visible before customers build. The network also provides a repeatable way to pursue deeper integrations, joint go-to-market work, and shared innovation.

The partners emphasize different parts of that system. VAST Data co-founder and chief technology officer Alon Horev describes combining CoreWeave’s AI cloud with the VAST AI Operating System across large-scale training, fine-tuning, production inference, and continuous agentic workloads. CrowdStrike CEO and founder George Kurtz emphasizes security built into the infrastructure running AI. Adam Lawrence, general manager of IBM Americas, points to IBM and Red Hat technologies as a way to expand choice and reduce integration work when deploying and scaling AI.

Carry over the operations you already use

Your team has already built runbooks around its observability tools. Grafana and Datadog integrations let you carry that work over. CrowdStrike, Zscaler, and Cranium provide security integrations, bringing existing threat visibility and response tooling.

Inference integrations serve a similar purpose: keeping the models available as teams expand onto CoreWeave, without rebuilding the serving layer. The goal is to add capacity while retaining the parts of the stack that already work.

“A marketplace listing still leaves your team to validate the integration. An LLM can suggest what’s popular without telling you what works on your infrastructure.”

A marketplace listing still leaves your team to validate the integration. An LLM can suggest what’s popular without telling you what works on your infrastructure. We select partners based on customer demand and test their integrations across all four segments.

Include the tools your agents call

The integration problem also reaches inside agent execution. Most closed models include web search, while most open-weight models do not. Teams choosing open weights for cost, control, or customization have often had to connect a search provider separately.

CoreWeave is extending its partner approach to these tools, beginning with Exa, Parallel Web Systems, and You.com. The announced search layer provides access through one integration instead of a separate contract for each provider.

Teams can enable it through a single setting on a standard inference request or access it from inside a CoreWeave Sandbox, without setting up a separate software development kit (SDK) or account. The arrangement is nonexclusive, so using the integration does not restrict future provider choices.

We announced the search partnerships at Fully Connected, with the remaining integration work planned for the following weeks. As those integrations become available, they will meet the same production-validation standard as the rest of the network.

Start with the connections your workload needs

Before adding another cloud, check the serving, data, observability, and security connections your workload depends on. Ask which integrations have been tested under production conditions and what your team will still need to validate.

“Before adding another cloud, check the serving, data, observability, and security connections your workload depends on.”

The CoreWeave Partner Network lists validated partners. CoreWeave account teams can discuss a specific integration and workload, and partner sessions from Fully Connected 2026 offer another way to see the integrations in use. Teams interested in new collaborations can explore the partner program.

The post CoreWeave tests AI integrations on its cloud before you migrate appeared first on The New Stack.

Your agent is only as good as your infrastructure

Dark abstract 3D glass ribbon rendering symbolizing complex AI agent infrastructure and bursty data workflows.

You built a great agent, but something happened when it moved into production.

In testing, your agent reviewed pull requests efficiently on its own. It read the diff, grepped the codebase for related usages, ran the test suite, checked whether CI was still red from an earlier commit, and drafted a comment—all before you’d finished reading the diff yourself.

In production, however, imagine the same five steps ran behind every other PR review your team’s agents performed that hour. Some reviews landed in seconds; others sat for minutes because the test-suite step landed on a node that was mid-burst from someone else’s agent. 

The agent didn’t change. The execution environment did, and that’s what decided whether review time held steady or crept up.

Agentic applications introduce a different execution pattern than traditional chat applications. As those workflows become longer and more dynamic, infrastructure has a much larger influence on latency, reliability, and cost than it does for a simple chatbot.

That’s why your agent is only as good as your infrastructure.

Agents aren’t chatbots with more steps

The difference between serving inference for a chatbot vs. an AI agent isn’t simply that one is “more capable.” They execute work differently:

  • A chatbot usually makes one inference call per user message. The model receives a prompt, generates a response, and waits for the next user input before proceeding.
  • An agent executes the entire workflow, turning one user message into a chain of inference calls. It might decide to search documentation, retrieve data from a database, call an API, execute code, evaluate the result, and then repeat that process before producing an answer. Each of those decisions may trigger another inference call, and every result becomes additional context for the next step.

“Infrastructure has a much larger influence on latency, reliability, and cost than it does for a simple chatbot.”

That execution model changes the infrastructure requirements for AI agents. 

One question, many steps behind it

Instead of optimizing for individual inference requests, the system has to support long-running workflows whose latency and reliability depend on every component in the chain.

A single user request often expands into a sequence of inference and tool execution steps, sometimes called multi-turn tool calls or the agentic loop. Rather than generating one response, the model alternates between reasoning and interacting with external systems.

For example, you ask an agent why checkout latency spiked overnight. The agent pulls the deploy log, queries the monitoring system, runs a diagnostic against the connection pool, weighs whether the culprit is a bad deploy or a capacity issue, and then folds that into another inference call before finally producing a full-fledged response.

Each reasoning step becomes another inference request, and every tool result is added to the model’s context before the next step.

This workflow changes what reliability means

Multi-turn workflows are inherently sequential, which is why even low latency can quickly add up to a significant amount. Every inference step waits for the previous one to finish. If a database query takes two seconds, the model can’t begin the next reasoning step until that result returns. The model may generate tokens quickly, but the other steps slow it down.

“In this agentic workflow, every step in the chain has to hold, because the chain is only as strong as its slowest link.”

In this agentic workflow, every step in the chain has to hold, because the chain is only as strong as its slowest link. Instead of processing isolated inference requests, the inference stack has to orchestrate a chain of dependent model invocations and external tool calls. As those workflows become longer, the stack increasingly determines how quickly, reliably, and cost-effectively the application performs.

But the user doesn’t see an orchestration hiccup. They see an agent that hung or gave up.

That’s why end-to-end agent reliability depends on much more than model quality. Infrastructure determines whether each step has the resources it needs to execute predictably under load.

Why the bill and the performance both feel unpredictable

A second difference in agentic workflows catches teams off guard: demand patterns and their impact on your inference bill. 

Most inference services, and the pricing built on top of them, assume traffic arrives at a predictable pace. A typical inference solution knows the predictable demand pattern: User traffic increases, request volume increases, and capacity scales accordingly. Cloud infrastructure is typically optimized for these steady request patterns using mechanisms such as autoscaling, load balancing, and capacity planning.

Agent workloads don’t behave that way. Individual workflows pause while waiting on external systems, then resume as soon as new information becomes available.

  • The pause: The agent waits on an external API or database, so the GPU serving that workflow has no inference work to perform, and its accumulated context may be evicted from GPU memory while it waits
  • The burst: As soon as external systems return results, inference resumes simultaneously across many workflows, creating short and sharp spikes in GPU demand, each re-processing its full accumulated context

If you’re watching GPU utilization and it looks less like steady load and more like a heartbeat — flat, then a spike every time tool results come back — that’s the signature. It means you’re provisioning for the average when you should be provisioning for the peak, and it’s usually the first place p99 latency quietly blows out.

“If you’re watching GPU utilization and it looks less like steady load and more like a heartbeat, that’s the signature.”

Inference services designed around steady or predictable request streams can struggle to allocate resources efficiently under these conditions. This leads to inconsistent latency, GPU underutilization, or higher operating costs.

When performance becomes unpredictable, or a bill doesn’t match what you expected, it’s evidence your infrastructure was built for a different workload than the one you’re actually running.

What infrastructure built for agents actually looks like

Agentic applications place different demands on infrastructure than traditional AI workloads: long dependency chains, bursty demand, and continuous evolution. Because of this, the infrastructure is deciding whether the chain holds, and whether the bill holds too.

For an agent to run, it needs infrastructure purpose-built to support:

Performance that holds across the whole chain. The infrastructure must keep latency consistent across multi-step and multi-tool workflows.

Scalability that responds to bursty demand. Infrastructure should scale quickly as inference demand fluctuates, without requiring capacity to remain provisioned during idle periods.

  • Predictable economics even for dynamic workloads. The infrastructure bill should reflect actual usage.
  • None of this makes bursty demand disappear, but it changes how the system absorbs it. A large enough simultaneous burst, or a workflow that accumulates enough context before pausing, still costs something. The goal isn’t zero cost or zero limit; it’s making both predictable.

Your agent is only as good as your infrastructure. Get it right, and your agent’s responsiveness, reliability, and cost-effectiveness will improve your work. 

Learn more about how infrastructure can be purpose-built for agentic workflows: Check out the documentation to get started.

The post Your agent is only as good as your infrastructure appeared first on The New Stack.

❌