❌

Reading view

Keep Distributed Training Running with Automated Workload Recovery

As distributed AI training scales, failures across hardware and software dependencies increasingly disrupt training progress, making fast, reliable recovery...

As distributed AI training scales, failures across hardware and software dependencies increasingly disrupt training progress, making fast, reliable recovery essential. A stalled training rank can trigger an NVIDIA Collective Communications Library (NCCL) timeout, a GPU node can stop responding to the scheduler, or a straggler can slow the entire job. Each disruption is manageable on its own…

Source

πŸ’Ύ

  •  

Making Kubernetes AI-Ready for NVIDIA Vera Rubin on Day One

Configuring and optimizing Kubernetes for new GPU hardware such as NVIDIA Vera Rubin is complex and error-prone. Drivers, kernels, runtimes, networking,...

Configuring and optimizing Kubernetes for new GPU hardware such as NVIDIA Vera Rubin is complex and error-prone. Drivers, kernels, runtimes, networking, operators, and schedulers must work together, with settings tuned to the hardware and workload. Small mismatches can prevent workloads from running or limit performance. The NVIDIA AI Cluster Runtime (AICR)β€”part of the NVIDIA DSX AI Factory…

Source

  •  

Support New AI Frameworks Faster Using NVIDIA Workload Map

NVIDIA DSX AI Factory Platform brings together technologies to help the ecosystem design, build, and operate more efficient AI factories, including open source,...

NVIDIA DSX AI Factory Platform brings together technologies to help the ecosystem design, build, and operate more efficient AI factories, including open source, modular software components built to support multi-tenant operations and scaling. Operating an AI factory requires a shared understanding of distributed workloads: their components, pod templates, replica counts, and status.

Source

  •  

Build a Validated Kubernetes Runtime for NVIDIA Dynamo

Glowing green AI network illustration in front of server racks.Teams building AI applications need a repeatable path from a model configuration to a working inference endpoint. NVIDIA Dynamo is a distributed inference...Glowing green AI network illustration in front of server racks.

Teams building AI applications need a repeatable path from a model configuration to a working inference endpoint. NVIDIA Dynamo is a distributed inference framework that coordinates model serving across GPU workers. Deploying it on Kubernetes is only a first step. A successful model deployment requires more: GPU drivers, resource allocation, scheduling, networking, and monitoring must all work…

Source

  •  

How to Check and Configure NVIDIA GPUDirect Storage for Your Platform

Data center server racks with network cables connecting rack-mounted equipment.NVIDIA GPUDirect Storage (GDS) accelerates GPU storage access in on-premises and cloud environments. It provides a robust, portable, and efficient data path...Data center server racks with network cables connecting rack-mounted equipment.

NVIDIA GPUDirect Storage (GDS) accelerates GPU storage access in on-premises and cloud environments. It provides a robust, portable, and efficient data path between NVIDIA GPUs and storage targets to increase bandwidth, minimize latency, and reduce CPU and memory utilization across a broad range of platforms, application frameworks, and workload patterns. Configuring NVIDIA GPUDirect Storage…

Source

  •  

Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4

Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads...

Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads with the infrastructure they already have. NVIDIA GPUs already support several approaches to sharing. Multi-instance GPU (MIG) splits a GPU into hardware-isolated instances that are transparent to the application…

Source

  •  

How NVIDIA Minerva Turns Telemetry Chaos Into Peak GPU Yield

AI Factories are some of the most technologically complex operational environments ever created. Hundreds of thousands of GPUs, NVIDIA Spectrum-X Ethernet...

AI Factories are some of the most technologically complex operational environments ever created. Hundreds of thousands of GPUs, NVIDIA Spectrum-X Ethernet scale-out, scale-across and scale-in fabrics, NVIDIA NVLink scale-up fabrics, distributed schedulers, and multi-week training runs operate in continuous lockstep.These layers function as one production system even when different teams own their…

Source

  •  

How NVIDIA Mission Control 2.5 Brings Autonomous Recovery to Kubernetes

Training large-scale AI models on thousands of GPUs is a high-stakes operation where monitoring and automation must be built-in to optimize cluster resiliency...

Training large-scale AI models on thousands of GPUs is a high-stakes operation where monitoring and automation must be built-in to optimize cluster resiliency and uptime at scale. This concept isn’t new to data center operators who have measured mean-time-to-repair (MTTR) and mean-time-between-failure (MTBF) long before AI factories existed. Now the stakes are higher. Modern AI factories run…

Source

  •  

Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2

The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and...

The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and end-to-end performance tuning within TensorRT by customizing different optimizations for different models. For more details, see Part 1 of this series, which introduces the challenges associated with scheduling (a critical stage in deep learning…

Source

  •  

Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For...

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For example, when NVIDIA TensorRT builds an engine, it makes interdependent decisions about graph-IR transformations, fusion, implementation selection, memory placement, and quantization. This process is called scheduling.

Source

  •  

How to Read Data into GPU Memory in Python with NVIDIA GPUDirect Storage

Deocrative image.GPU applications are processing ever-growing datasets with increasingly powerful accelerators. According to the 2025 Stanford AI Index, compute for notable AI...Deocrative image.

GPU applications are processing ever-growing datasets with increasingly powerful accelerators. According to the 2025 Stanford AI Index, compute for notable AI models has been doubling approximately every five months, while the size of datasets used to train LLMs has been doubling approximately every eight months. As computation and data scale together, moving data efficiently from storage becomes…

Source

  •  

Design Proteins with RFD4-Proteina Using Reward-Guided Inference-Time Search

Protein design is a search problem. Choose amino acids for an eight-residue protein sequence and more than 25 billion combinations are possible. Choose amino...

Protein design is a search problem. Choose amino acids for an eight-residue protein sequence and more than 25 billion combinations are possible. Choose amino acids for a 100-residue protein sequence and the number of possibilities exceeds the number of atoms in the observable universe. Somewhere in that space is a protein that opens a path to a new therapy. There are too many combinations to…

Source

  •  

Developing an AI Factory Digital Twin with an Agent-Assisted Workflow

Building an AI factory digital twin requires developers to bring site plans, equipment assets, and engineering requirements into a consistent model that teams...

Building an AI factory digital twin requires developers to bring site plans, equipment assets, and engineering requirements into a consistent model that teams can inspect, revise, and prepare for simulation. Connecting these inputs into an editable workflow helps teams explore design alternatives and incorporate engineering feedback as the design evolves. The NVIDIA Omniverse DSX Blueprint…

Source

  •  

Achieve Productive AI Capacity Faster and More Predictably with NVIDIA DSX POD

AI factories turn energy into intelligence. But before an AI workload can produce its first token, operators must bring an interconnected system of compute,...

AI factories turn energy into intelligence. But before an AI workload can produce its first token, operators must bring an interconnected system of compute, networking, storage, power, cooling, and software online as one productive unit. In a conventional data center build, teams must first select the site, design power and cooling for the deployment, lay out the data center…

Source

  •  

The neocloud shakeout is coming, and Nvidia has the most to lose

After stock crashes and Firmus's scrapped IPO, industry executives expect only seven to ten of the hundreds of AI cloud providers to survive. CoreWeave lost a quarter of its market value, and Microsoft pulled out of a deal with Nscale for 23,000 Nvidia chips. Nvidia has a lot riding on the outcome, since customers outside the hyperscalers make up 45 percent of its data center revenue.

The article The neocloud shakeout is coming, and Nvidia has the most to lose appeared first on The Decoder.

  •  

NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform

NVIDIA is making the full NVIDIA Run:ai platform available as open source software, giving customers, developers, and ecosystem partners access to...

NVIDIA is making the full NVIDIA Run:ai platform available as open source software, giving customers, developers, and ecosystem partners access to production-grade AI workload and GPU orchestration capabilities. This release expands the NVIDIA DSX AI Factory Platform by establishing NVIDIA Run:ai as an open foundation that enables the broader ecosystem to bring differentiated solutions to market…

Source

  •  

Scale Bitwise-Deterministic Pretraining with NVIDIA Megatron Core

Bitwise determinism makes large-scale pretraining easier to debug, validate, and resume reproducibly. These benefits become especially valuable when training...

Bitwise determinism makes large-scale pretraining easier to debug, validate, and resume reproducibly. These benefits become especially valuable when training models with trillions of parameters across thousands of GPUs, where multiple parallelism dimensions, low-precision computation, and distributed checkpointing complicate failure reproduction and fix validation.

Source

  •  

Validate AI Factory Changes with Digital Twins and AI Agents

Three monitors display a network simulation and infrastructure monitoring dashboards in front of data center server racks.AI factories are some of the most complex operations in the world, combining GPUs, CPUs, switches, DPUs, and SuperNICs alongside schedulers, orchestration...Three monitors display a network simulation and infrastructure monitoring dashboards in front of data center server racks.

AI factories are some of the most complex operations in the world, combining GPUs, CPUs, switches, DPUs, and SuperNICs alongside schedulers, orchestration services, security controls, and a rapidly changing software stack. Deploying this infrastructure effectively is a challenge. But so is being able to validate that the infrastructure, software, and policies work together for the workloads the…

Source

  •  

How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack

GPU applications increasingly need networking and data movement to behave like first-class GPU-controlled operations rather than host-driven services. When the...

GPU applications increasingly need networking and data movement to behave like first-class GPU-controlled operations rather than host-driven services. When the CPU sits in the middle of every network transaction, it becomes a bottleneck on the critical path, adding latency and limiting how efficiently distributed applications can respond in real time. NVIDIA DOCA GPUNetIO…

Source

  •  
❌