At a glance: NVIDIA automates workflows in Physical AI research through new Agent Skills that make scene reconstruction, data generation, and policy training for autonomous vehicles, robotics, and Vision AI scalable.
NVIDIA has released new Agent Skills at CVPR that help developers automate workflows in physical AI research – from 3D reconstruction of real-world scenes to the generation of edge cases for autonomous vehicles and robotics.
The core challenge in Physical AI research is not solely about stronger models, but about integrating a complete workflow: reconstruction of real-world scenes, generation of edge-case scenarios, policy training, behavior evaluation, and rapid iteration. These steps are currently distributed across separate tools, which slows down experiments.
NVIDIA combines its new Agent Skills with Cosmos 3 – its proprietary foundation model for Physical AI and the first omnimodel that unifies vision reasoning, world generation, and action generation. For autonomous vehicles, NVIDIA thereby addresses the “long-tail” problem: rare driving situations, unusual road geometries, and lighting changes that are difficult to capture in real-world data. The Neural Reconstruction Skills enable fleet data to be transformed into editable 3D scenes for simulation and synthetic data generation. Technologies such as InstantNuRec enable rapid 3D Gaussian reconstruction of street scenes without scene-specific optimization.
The open-source framework AlpaGym connects policy rollouts and high-fidelity simulation through Agent Skills and scales across thousands of GPUs. The generative world model OmniDreams adds photorealistic real-time rendering. Alpamayo 2 Super, a 32-billion-parameter Vision Language Action (VLA) model, extends reasoning, planning, and action capabilities across the entire driving stack.
For Vision AI systems, the new Metropolis Skills solve the data problem in generating controlled examples: they support synthetic scenario generation, anomaly creation, data augmentation, and pseudo-labeling. This is central to zero-shot anomaly detection and few-shot defect detection in industrial image processing.
Source: blogs.nvidia.com · Published June 3, 2026
Lumi AI News — AI-assisted curation pursuant to Article 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.2.9.