
NVIDIA has updated its open-weights nvidia/Cosmos3-Nano repository on Hugging Face, bringing a 16-billion-parameter omnimodal world foundation model to robotics, autonomous vehicle, and industrial simulation developers. While the Hugging Face page carries a November 12, 2026 timestamp, the model card itself records a May 31, 2026 release across both the Hugging Face Cosmos 3 collection and NVIDIA's GitHub repository. Positioned within the broader NVIDIA Cosmos platform for Physical AI, Cosmos3-Nano is engineered to understand, simulate, and predict physical environments by unifying text, image, video, audio, and robotic action trajectories inside a single downloadable checkpoint.
Omnimodal World Modeling for Physical AI
Unlike conventional multimodal assistants that only produce text from visual prompts, Cosmos3-Nano targets Physical AI pipelines where machines must interpret and simulate physically coherent interactions. According to NVIDIA's model card, the checkpoint supports world understanding, world generation, simulation, future prediction, action reasoning, and embodied policy learning across robotics, autonomous vehicles, and smart space environments, including industrial and factory-scale deployments. Rather than separating perception from synthesis across isolated models, Cosmos3-Nano operates in both Reasoner and Generator configurations. It ingests combinations of text, images, video, audio, and structured control trajectories, then synthesizes discrete language outputs or continuous audiovisual and physical action sequences.
Mixture-of-Transformers Architecture and Dual-Mode Specifications
Under the hood, Cosmos3-Nano uses a 16-billion-parameter Mixture-of-Transformers (MoT) architecture developed on the Cosmos Framework. NVIDIA pairs two complementary transformer towers inside the checkpoint: an autoregressive transformer performs standard next-token decoding for text, while a diffusion transformer synthesizes continuous modalities—images, video, audio, and action trajectories—through iterative denoising. This split preserves dedicated generation mechanics for each data type while sharing multimodal conditioning. In Generator mode, Cosmos3-Nano accepts 1D text up to 4,096 tokens, 2D images (JPG, PNG, JPEG, or WebP), 3D MP4 video at 256p, 480p, or 720p across 16:9, 4:3, 1:1, 3:4, and 9:16 aspect ratios (capped at five input frames), stereo 48 kHz audio up to 0.5 seconds muxed inside MP4, and 16 to 400 frames of JSON action sequences. Generator outputs scale from 5 to 400 video frames—defaulting to 189 frames—alongside JPEG frames, 48 kHz stereo AAC audio muxed into MP4, JSON action lists, and text. In Reasoner mode, the context window expands up to 256,000 tokens for text, image, or 4 fps MP4 video inputs, generating 4,096 or more output tokens that can include structured chain-of-thought reasoning, 2D and 3D point localization, and bounding-box coordinates.
Embodied Action Spaces and Robotics Compatibility
A defining engineering detail in Cosmos3-Nano is its explicit parameterization of robotic and agent kinematics. Action trajectories are formatted as two-dimensional arrays shaped by frame count and embodiment dimensionality, encoding per-frame joint positions, gripper states, or camera poses. NVIDIA restricts action conditioning and generation to ten supported embodiments: general camera motion (9D), autonomous vehicle control (9D), UMI (9D), single Franka Panda arm with RobotiQ gripper (10D), UR (10D), Google robot (10D), WidowX 250 (10D), dual Franka Panda arms with RobotiQ grippers (20D), Agibot (29D), and egocentric motion (57D). For robotics engineers, pairing a 256K-token visual-spatial Reasoner with 400-frame action-conditioned video synthesis offers a unified way to prototype manipulation rollouts or scene predictions before deploying policies onto physical hardware.
Runtime Stack, Hardware Support, and Developer Workflow
NVIDIA packages Cosmos3-Nano in Safetensors format with documented integration across PyTorch, Hugging Face Diffusers, vLLM, vLLM-Omni, SGLang, and sglang-diffusion. Hugging Face's auto-generated Diffusers snippet illustrates loading the checkpoint in bfloat16 precision via DiffusionPipeline.from_pretrained on CUDA devices, though the hub interface notes that dedicated Cosmos library snippets are not yet populated and directs developers to the repository files for full usage. Execution is designed and optimized for NVIDIA GPU-accelerated systems using CUDA libraries, with documented microarchitecture compatibility explicitly covering NVIDIA Ampere and NVIDIA Blackwell hardware. Supporting both high-throughput serving stacks like vLLM-Omni and SGLang alongside Diffusers lets teams test simulation and spatial reasoning inside familiar inference workflows.
OpenMDW1.1 Licensing, Input Constraints, and Verification Caveats
Cosmos3-Nano is distributed globally under the OpenMDW1.1 license (openmdw1.1-license), which NVIDIA states is ready for both commercial and non-commercial use. Even with permissive access, developers face strict technical boundaries. Generator visual inputs require 8-bit-per-channel sRGB color imagery and explicitly reject grayscale frames, while video and audio conditioning are tightly limited to five frames and half-second clips. Crucially, although the model card's table of contents lists overall reasoning and generation benchmark sections, quantitative benchmark scores and independent evaluations are absent from the inspected summary, leaving trajectory accuracy, physical fidelity, and failure rates unverified here. Engineering teams evaluating Cosmos3-Nano for safety-critical robotics or autonomous driving must therefore validate outputs rigorously in domain-specific testing—especially when target robots differ from the ten fixed action schemas.
Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-19.



