Skip to content
AIBRAINWORX / THE INDEPENDENT AI LABRESEARCH · HARDWARE · OPEN INTELLIGENCE
AI News / Model Watch

NVIDIA Releases Qwen3.5-0.8B-Qwen3.8-27B-Draft for Speculative Decoding

NVIDIA published a 0.75B external speculative decoding draft model for Qwen3.8-27B, highlighting Groq 3 LPU deployment alongside vLLM, SGLang, and TRT-LLM compatibility.

Official project or press image for nvidia-qwen3-5-0-8b-qwen3-8-27b-draft-speculative-decoding
Official image from huggingface.co; identification only, no endorsement.
Event / context: 2026-11-11. Source check: October 28, 2026.

NVIDIA has published nvidia/Qwen3.5-0.8B-Qwen3.8-27B-Draft on Hugging Face, an external speculative decoding draft model built to accelerate inference for Alibaba's Qwen3.8-27B target model. While the repository carries a November 11, 2026 upload timestamp, NVIDIA's model card lists a November 17, 2026 Hugging Face release date.

How the External Draft Model Works

Built on the Qwen3_5ForCausalLM architecture from Qwen/Qwen3.5-0.8B, the checkpoint totals 0.75 billion parameters, split between 0.25 billion embedding parameters and 0.50 billion non-embedding parameters. Unlike a lightweight Multi-Token Prediction (MTP) head attached directly to a target model, this release is a standalone language model that shares Qwen3.8-27B's 248,320-token vocabulary and tokenizer, with its input embedding tied to its output head.

At serving time, the draft autoregressively proposes candidate token blocks under the target model's serving policy (temperature 1.0, top-k 20, top-p 0.95). Qwen3.8-27B verifies each block through rejection sampling and resamples rejected tokens from the residual distribution, aiming to reduce decode latency without altering the target model's output distribution.

Training, Hardware Targets, and Licensing

NVIDIA trained version 1.0 of the draft using NVIDIA Model Optimizer (nvidia-modelopt v0.48.0rc0) at a sequence length of 12,288 tokens. Rather than using original dataset completions, NVIDIA sampled 199,998 prompts under 3,072 tokens across nine categories of Nemotron-Post-Training-Dataset-v2—STEM, code, math, chat, and five non-English languages—and synthesized target responses with Qwen3.8-27B, recording its top-20 output distribution at each position.

Notably, the model card identifies NVIDIA Groq 3 LPU (Language Processing Unit) hardware as the test hardware and intended deployment target, where drafts are compiled for the LPU runtime. The Safetensors checkpoint also loads in standard GPU runtimes including vLLM, SGLang, and TRT-LLM, with NVIDIA Blackwell compatibility listed for Linux environments.

Although both the Qwen3.5-0.8B base model and the Qwen3.8-27B target model carry Apache 2.0 licenses, NVIDIA distributes this draft checkpoint under the NVIDIA Open Model Agreement, permitting commercial and non-commercial use under NVIDIA's terms rather than standard OSI-approved licensing.

Reported Benchmarks and Tradeoffs

According to NVIDIA's self-reported evaluation on SPEED-Bench at a draft block size of 16, the 0.75B draft achieved an overall average acceptance rate of 3.92 tokens per target step on the 1,138-turn qualitative subset, edging out Qwen3.8-27B's built-in 0.42B MTP head at 3.81. NVIDIA reported higher acceptance rates for the external draft across multilingual (4.67 vs. 4.38), reasoning (4.12 vs. 3.93), humanities (3.79 vs. 3.61), math (3.79 vs. 3.63), summarization (4.26 vs. 4.13), and coding (3.79 vs. 3.78), while trailing in roleplay (3.10 vs. 3.16) and writing (3.22 vs. 3.29).

On SPEED-Bench's long-context throughput-32k subset covering 1,536 prompts between 33,000 and 46,000 tokens, the external draft dropped to a 3.60 average acceptance rate compared with 3.86 for the built-in MTP head—a gap visible across low-entropy, mixed, and high-entropy prompts, and one that reflects the draft's 12,288-token training ceiling. Within NVIDIA's three-draft comparison, a larger 4.21B Qwen3.5-4B draft reached 5.58 qualitative and 5.11 long-context acceptance, while a 1.48B Llama-3.2-1B draft scored 3.42 and 3.07.

NVIDIA has not published independent third-party throughput measurements alongside the card, and actual end-to-end latency gains will depend on serving stack overhead and hardware topology. NVIDIA also cautions that inherited web-crawled pretraining data can produce inaccuracies, bias, or undesirable text, advising system-level validation prior to production deployment.

Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-17.

Follow your curiosity.

Pass the signal.