
NVIDIA has published Qwen3.5-4B-Qwen3.8-27B-Draft on Hugging Face, introducing a standalone 4.21-billion-parameter external draft model designed to cut decode latency when serving Alibaba's Qwen3.8-27B. While the Hugging Face repository timestamp records publication on November 11, 2026, the model card itself lists a November 17, 2026 release date.
How the External Draft Architecture Works
Rather than attaching a small speculative head directly to the target model, NVIDIA built a complete causal language model (Qwen3_5ForCausalLM) derived from Qwen/Qwen3.5-4B that shares Qwen3.8-27B's 248,320-token vocabulary and tokenizer. Of its 4.21 billion total parameters, 0.64 billion sit in tied input-embedding and output-head matrices and 3.57 billion in non-embedding layers.
Trained with NVIDIA Model Optimizer (nvidia-modelopt v0.48.0rc0) at a 12,288-token sequence length, the draft learned from 199,998 single-turn prompts sampled across nine categories of Nemotron-Post-Training-Dataset-v2 (STEM, code, math, chat, and five European and Asian languages). For every response position, Qwen3.8-27B generated synthetic outputs and NVIDIA recorded the target's top-20 serving distribution as the training signal. At inference time, the draft proposes candidate token blocks probabilistically (temperature 1.0, top-k 20, top-p 0.95) and Qwen3.8-27B verifies them via rejection sampling, preserving the target model's output distribution.
Reported SPEED-Bench Acceptance Rates
In evaluations reported by NVIDIA using a draft block size of 16 on SPEED-Bench, the 4.21B draft averaged 5.58 accepted tokens per target step on the 1,138-turn qualitative subset, compared with 3.81 for Qwen3.8-27B's built-in multi-token prediction (MTP) head. On the 1,536-prompt throughput-32k long-context subset (33k to 46k tokens), NVIDIA reported an average acceptance rate of 5.11 tokens versus 3.86 for the MTP head. Across the same training corpus, NVIDIA also reported lower acceptance rates for two smaller external drafts it evaluated: Qwen3.5-0.8B (3.92 qualitative, 3.60 long-context) and a vocabulary-adapted Llama-3.2-1B checkpoint (3.42 qualitative, 3.07 long-context).
Deployment, Licensing, and Practical Tradeoffs
NVIDIA states that the checkpoints are intended for deployment on NVIDIA Groq 3 LPU (Language Processing Unit) hardware, where the company conducted its tests, and also load as standard safetensors models in vLLM, SGLang, and TRT-LLM on Linux systems with Blackwell compatibility noted.
For inference teams, the main tradeoff is memory footprint versus acceptance rate. Although Qwen3.8-27B's built-in MTP head adds only 0.42 billion non-embedding parameters and reuses the target's 2.54-billion-parameter embedding and output layers, the external 4.21B draft requires storing and running a separate multi-billion-parameter model. In addition, NVIDIA's model card provides acceptance-rate metrics rather than independent end-to-end wall-clock latency or cost-per-token measurements across GPU and LPU clusters. Licensing also differs from the underlying Qwen releases: while Qwen3.5-4B and Qwen3.8-27B carry the Apache 2.0 License, NVIDIA distributes Qwen3.5-4B-Qwen3.8-27B-Draft under the NVIDIA Open Model Agreement for commercial or non-commercial use.
Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-17.



