
DeepSeek has open-sourced DeepInfer under the Apache 2.0 license, releasing the core inference engine used behind its production services. Rather than tying its runtime to a single accelerator vendor, DeepInfer ships with native support for both NVIDIA SM100 (Blackwell B200/GB200) GPUs and Huawei Ascend 950 NPUs.
Targeted kernels and batch scope
The initial release targets DeepSeek-V4.1-Flash Mixture-of-Experts serving, combining FlashMLA, DeepGEMM, DeepEP, asynchronous batch scheduling, grammar-constrained structured outputs, and DSpark speculative decoding.
What is not included out of the box
DeepInfer is positioned as an offline batch inference engine and reference implementation rather than a turnkey online microservice. It does not bundle a full HTTP serving layer, distributed KV-cache disaggregation, or replica load balancing, leaving production web-serving orchestration to downstream teams. AiBrainWorX has not independently benchmarked DeepInfer on Blackwell or Ascend hardware.
Original reporting; repository and corroborating sources reviewed November 16, 2026. No independent hardware benchmark claimed.



