
The maintainers of llama.cpp have issued pre-release b12473 on November 17, 2026, introducing a targeted cleanup to the project's CUDA quantized matrix multiplication (MMQ) dispatch path via pull request #32392.
What Changed in the CUDA Launcher
In commit a49db73, the update removes a redundant invocation of the CUDA_SET_SHARED_MEMORY_LIMIT macro inside launch_mul_mat_q. Earlier architectural changes—specifically PR #24127, which introduced a fallback template parameter, and PR #29948, which added a has_fusion template parameter for fusing feed-forward network gate/up matrix multiplications and GLU operations in MMQ—had already shifted kernel selection earlier in the templated call chain. Despite that refactoring, both macro calls had remained in place.
With launch_mul_mat_q now dispatching only a single kernel type per call, each template specialization sets the CUDA shared memory limit solely for the kernel actually being launched. According to the release notes, the change is intended to simplify the codebase and improve readability rather than alter external user flags or model formats.
Scope, Builds, and Practical Limitations
Because this update is labeled a pre-release maintenance build focused on code structure, the release notes do not include performance benchmarks or claim inference speedups. Developers maintaining custom CUDA forks or downstream GGML integrations that touch MMQ kernel templates should review the updated specialization behavior, while typical end users running prebuilt binaries will mainly see routine continuous-integration builds.
Alongside the CUDA code change, the automated GitHub Actions release distributes prebuilt artifacts for Windows, Linux, macOS, Android, and iOS spanning CPU, CUDA 12 and CUDA 13, Vulkan, ROCm 10.0, SYCL, OpenVINO, and Snapdragon NPU/Adreno targets, while macOS KleidiAI and openEuler builds are listed as disabled in this build matrix.
EXPLORE THE SOURCE
Projects in this story
GitHub repositories
- ggml-org/llama.cpp ↗View project details
Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-17.



