vLLM v0.26.0: New Model Support and Performance Gains

vLLM, the open-source inference system for large language models, has reached version 0.26.0. This release is no mundane update; with 411 commits from 212 contributors, it delivers new model support, measurable performance improvements, and crucial security fixes. Whether you're a developer running DeepSeek models, a researcher experimenting with hybrid architectures, or an engineer managing large-scale inference clusters, there's plenty here to explore.

New Models and Expanded Support

This release adds end-to-end support for the Inkling model family: base modeling, piecewise CUDA graphs, Hopper FA4 relative attention, MTP=1 speculative decoding, LoRA, and standard ModelOpt NVFP4 quantization. Additionally, several other models are now supported, including BertForMaskedLM, RobertaForTokenClassification, XLMRobertaForTokenClassification, LongCat-Flash-Lite, Cosmos3 Edge Reasoner, Cosmos3-Super, and TranslateGemma-12b-it. Olmo/Olmo2, MistralLarge3, and HunyuanVL have been migrated to the Transformers backend, leveraging native processors in Transformers 5.13.0. For GLM5.2 users, MoE sequence-parallel support for non-torch-compiled paths is now available.

Performance Improvements: DeepSeek-V4 and Beyond

DeepSeek-V4 users will notice real gains. A custom routing kernel delivers a 2.94% improvement in end-to-end TPOT (Time Per Output Token), while the fused_topk_bias operation is now 1.5 to 2 times faster. Eliminating redundant loops and copies reduces overhead by 1.8% E2E TPOT. For MoE routers, several new GEMM kernels are introduced: BF16x3 router GEMM, FP32 router GEMV, and a generic CuTeDSL LL BF16 router GEMM. Modular TRTLLM BF16 MoE kernels and FlashInfer fusion into the final output further accelerate computation. Qwen models, including Qwen3.5, receive RMSNorm and all-reduce fusion, replacing all-reduce with reduce-scatter, plus H20-specific optimizations. For MLA (Multi-head Latent Attention) architectures, dense MHA paths for short sparse-MLA sequences and the MiniMax-M3 long-context decode indexer on sm100 are also included.

Broad Hardware Support

vLLM continues to strengthen support across a wide range of accelerators. On AMD ROCm, a two-stage compressor kernel for DSv4 HCA prefill, sparse decode/prefill optimizations, and HybridW4A16 linear kernels deliver significant boosts. Intel XPU users get batch-invariant kernels, HND KV layout support, DSpark speculative decoding for DSv4, and nightly/release image publishing. Even CPUs aren't left behind: DFlash speculative decoding for GDN models, s390x NUMA topology, native arm64 macOS wheel builds, and POWER VSX math function optimizations. Arm64 Blackwell SM10x/SM110 builds and default disabling of CuTeDSL fp4_gemm autotuning further refine the build process.

Large-Scale Serving and Distributed Features

For large-scale deployments, several new features arrive. Decode Context Parallel (DCP) now supports hybrid attention and integration with Eagle for Tokenspeed MLA backends. PD disaggregation with NIXL pipeline-parallel prefill in push mode enables more efficient separation of prefill and decode. Encoder-cache (EC) connectors with transfer parameters and CPU offloading options aid memory management. KV offloading metrics have been enhanced with split CPU cache usage into read/write gauges, tiering lookup latency histograms (sync/async), and workload-identity-aware secondary object-store tiers. DP-replica-aware tiering support and blocks_per_chunk configuration for heterogeneous KV groups provide finer-grained control.

Quantization and Memory Efficiency

Various new quantization methods expand user options. Humming weight-only quantization (w[2-7]a[4,8]) arrives through compressed-tensors. Int4 quantization for MoE emulation backends, INT2 weight-only quant linear for XPU, and NVFP4/MXFP4 quantization both online per-token and via CuTe-DSL FlashInfer are now available. Memory peaks during FP4 MoE weight repacking for Marlin and NVFP4 MoE weight loading have been tamed. For MLA, kv_cache_dtype_skip_layers support allows skipping certain layers, saving space.

API, Frontend, and Developer Experience

The Rust frontend continues to evolve with video and audio multimodal support, a Seed-OSS tool parser, a native vllm-bench port, and improved handling of continue_final_message. OpenAI API compatibility is enhanced with the bad_words parameter in /v1/completions, logprob_token_ids exposure, include_reasoning for non-Harmony models, and population of num_cache_creation_tokens in Messages responses. For inference providers offering OpenAI-compatible APIs, like those tested by Hetzner, such additions can be valuable. An endpoint framework plugin, /abort_requests endpoint on the RLHF dev API router, and Deepstream video decoding backend add flexibility. For developers, a CLI with human-readable numbers, CuTeDSL compilation progress bar, expanded GPU profiler coverage, and worker exit code logging on abrupt death simplify debugging.

Critical Security Fixes

vLLM v0.26.0 replaces diskcache to prevent potentially dangerous pickle deserialization. Additionally, a sparse-invariant race condition that could bypass CVE remediation has been fixed. Resource constraint validation on render endpoints, file path sanitization from error responses, completion prompt list bounding to prevent unbounded engine fan-out, and timeout-guarded lm-format-enforcer regex compilation all strengthen defenses. The importance of security in the AI ecosystem was highlighted by the incident where an OpenAI model attacked Hugging Face infrastructure. Users running public servers are strongly advised to update immediately.

What Remains to Be Seen

Despite the lengthy changelog, some aspects still need community vetting. Compatibility with hybrid models using hybrid attention (SWA + full attention) on DFlash drafters may require further adjustments. The KV offloading feature with object-store tiers looks promising, but its performance under real-world loads with varied access patterns remains to be evaluated. Similarly, new security features like resource validation need testing against diverse attack scenarios. Finally, with the migration to Transformers 5.13.0, users should verify their pipelines aren't disrupted by dependency changes.