DeepInfer open-source release

DeepSeek: DeepInfer open-source release, Tuesday, October 20, 2026, Online, on GitHub (deepseek-ai/DeepInfer). Ended.

Open-source high-performance inference framework released under the Apache License 2.0. Provides offline inference and supports DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with optimized kernels, parallelism, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured outputs. Originally built primarily on vLLM. A KV cache system, prefill/decode disaggregation, and load balancing across data-parallel replicas are not included in this release.

Who’s involved

• DeepSeek (host, organizer)

Tickets and details: github.com

Last updated 2026-10-20.