neuralmagic/deepsparse

Health v2 · maintenance only

Flags: archived no_license

Adoption not part of the score

3158 stars · 193 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

DeepSparse is a sparsity-aware deep learning inference runtime that delivers GPU-class performance on x86 CPUs for ONNX models, covering computer vision, NLP, and LLM workloads. Development ceased and the project was deprecated on June 2, 2025 following Neural Magic's acquisition by Red Hat.

Use cases

• run ONNX models fast on CPUs without a GPU
• deploy sparse pruned and quantized models for inference
• serve object detection models on x86 servers
• run LLM inference on CPU hardware
• integrate ML inference into Python applications

When to choose

• you need fast CPU-only inference for ONNX models and are comfortable with an unmaintained runtime
• you already have sparsified models from Neural Magic's tooling and accept no future updates

When to avoid

• you need ongoing support, updates, or security patches
• you are starting a new project - use vLLM or other actively maintained runtimes instead
• you need GPU or non-x86 (e.g. ARM) inference

Facets

library · maturity abandoned

llm-inference machine-learning computer-vision nlp machine-learning deep-learning large-language-models computer-vision python cpp onnx inference-runtime sparsity cpu-optimization deprecated pruning quantization natural-language-processing linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="neuralmagic/deepsparse")