# intel/intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

Repository: https://github.com/intel/intel-extension-for-transformers
Canonical: https://ross.abutalabs.com/products/intel-extension-for-transformers
Language: Python
License: Apache-2.0
License Family: permissive
Topics: large-language-model, chatbot, 4-bits, llm-inference, llm-cpu, chatpdf, streamingllm, intel-optimized-llamacpp, speculative-decoding, neural-chat, habana, neural-chat-7b, rag, retrieval, autoround, gaudi3
Archived: true
Last push: 2024-10-08T21:09:46+00:00

## Health v2 (maintenance only)
Score: 10/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 99
- inputs: {"age_days": 1391, "days_push": 694, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: archived
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2174, forks 216 (observed 2026-08-28T04:06:22.001868+00:00)

## What it is
Intel's toolkit for accelerating transformer-based GenAI/LLM workloads on Intel platforms, offering state-of-the-art compression (e.g., INT4 weight-only quantization) and the NeuralChat framework for building chatbots quickly. It supports inference on Intel CPUs, GPUs, and Habana Gaudi accelerators, plus fine-tuning techniques like QLoRA on CPUs.

## Use cases
- run quantized LLM inference on Intel Xeon CPUs
- build a chatbot with RAG on my own documents
- compress a large language model to 4-bit precision
- fine-tune an LLM with QLoRA on a laptop CPU
- accelerate Llama 3 inference on Intel GPUs
- deploy LLMs on Habana Gaudi accelerators

## When to choose
- you are deploying or optimizing LLMs on Intel hardware (Xeon, Arc, Gaudi)
- you need aggressive model compression like INT4 quantization with minimal accuracy loss
- you want a quick path to a chatbot or RAG pipeline on Intel platforms

## When to avoid
- your inference runs exclusively on NVIDIA GPUs or non-Intel hardware
- you need a general-purpose training framework rather than Intel-optimized inference and compression
- you require the latest model support on day one, as releases may lag upstream transformers

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, llm-training, rag, chatbot, machine-learning, deep-learning
- domain: large-language-models, artificial-intelligence, machine-learning, chatbots
- platform: python, cloud
- tags: quantization, int4, intel-cpu, intel-gpu, habana-gaudi, neural-chat, speculative-decoding, transformers, compression, chatpdf, retrieval-augmented-generation, linux, gpu

## Member repositories
- intel/intel-extension-for-transformers (main) score 10

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:22.001868+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:48:59.095760+00:00, confidence not recorded.
  - readme: https://github.com/intel/intel-extension-for-transformers (fetched 2026-08-28T04:06:22.001868+00:00, sha e2d9ff4b2e42)
  - registry_pypi: https://pypi.org/pypi/intel-extension-for-transformers/json (fetched 2026-08-29T10:29:01.481917+00:00, sha 3225e50b9338)
- Data as of 2026-08-30T08:39:29.467469+00:00.