intel/intel-extension-for-transformers

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

2174 stars · 216 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Intel's toolkit for accelerating transformer-based GenAI/LLM workloads on Intel platforms, offering state-of-the-art compression (e.g., INT4 weight-only quantization) and the NeuralChat framework for building chatbots quickly. It supports inference on Intel CPUs, GPUs, and Habana Gaudi accelerators, plus fine-tuning techniques like QLoRA on CPUs.

Use cases

• run quantized LLM inference on Intel Xeon CPUs
• build a chatbot with RAG on my own documents
• compress a large language model to 4-bit precision
• fine-tune an LLM with QLoRA on a laptop CPU
• accelerate Llama 3 inference on Intel GPUs
• deploy LLMs on Habana Gaudi accelerators

When to choose

• you are deploying or optimizing LLMs on Intel hardware (Xeon, Arc, Gaudi)
• you need aggressive model compression like INT4 quantization with minimal accuracy loss
• you want a quick path to a chatbot or RAG pipeline on Intel platforms

When to avoid

• your inference runs exclusively on NVIDIA GPUs or non-Intel hardware
• you need a general-purpose training framework rather than Intel-optimized inference and compression
• you require the latest model support on day one, as releases may lag upstream transformers

Facets

library · maturity active

llm-inference llm-training rag chatbot machine-learning deep-learning large-language-models artificial-intelligence machine-learning chatbots python cloud quantization int4 intel-cpu intel-gpu habana-gaudi neural-chat speculative-decoding transformers compression chatpdf retrieval-augmented-generation linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="intel/intel-extension-for-transformers")