S-LoRA/S-LoRA

Health v2 · maintenance only

Flags: no_releases archived

Adoption not part of the score

1923 stars · 126 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

S-LoRA is a serving system for large language models that scales to thousands of concurrent LoRA adapters on one or more GPUs. It uses Unified Paging to manage adapter weights and KV cache in a unified memory pool, plus custom CUDA kernels and tensor parallelism for heterogeneous batching.

Use cases

• serve thousands of LoRA adapters from one base model
• host many task-specific fine-tuned LLMs on a single GPU
• improve LLM serving throughput with batched LoRA inference
• run multi-adapter inference across multiple GPUs
• deploy a fine-tuning-as-a-service backend
• benchmark LoRA serving performance against vLLM and PEFT

When to choose

• you need to serve many LoRA adapters derived from a single base model
• adapter count or memory fragmentation limits your current vLLM/PEFT setup
• you want research-grade multi-adapter serving with tensor parallelism

When to avoid

• you serve a single model without adapters - use vLLM or similar instead
• you need recent model support or active maintenance - the project is in maintenance mode
• you lack an Ampere-class CUDA GPU with bfloat16 support
• you need a production-supported inference stack with broad ecosystem compatibility

Facets

service · maturity maintenance

llm-inference machine-learning gpu-computing large-language-models machine-learning deep-learning python lora model-serving inference-server cuda-kernels tensor-parallelism batched-inference linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="S-LoRA/S-LoRA")