AnswerDotAI/fsdp_qlora

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1550 stars · 201 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A training script/library from Answer.AI that combines QLoRA (quantized LoRA) with PyTorch FSDP to fine-tune large language models like Llama-2 70B on limited GPU hardware. It supports bitsandbytes and HQQ quantization, gradient checkpointing, and CPU offloading.

Use cases

• fine-tune llama 2 70b on two 24gb gpus
• train llms with qlora and fsdp
• quantized lora finetuning of large language models
• low-memory finetuning of llms with cpu offload
• finetune llama on alpaca dataset with limited vram

When to choose

• you need to fine-tune very large LLMs on consumer or limited GPU hardware
• you want to combine QLoRA quantization with FSDP distributed training
• you're comfortable with alpha-stage tooling and debugging training runs

When to avoid

• you need a production-stable, fully supported training framework
• you prefer higher-level fine-tuning tools like Axolotl or Hugging Face TRL
• you don't have CUDA GPUs or sufficient CPU RAM for offloading

Facets

library · maturity experimental

llm-training machine-learning gpu-computing large-language-models machine-learning deep-learning python qlora fsdp fine-tuning llama quantization bitsandbytes hqq distributed-training gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="AnswerDotAI/fsdp_qlora")