AberHu/Knowledge-Distillation-Zoo resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1759 stars · 267 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch reference collection implementing many classic knowledge distillation methods (logits, soft target, attention transfer, FitNet, PKT, RKD, etc.). It serves as a study and research baseline rather than a production library.

Use cases

• implement knowledge distillation in pytorch
• compress a large model into a smaller student network
• compare different KD loss functions
• learn how teacher-student distillation works
• reproduce classic KD papers like attention transfer or FitNet
• find a starting point for model compression research

When to choose

• you want simple, readable reference implementations of classic KD methods
• you are doing research or learning about knowledge distillation
• you need baseline KD losses to build on in PyTorch

When to avoid

• you need a production-ready, maintained distillation framework
• you need advanced techniques like data-free, online, or self-distillation
• you require a licensed dependency in your project (no license is provided)

Facets

learning-resource · maturity maintenance

machine-learning deep-learning llm-training deep-learning machine-learning developer-tools python knowledge-distillation model-compression pytorch teacher-student reference-implementation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="AberHu/Knowledge-Distillation-Zoo")