FLHonker/Awesome-Knowledge-Distillation resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

2685 stars · 333 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A curated awesome-list of knowledge distillation papers (2014-2021), organized by categories such as knowledge forms, KD+GAN, data-free KD, and applications. It also links to distiller tools and related model compression techniques like pruning and quantization.

Use cases

• find papers on knowledge distillation
• learn about model compression techniques
• survey research on distilling knowledge from logits and intermediate layers
• find data-free knowledge distillation papers
• discover tools for distilling neural networks
• research teacher-student network training

When to choose

• you need a categorized bibliography of KD research
• you are starting research in model compression or distillation
• you want to survey KD applications in NLP or recommender systems

When to avoid

• you need runnable distillation code rather than paper references
• you need papers published after 2021
• you need a maintained software library

Facets

learning-resource · maturity maintenance

machine-learning deep-learning developer-tools deep-learning machine-learning awesome-lists tutorials cross-platform awesome-list knowledge-distillation model-compression papers transfer-learning curated-list

Member repositories

For agents

markdown · JSON · MCP: product_card(name="FLHonker/Awesome-Knowledge-Distillation")