Tebmer/Awesome-Knowledge-Distillation-of-LLMs resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1305 stars · 73 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A curated awesome-list of research papers accompanying the survey 'A Survey on Knowledge Distillation of Large Language Models'. It organizes KD literature into knowledge elicitation, distillation algorithms, and skill/vertical distillation of LLMs, updated weekly.

Use cases

• find papers on distilling GPT-4 capabilities into smaller open-source models
• learn how to compress a large language model via knowledge distillation
• research self-improvement and self-training techniques for LLMs
• find data augmentation methods using LLM-generated data for fine-tuning
• survey multi-modal and alignment distillation approaches
• prepare a literature review on LLM knowledge distillation

When to choose

• you need a curated, regularly updated reading list on LLM knowledge distillation
• you want the taxonomy from the accompanying survey paper to guide research
• you are exploring teacher-student training, self-distillation, or LLM-based data synthesis

When to avoid

• you need runnable code or a software library rather than paper references
• you want general machine learning distillation outside the LLM context
• you need a maintained tool with a license and releases

Facets

learning-resource · maturity active

machine-learning llm-training rag prompt-engineering data-science large-language-models machine-learning artificial-intelligence awesome-lists tutorials cross-platform awesome-list knowledge-distillation survey papers model-compression self-training data-augmentation alignment instruction-tuning

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Tebmer/Awesome-Knowledge-Distillation-of-LLMs")