haitongli/knowledge-distillation-pytorch

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2000 stars · 353 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch framework for running knowledge distillation experiments, supporting both shallow (teacher-to-small-CNN) and deep distillation on CIFAR-10. It includes pretrained teacher checkpoints, hyperparameter search, and result synthesis utilities.

Use cases

• distill a large model's knowledge into a smaller network
• run knowledge distillation experiments in pytorch
• compress a resnet into a lightweight cnn for cifar-10
• search hyperparameters for distillation training
• learn how knowledge distillation works with example code
• train student models using dark knowledge from pretrained teachers

When to choose

• you want a ready-to-run PyTorch codebase for KD experiments on CIFAR-10
• you need pretrained teacher models and hyperparameter search tooling for distillation
• you are exploring model compression for efficient hardware deployment

When to avoid

• you need distillation for datasets other than CIFAR-10 or for transformer/LLM models
• you want a production-grade, actively maintained training framework
• you need a plug-and-play library API rather than a research codebase driven by CLI scripts

Facets

library · maturity maintenance

machine-learning deep-learning llm-training deep-learning computer-vision machine-learning python knowledge-distillation model-compression pytorch cifar10 dark-knowledge teacher-student research-code linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="haitongli/knowledge-distillation-pytorch")