HobbitLong/RepDistiller

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2437 stars · 398 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

RepDistiller is a PyTorch research codebase implementing Contrastive Representation Distillation (CRD) from ICLR 2020. It also benchmarks 12 state-of-the-art knowledge distillation methods for training smaller student networks from larger teacher models.

Use cases

• compress a large neural network into a smaller student model
• compare knowledge distillation methods on CIFAR/ImageNet
• reproduce the CRD paper results
• train a student network with contrastive representation distillation
• experiment with combining KD loss with other distillation objectives
• download pretrained teacher models for distillation experiments

When to choose

• you need a unified PyTorch framework to benchmark multiple knowledge distillation techniques
• you want to implement or extend CRD for model compression research
• you need pretrained teacher checkpoints and reproducible distillation pipelines on CIFAR datasets

When to avoid

• you need production-ready model compression tooling rather than research code
• you require support for the latest PyTorch versions with active maintenance
• you want distillation for NLP or transformer models, since the codebase focuses on CNN image classifiers

Facets

library · maturity maintenance

machine-learning deep-learning benchmarking machine-learning deep-learning python knowledge-distillation pytorch model-compression contrastive-learning research-code iclr-2020 algorithms linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="HobbitLong/RepDistiller")