szagoruyko/attention-transfer

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1463 stars · 273 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

PyTorch reference implementation of the ICLR 2017 paper 'Paying More Attention to Attention', which improves convolutional neural networks by transferring spatial attention maps from a teacher network to a student network. It includes CIFAR-10 and ImageNet experiment code, pretrained ResNet-18 models, and a Jupyter notebook for visualizing attention maps.

Use cases

• implement attention transfer for knowledge distillation in pytorch
• reproduce cifar-10 and imagenet results from the attention transfer paper
• train a smaller student resnet using a larger teacher network
• visualize attention maps of a resnet-34 in a jupyter notebook
• download a pretrained resnet-18 model trained with attention transfer
• compare attention transfer against classic knowledge distillation

When to choose

• you want the official reference code for the attention transfer distillation method
• you are doing research on knowledge distillation or attention-based transfer in CNNs
• you need reproducible CIFAR-10 baselines for teacher-student training in PyTorch

When to avoid

• you need a maintained, production-ready training framework
• you require a permissively licensed dependency - the repo has no explicit license
• you want transformer or NLP attention mechanisms rather than CNN spatial attention
• you need grad-based attention transfer, which was never added to the repo

Facets

library · maturity maintenance

machine-learning deep-learning image-processing computer-vision deep-learning computer-vision machine-learning python pytorch knowledge-distillation attention-maps student-teacher-networks cifar-10 imagenet resnet research-code iclr-2017 jupyter-notebook research gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="szagoruyko/attention-transfer")