epfml/attention-cnn resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1121 stars · 129 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Official PyTorch source code for the ICLR 2020 paper 'On the Relationship between Self-Attention and Convolutional Layers', which proves and empirically shows that multi-head self-attention layers can learn to perform convolution. It includes experiment scripts to reproduce all results from the paper.

Use cases

• reproduce experiments from the self-attention vs convolution paper
• study how attention layers learn convolutional behavior
• compare multi-head self-attention with CNN layers on image tasks
• get a reference PyTorch implementation of attention-based vision models
• explore research on attention mechanisms in computer vision

When to choose

• you want to reproduce or extend the paper's experiments
• you are researching the relationship between self-attention and convolution
• you need a reference implementation of multi-head self-attention for vision

When to avoid

• you need a production-ready vision transformer library
• you want maintained, general-purpose deep learning tooling rather than research code
• you need a framework with broad model support and community maintenance

Facets

learning-resource · maturity maintenance

deep-learning machine-learning image-processing benchmarking deep-learning computer-vision python self-attention convolution vision-transformer iclr-2020 research-code pytorch reproducibility research linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="epfml/attention-cnn")