vacancy/Synchronized-BatchNorm-PyTorch

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1502 stars · 188 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch library implementing Synchronized Batch Normalization, which computes batch statistics across all GPUs during multi-device training instead of per-device. It provides drop-in replacement modules and a DataParallel wrapper with replication callbacks.

Use cases

• train object detection models with small per-GPU batch sizes
• normalize batch statistics across multiple GPUs in PyTorch
• improve segmentation model training with tiny batches per device
• convert existing PyTorch models to use synchronized batchnorm
• reproduce MegDet-style large mini-batch training

When to choose

• training on multiple GPUs with small per-device batch sizes where batch statistics matter
• using nn.DataParallel and needing cross-device normalization
• research prototypes needing sync batchnorm before native PyTorch support

When to avoid

• single-GPU or CPU training (built-in BatchNorm is identical)
• new projects that can use PyTorch's native SyncBatchNorm
• production systems needing actively maintained dependencies
• DistributedDataParallel workflows with native support

Facets

library · maturity maintenance

machine-learning deep-learning deep-learning computer-vision machine-learning python cross-platform pytorch batch-normalization multi-gpu data-parallel sync-bn gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vacancy/Synchronized-BatchNorm-PyTorch")