cybertronai/gradient-checkpointing

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2843 stars · 279 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Python library that reduces GPU memory usage when training very deep neural networks via gradient checkpointing, trading computation for memory. It rewrites the TensorFlow computation graph so backpropagation memory scales as O(sqrt(n)) instead of linearly with layer count.

Use cases

• fit a huge neural network into GPU memory
• train deep networks with large batch sizes
• reduce memory cost of backpropagation
• train 10x larger models on the same GPU
• apply sublinear-memory gradient computation in TensorFlow

When to choose

• you train very deep feed-forward models in TensorFlow 1.x and hit GPU memory limits
• you can afford ~20% extra compute time to fit a much larger model or batch size

When to avoid

• you use PyTorch or modern frameworks with built-in checkpointing
• you need actively maintained TensorFlow 2.x support
• compute time is more constrained than memory

Facets

library · maturity maintenance

machine-learning deep-learning llm-training gpu-computing deep-learning machine-learning gpu-computing python gradient-checkpointing tensorflow memory-optimization backpropagation training gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="cybertronai/gradient-checkpointing")