kyegomez/BitNet

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1945 stars · 172 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch implementation of the BitNet architecture from the paper 'BitNet: Scaling 1-bit Transformers for Large Language Models', providing BitLinear layers that replace standard nn.Linear with 1-bit quantized projections. It also includes a full BitNet Transformer and work toward the newer 1.58-bit LLM paper.

Use cases

• implement 1-bit quantized transformer layers in pytorch
• train large language models with bitlinear layers
• reduce LLM memory footprint with 1-bit weights
• experiment with the BitNet paper architecture
• swap nn.Linear for quantized BitLinear modules
• build a transformer with bit attention and grouped query attention

When to choose

• you want to reproduce or experiment with BitNet / 1-bit LLM research in PyTorch
• you are training a model from scratch and want 1-bit quantized linear layers
• you need a ready-made BitLinear layer or BitNet transformer implementation

When to avoid

• you want to quantize an already-trained model - BitLinear requires training or finetuning from scratch
• you need a production-ready, bug-free 1.58-bit implementation - parts are still in progress with known dequantization bugs
• you need inference-optimized 1-bit LLM runtimes rather than a research codebase

Facets

library · maturity active

machine-learning deep-learning llm-training transformers deep-learning large-language-models machine-learning python quantization 1-bit-llm pytorch bitlinear transformer-architecture research-implementation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="kyegomez/BitNet")