# ridgerchu/matmulfreellm

Implementation for MatMul-free LM.

Repository: https://github.com/ridgerchu/matmulfreellm
Canonical: https://ross.abutalabs.com/products/matmulfreellm
Language: Python
License: Apache-2.0
License Family: permissive
Topics: llm, large-language-model, linear-transformer
Last push: 2025-12-02T00:25:05+00:00

## Health v2 (maintenance only)
Score: 49/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 55, release rhythm 35, longevity 61
- inputs: {"age_days": 862, "days_push": 275, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3089, forks 201 (observed 2026-08-28T04:07:42.259737+00:00)

## What it is
A Python implementation of MatMul-Free LM, a language model architecture that eliminates matrix multiplication operations using ternary weights and linear attention. It is compatible with the Hugging Face Transformers library and provides pretrained models from 370M to 2.7B parameters.

## Use cases
- train a language model without matrix multiplication
- run efficient LLM inference with ternary weights
- experiment with linear transformer architectures
- load a matmul-free model with huggingface AutoModel
- research hardware-efficient language model architectures
- compare scaling laws of matmul-free vs transformer models

## When to choose
- you want to experiment with matmul-free or bit-linear LLM architectures
- you need a Hugging Face-compatible linear transformer implementation
- you are researching efficient language model scaling

## When to avoid
- you need a production-ready general-purpose LLM framework
- you require standard transformer architectures with mature tooling
- you work outside the PyTorch/Triton ecosystem

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, llm-inference, llm-training, deep-learning
- domain: large-language-models, deep-learning, machine-learning, artificial-intelligence
- platform: python
- tags: matmul-free, linear-transformer, ternary-weights, huggingface-transformers, efficient-inference, research, gpu, linux

## Member repositories
- ridgerchu/matmulfreellm (main) score 49

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:42.259737+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:46:29.182095+00:00, confidence not recorded.
  - readme: https://github.com/ridgerchu/matmulfreellm (fetched 2026-08-28T04:07:42.259737+00:00, sha 79b78803d5e5)
- Data as of 2026-08-30T08:39:29.467469+00:00.