CStanKonrad/long_llama

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1465 stars · 84 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LongLLaMA is a large language model handling long contexts of 256k tokens or more, built on OpenLLaMA and fine-tuned with the Focused Transformer (FoT) method. The repository provides model weights, inference code, and training resources including base, instruct, and code variants.

Use cases

• process documents with very long contexts
• run an LLM with 256k token context window
• fine-tune a long-context language model
• build retrieval over long documents with an LLM
• use a long-context code completion model
• research context extension methods for transformers

When to choose

• you need an open-weights model that handles extremely long inputs
• you want to experiment with the Focused Transformer context-scaling method
• you need a permissively licensed (Apache-2.0) long-context LLM

When to avoid

• you need a state-of-the-art model with active updates
• you lack GPU resources for large model inference
• you need production-grade support or commercial guarantees

Facets

library · maturity maintenance

llm-inference llm-training machine-learning large-language-models deep-learning python long-context focused-transformer openllama huggingface research-model 256k-context natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="CStanKonrad/long_llama")