facebookresearch/LAMA

Health v2 · maintenance only

Flags: no_releases archived no_license

Adoption not part of the score

1390 stars · 186 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LAMA is a research probe for analyzing the factual and commonsense knowledge stored in pretrained language models such as BERT, RoBERTa, GPT, ELMo, and Transformer-XL. It provides a unified interface to query these models with cloze-style probes and includes datasets like LAMA-UHN and Negated-LAMA.

Use cases

• measure how much factual knowledge a pretrained language model contains
• probe BERT or RoBERTa for commonsense and world knowledge
• evaluate how context affects a language model's factual predictions
• test whether language models handle negated statements
• create the harder LAMA-UHN subset for knowledge probing
• extract sentence embeddings from pretrained models for downstream tasks

When to choose

• you are doing NLP research on what knowledge language models encode
• you need a standardized benchmark to compare knowledge across multiple pretrained models
• you want to reproduce the 'Language Models as Knowledge Bases?' experiments

When to avoid

• you need a production tool for serving or fine-tuning language models
• you want to probe modern LLMs like GPT-3 or Llama, which are not supported
• you need a maintained library with active support - the codebase is a research artifact with an old Python 3.7 setup

Facets

library · maturity maintenance

nlp machine-learning benchmarking testing machine-learning python language-models knowledge-probing research pretrained-models factual-knowledge commonsense natural-language-processing linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/LAMA")