# Spark NLP

State of the Art Natural Language Processing

Repository: https://github.com/JohnSnowLabs/spark-nlp
Canonical: https://ross.abutalabs.com/products/spark-nlp
Homepage: https://sparknlp.org/
Language: Scala
License: Apache-2.0
License Family: permissive
Topics: nlp, natural-language-processing, spark, pyspark, named-entity-recognition, sentiment-analysis, lemmatizer, spell-checker, entity-extraction, part-of-speech-tagger, bert, transformers, tensorflow, language-detection, machine-translation, text-classification, llm, question-answering, llamacpp, onnx
Last push: 2026-08-26T16:16:26+00:00
Link (homepage): https://sparknlp.org/
Link (site_page): https://sparknlp.org/docs/en/quickstart
Link (site_page): https://sparknlp.org/docs/en/install

## Health v2 (maintenance only)
Score: 96/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 90, longevity 100
- inputs: {"age_days": 3265, "days_push": 7, "days_rel": 70, "gap_med": 18, "n_releases_24m": 26}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 4159, forks 743 (observed 2026-08-28T04:08:37.542604+00:00)

## What it is
Spark NLP is an open-source natural language processing library built natively on Apache Spark, providing scalable NLP annotations and transformer models (BERT, T5, Whisper, Llama, etc.) for Python, Scala, Java, and R. It ships with 100,000+ pretrained pipelines and models in 200+ languages for tasks like NER, sentiment analysis, translation, summarization, and question answering.

## Use cases
- run named entity recognition on large text corpora at scale
- perform sentiment analysis in a distributed Spark pipeline
- translate text across 180+ languages
- summarize documents with transformer models
- deploy BERT or Llama models in production on the JVM
- transcribe audio with Whisper in a Spark job
- classify documents and detect entities in enterprise data pipelines

## When to choose
- you need NLP at scale on big data using Apache Spark clusters
- you want production-ready pretrained models in many languages
- your team works in Python, Scala, or Java on the JVM
- you need GPU-accelerated transformer inference in distributed pipelines

## When to avoid
- you need lightweight single-machine NLP without a Spark dependency
- you want to fine-tune cutting-edge LLMs with the latest research tooling
- your project is small enough that Spark's overhead outweighs its benefits

## Facets
- artifact type: library
- maturity: stable
- function: nlp, machine-learning, llm-inference, speech-recognition, image-processing, rag
- domain: machine-learning, large-language-models, data-science, big-data
- platform: python, jvm, cloud
- tags: apache-spark, transformers, pretrained-models, distributed-computing, named-entity-recognition, sentiment-analysis, onnx, tensorflow, natural-language-processing, gpu, docker

## Member repositories
- JohnSnowLabs/spark-nlp (main) score 96
- JohnSnowLabs/spark-nlp-workshop (examples) score 77

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:37.542604+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:22:49.638466+00:00, confidence not recorded.
  - readme: https://github.com/JohnSnowLabs/spark-nlp (fetched 2026-08-28T04:08:37.542604+00:00, sha c17496da62b4)
  - homepage: https://sparknlp.org/ (fetched 2026-08-29T09:13:37.843398+00:00, sha 0102d12db5c5)
  - site_page: https://sparknlp.org/docs/en/quickstart (fetched 2026-08-29T09:13:37.846065+00:00, sha 9dd7a46d48ec)
  - site_page: https://sparknlp.org/docs/en/install (fetched 2026-08-29T09:13:37.848248+00:00, sha 1ebdf5fff307)
- Data as of 2026-08-30T08:39:29.467469+00:00.