Open-source nlp software
1574 open-source libraries, tools and applications tagged nlp, ranked by how closely nlp is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.
1574 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| salesforce/CodeT5 Official research release of CodeT5 and CodeT5+ open code large language models from Salesforce Research for code understanding and generat… | 10 | 3093 | maintenance |
| Whishper Whishper is a self-hosted, 100% local audio transcription and subtitling suite with a web UI, powered by FasterWhisper. It transcribes audi… | 58 | 3074 | maintenance |
| bigscience-workshop/promptsource PromptSource is a Python toolkit and repository for creating, sharing, and applying natural language prompts to Hugging Face datasets. It h… | 23 | 3037 | maintenance |
| yangjianxin1/GPT2-chitchat A GPT2-based Chinese chitchat dialogue model project built on HuggingFace transformers, including training, preprocessing, and interactive … | 32 | 2995 | maintenance |
| kotartemiy/newscatcher A Python package that programmatically collects normalized news articles from thousands of news websites, filterable by topic, country, and… | 32 | 2988 | maintenance |
| facebookresearch/XLM PyTorch implementation of Cross-lingual Language Model Pretraining (XLM) from Facebook AI Research, covering MLM, CLM, and TLM objectives p… | 10 | 2922 | maintenance |
| jbesomi/texthero Texthero is a Python toolkit for text preprocessing, representation, and visualization, designed to work on top of Pandas Series and DataFr… | 23 | 2905 | maintenance |
| EdgeTranslate/EdgeTranslate Edge Translate is a browser extension for translating selected text and entire web pages, supporting Chrome, Firefox, Edge, and QQ Browser.… | 23 | 2904 | maintenance |
| huggingface/neuralcoref NeuralCoref is a spaCy pipeline extension that annotates and resolves coreference clusters using a neural network, with a pre-trained Engli… | 10 | 2892 | maintenance |
| TeamHG-Memex/eli5 ELI5 is a Python library for debugging, inspecting, and explaining machine learning classifiers and regressors. It supports scikit-learn, X… | 63 | 2847 | maintenance |
| SCUTlihaoyu/open-chat-video-editor An open-source Python tool that automatically generates short videos from a short text prompt or a web URL, producing narration, background… | 30 | 2816 | maintenance |
| GerevAI/gerev Gerev is an AI-powered, self-hostable enterprise search engine that indexes workplace data sources like Slack, Confluence, Jira, and Google… | 31 | 2809 | maintenance |
| microsoft/CodeBERT A collection of pre-trained code models from Microsoft, including CodeBERT and successors like GraphCodeBERT and UniXcoder, usable via Hugg… | 32 | 2786 | maintenance |
| sxei/pinyinjs A small JavaScript library for converting between Chinese characters and pinyin, supporting output with tones, without tones, or as first l… | 32 | 2758 | maintenance |
| paperswithcode/galai galai is the official Python library for loading and running inference with the GALACTICA general-purpose scientific language models from P… | 23 | 2741 | maintenance |
| tdebatty/java-string-similarity A Java library implementing a dozen string similarity and distance algorithms, including Levenshtein, Damerau-Levenshtein, Jaro-Winkler, Lo… | 23 | 2741 | maintenance |
| smalot/pdfparser A standalone PHP library for parsing PDF files and extracting data such as text, metadata, and page content. It supports compressed PDFs an… | 85 | 2731 | maintenance |
| thisandagain/sentiment A Node.js module that performs sentiment analysis on text using the AFINN-165 wordlist and Emoji Sentiment Ranking. It supports custom lang… | 23 | 2678 | maintenance |
| baidu/Familia Familia is a C++ toolkit for industrial topic modeling, providing document topic inference and semantic matching tools along with pretraine… | 23 | 2644 | maintenance |
| blmoistawinde/HarvestText HarvestText is a Python library for Chinese text mining and preprocessing using unsupervised or weakly supervised methods. It integrates do… | 23 | 2633 | maintenance |
| pickhardt/betty Betty is a Ruby-based command line tool that translates plain English phrases into shell commands, acting like Siri for the terminal. It le… | 23 | 2601 | maintenance |
| baidu/AnyQ AnyQ is Baidu's open-source FAQ-based question answering framework with a configurable, plugin-based architecture covering question analysi… | 32 | 2573 | maintenance |
| OFA-Sys/OFA OFA is a unified sequence-to-sequence pretrained model supporting English and Chinese that unifies cross-modality, vision, and language tas… | 32 | 2555 | maintenance |
| shirosaidev/stocksight stocksight is an open-source Python application that analyzes stock market sentiment using Twitter streams and news headlines, storing data… | 23 | 2541 | maintenance |
| dmlc/gluon-nlp GluonNLP is a Python toolkit built on Apache MXNet that provides text data loading, processing tools, pretrained model zoos, and training u… | 10 | 2538 | maintenance |
| SUSI.AI SUSI.AI is an open-source personal assistant platform whose Java server holds the assistant's 'intelligence', answering chat and voice quer… | 10 | 2518 | maintenance |
| luin/readability A Node.js library that extracts clean, readable article content from any web page, based on arc90's readability project. It returns the art… | 32 | 2517 | maintenance |
| wiseman/py-webrtcvad A Python wrapper around Google's WebRTC Voice Activity Detector, classifying short frames of 16-bit mono PCM audio as speech or non-speech.… | 32 | 2499 | maintenance |
| filipedeschamps/video-maker An open-source Node.js project that automates video production by chaining robots that fetch content, analyze text with IBM Watson NLU, and… | 32 | 2493 | maintenance |
| saffsd/langid.py langid.py is a standalone language identification tool that classifies text into one of 97 pre-trained languages. It works as a Python libr… | 32 | 2461 | maintenance |
| mattpocock/ts-error-translator A VSCode extension by Matt Pocock that translates cryptic TypeScript compiler errors into plain English explanations. It helps developers u… | 23 | 2459 | maintenance |
| alephdata/aleph Aleph is a self-hosted platform for indexing, searching, and browsing large volumes of documents (PDF, Word, HTML) and structured data (CSV… | 63 | 2441 | maintenance |
| CyberZHG/keras-bert A Keras implementation of BERT that can load official Google pre-trained checkpoints for feature extraction and prediction. It includes a t… | 10 | 2419 | maintenance |
| Texar Texar is a modularized Python toolkit for machine learning, especially natural language processing and text generation, built on TensorFlow… | 61 | 2390 | maintenance |
| BrikerMan/Kashgari Kashgari is a Keras/TensorFlow 2-based NLP transfer learning framework for building text labeling (NER, PoS) and text classification models… | 23 | 2380 | maintenance |
| google-research/electra ELECTRA is a research library from Google for self-supervised pre-training of transformer text encoders using a discriminator-based objecti… | 10 | 2364 | maintenance |
| aboSamoor/polyglot Polyglot is a Python natural language processing toolkit designed for massive multilingual applications, offering language detection, token… | 32 | 2363 | maintenance |
| salesforce/decaNLP decaNLP is a PyTorch research framework and benchmark for the Natural Language Decathlon, a multitask NLP challenge spanning ten tasks from… | 10 | 2336 | maintenance |
| Determined22/zh-NER-TF A simple character-based BiLSTM-CRF sequence labeling model for Chinese Named Entity Recognition built with TensorFlow 1.x. It recognizes P… | 32 | 2331 | maintenance |
| lucasjinreal/weibo_terminater A Python-based web scraper that crawls Weibo (Sina's microblog platform) to collect user posts, comments, followers, and conversation pairs… | 32 | 2317 | maintenance |
| approximatelabs/sketch Sketch is an AI code-writing assistant for pandas users that understands the context of your dataframe content to generate relevant suggest… | 32 | 2281 | maintenance |
| jianfch/stable-ts A Python library that modifies OpenAI's Whisper to produce more reliable timestamps, adding transcription, forced alignment, and audio inde… | 10 | 2281 | maintenance |
| rsennrich/subword-nmt A Python library and CLI toolset for unsupervised word segmentation into subword units, best known for byte pair encoding (BPE) used in neu… | 23 | 2277 | maintenance |
| crownpku/Information-Extraction-Chinese A collection of deep learning models for Chinese information extraction, including named entity recognition with IDCNN or biLSTM+CRF and re… | 32 | 2263 | maintenance |
| TigerResearch/TigerBot TigerBot is a multi-language, multi-task large language model project from TigerResearch, providing pretrained and chat-tuned model weights… | 30 | 2262 | maintenance |
| namisan/mt-dnn A PyTorch library implementing Multi-Task Deep Neural Networks (MT-DNN) for natural language understanding, based on Microsoft research pap… | 32 | 2256 | maintenance |
| microsoft/DeBERTa Official implementation of DeBERTa and DeBERTa V3, transformer-based language models with disentangled attention and decoding-enhanced BERT… | 23 | 2248 | maintenance |
| chartbeat-labs/textacy textacy is a Python NLP library built on spaCy that handles tasks before and after spaCy's core pipeline, such as text cleaning, informatio… | 23 | 2240 | maintenance |
| lonePatient/BERT-NER-Pytorch A PyTorch library for Chinese Named Entity Recognition using BERT-based models with Softmax, CRF, and Span decoding heads. It also supports… | 32 | 2236 | maintenance |
| ProsusAI/finBERT FinBERT is a pre-trained BERT language model further trained on financial text and fine-tuned for financial sentiment classification. It is… | 32 | 2236 | maintenance |
| galaxykate/tracery Tracery is a JavaScript story-grammar generation library that expands grammar objects into generative text. You define rules with symbols a… | 32 | 2223 | maintenance |
| Turing-Project/AntiFraudChatBot A prompt-based Chinese conversational AI framework that connects a fine-tuned large language model (YUAN-1.0) to WeChat via the wechaty fra… | 23 | 2220 | maintenance |
| plurals/pluralize A small JavaScript library that pluralizes or singularizes English words based on a count, using an ordered list of predefined rules. It su… | 23 | 2216 | maintenance |
| BioBERT BioBERT is a pre-trained biomedical language representation model (based on BERT) with code for fine-tuning it on biomedical text mining ta… | 32 | 2213 | maintenance |
| epfLLM/meditron Meditron is a suite of open-source medical large language models (7B and 70B) adapted from Llama-2 via continued pretraining on a curated m… | 27 | 2211 | maintenance |
| allenai/longformer Longformer is a pretrained transformer model family (including the LongformerEncoderDecoder/LED variant) that processes long documents up t… | 23 | 2208 | maintenance |
| google-research/uda Google Research's reference implementation of Unsupervised Data Augmentation (UDA), a semi-supervised learning method that uses advanced da… | 10 | 2205 | maintenance |
| alibaba/EasyNLP EasyNLP is a comprehensive PyTorch-based NLP toolkit from Alibaba that provides training, inference, and deployment for pre-trained languag… | 23 | 2186 | maintenance |
| symfony/inflector A Symfony component that converts English words between their singular and plural forms. It has been deprecated since Symfony 5.1 in favor … | 10 | 2169 | maintenance |
| ageitgey/node-unfluff A Node.js library and CLI tool that automatically extracts the main body content and metadata (title, author, date, images, tags, links) fr… | 32 | 2160 | maintenance |
| songyouwei/ABSA-PyTorch PyTorch implementations of aspect-based sentiment analysis (ABSA) models, including attention-based and BERT-based approaches for aspect-te… | 10 | 2114 | maintenance |
| observerss/textfilter A small Python library implementing several sensitive-word filtering algorithms (including DFA) bundled with a ~10k-word sensitive word lis… | 32 | 2111 | maintenance |
| facebookresearch/SentEval SentEval is a Python library from Facebook Research for evaluating the quality of sentence embeddings via a suite of downstream transfer ta… | 10 | 2111 | maintenance |
| asappresearch/sru SRU is a PyTorch library implementing the Simple Recurrent Unit, a highly parallelizable RNN that runs 10-16x faster than cuDNN LSTM with c… | 23 | 2108 | maintenance |
| jalammar/ecco Ecco is a Python library for exploring and explaining Transformer-based NLP language models through interactive visualizations inside Jupyt… | 23 | 2104 | maintenance |
| THULAC THULAC is a Chinese lexical analysis toolkit from Tsinghua University's NLP lab that performs Chinese word segmentation and part-of-speech … | 32 | 2082 | maintenance |
| THUDM/P-tuning-v2 P-tuning v2 is a Python implementation of deep prompt tuning, applying trainable continuous prompts at every transformer layer so prompt tu… | 32 | 2079 | maintenance |
| UniversalDataTool/universal-data-tool Universal Data Tool is a web and desktop application for collaboratively annotating images, text, audio, and documents, supporting tasks li… | 24 | 2073 | maintenance |
| koth/kcws A deep learning Chinese word segmentation toolkit implemented in C++ with TensorFlow, supporting BiLSTM+CRF and IDCNN+CRF models, plus part… | 23 | 2068 | maintenance |
| weixsong/elasticlunr.js Elasticlunr.js is a lightweight full-text search engine library written in JavaScript, forked from Lunr.js with added flexibility. It suppo… | 32 | 2066 | maintenance |
| amazon-science/auto-cot Official implementation of Auto-CoT (ICLR 2023), a method that automatically constructs chain-of-thought demonstrations for large language … | 32 | 2052 | maintenance |
| marcotcr/checklist CheckList is a Python library for behavioral testing of NLP models, implementing the ACL 2020 paper 'Beyond Accuracy: Behavioral Testing of… | 32 | 2051 | maintenance |
| alibaba/AliceMind AliceMind is Alibaba's collection of pre-trained encoder-decoder language models and related NLP techniques, including StructBERT, PALM, VE… | 23 | 2040 | maintenance |
| xlang-ai/instructor-embedding INSTRUCTOR is an instruction-finetuned text embedding model and Python library that generates task-tailored embeddings by simply providing … | 32 | 2026 | maintenance |
| hila-chefer/Transformer-Explainability Official PyTorch implementation of the CVPR 2021 paper 'Transformer Interpretability Beyond Attention Visualization', providing relevance-b… | 32 | 2019 | maintenance |
| baidu/Senta Senta is Baidu's open-source sentiment analysis system built on the SKEP (Sentiment Knowledge Enhanced Pre-training) model. It provides pre… | 32 | 2016 | maintenance |
| anthropics/jacobian-lens A Python library implementing the Jacobian lens, an interpretability technique that linearly transports residual-stream activations at any … | 56 | 2011 | maintenance |
| salesforce/awd-lstm-lm A PyTorch toolkit for training LSTM and QRNN language models, implementing Salesforce Research's AWD-LSTM papers. It supports word-level an… | 10 | 1990 | maintenance |
| HazyResearch/deepdive DeepDive is a Stanford-developed system for extracting structured data from unstructured sources and building knowledge bases using distant… | 23 | 1979 | maintenance |
| MojoJolo/textteaser TextTeaser is an automatic text summarization algorithm written in Scala that combines natural language processing and machine learning to … | 32 | 1964 | maintenance |
| Named Entity Recognition with Tensorflow A TensorFlow implementation of a Bi-LSTM + CRF model with character embeddings for named entity recognition and sequence tagging. It trains… | 32 | 1952 | maintenance |
| psal/anonymouth Anonymouth is a Java-based desktop application that helps users anonymize documents by detecting and suggesting edits to stylometric patter… | 32 | 1950 | maintenance |
| ysc/QuestionAnsweringSystem A Java implementation of an open-domain question answering system inspired by IBM Watson, which analyzes questions and returns ranked candi… | 32 | 1947 | maintenance |
| Julius Speech Recognition Engine Julius is an open-source large vocabulary continuous speech recognition (LVCSR) decoder written in C, based on word N-gram language models … | 31 | 1942 | maintenance |
| google/sling SLING is a natural language frame semantics parser that annotates text with frame semantic graph representations using bi-directional LSTMs… | 10 | 1932 | maintenance |
| PandaOCR PandaOCR is a free Windows desktop OCR tool that captures screen regions and recognizes text using many cloud OCR engines (Sogou, Tencent, … | 71 | 1925 | maintenance |
| Tencent/NeuralNLP-NeuralClassifier NeuralClassifier is a PyTorch-based open-source toolkit for neural text classification, with a focus on hierarchical multi-label classifica… | 52 | 1924 | maintenance |
| Tiiiger/bert_score BERTScore is a PyTorch implementation of the BERTScore automatic evaluation metric for text generation, which scores generated text against… | 23 | 1920 | maintenance |
| appvision-ai/fast-bert Fast-Bert is a Python deep learning library for training and deploying BERT, RoBERTa, and XLNet based models for NLP tasks, starting with m… | 23 | 1915 | maintenance |
| Mimino666/langdetect A Python port of Google's language-detection library that identifies the language of text, supporting 55 languages via ISO 639-1 codes. It … | 23 | 1903 | maintenance |
| NCRF++ NCRF++ is a PyTorch-based toolkit for neural sequence labeling tasks such as NER, POS tagging, and word segmentation. Model architectures c… | 32 | 1901 | maintenance |
| mourner/bullshit.js Bullshit.js is a JavaScript bookmarklet that translates corporate marketing jargon into plain, human-readable text on any web page. It runs… | 47 | 1899 | maintenance |
| microsoft/BlingFire Bling Fire is a fast finite state machine and regular expression manipulation library from Microsoft, primarily used for high-performance n… | 23 | 1892 | maintenance |
| scrapy/scrapely Scrapely is a pure-Python library for extracting structured data from HTML pages. It learns a parser from example pages annotated with the … | 32 | 1883 | maintenance |
| nlplab/brat brat is a web-based collaborative text annotation tool for creating text-bound and relational annotations, originally built for NLP and Bio… | 32 | 1879 | maintenance |
| ANYbotics/elevation_mapping A ROS package for robot-centric elevation mapping of rough terrain, built for mobile and legged robots equipped with pose estimation and di… | 32 | 1878 | maintenance |
| salesforce/ctrl CTRL is a 1.63 billion-parameter conditional Transformer language model from Salesforce that generates text controllable via control codes … | 10 | 1877 | maintenance |
| WASasquatch/was-node-suite-comfyui An extensive custom node suite for ComfyUI with over 210 nodes covering image processing, masking, text processing, animation, and AI-assis… | 76 | 1875 | maintenance |
| clue-ai/ChatYuan ChatYuan is an open-source bilingual (Chinese and English) dialogue large language model in ChatGPT style, with a lightweight variant runna… | 32 | 1860 | maintenance |
| nlpodyssey/spago Spago is a self-contained machine learning and NLP library written in pure Go, featuring an asynchronous computation graph with automatic d… | 26 | 1849 | maintenance |