microsoft/SynapseML

Health v2 · maintenance only

Adoption not part of the score

5247 stars · 868 forks observed · 2026-09-29

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

SynapseML (formerly MMLSpark) is an open-source machine learning library built on Apache Spark that provides simple, composable, distributed APIs for tasks like text analytics, vision, and anomaly detection. It shares the SparkML/MLlib API so models integrate seamlessly into existing Spark workflows across Python, R, Scala, Java, and .NET.

Use cases

• train machine learning models on large distributed Spark clusters
• run distributed LightGBM training on Spark DataFrames
• integrate Azure Cognitive Services into Spark pipelines
• build scalable text analytics and NLP pipelines
• perform distributed deep learning and ONNX model inference
• detect anomalies at scale in big data workflows
• deploy and evaluate ML models on multi-node clusters

When to choose

• you already use Apache Spark and want ML pipelines with the same API
• you need distributed training of algorithms like LightGBM on big data
• you want to combine cloud AI services with Spark workflows
• you need multi-language access (Python, R, Scala, Java, .NET) to ML tools

When to avoid

• you work on a single machine with small datasets where scikit-learn suffices
• you need a lightweight library without JVM/Spark dependencies
• your project does not use Spark or a distributed computing framework

Facets

library · maturity active

machine-learning deep-learning nlp computer-vision etl data-science machine-learning big-data data-science deep-learning artificial-intelligence jvm python cloud apache-spark pyspark distributed-ml lightgbm onnx databricks azure mmlspark spark

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/SynapseML")