dotnet/spark

Health v2 · maintenance only

Adoption not part of the score

2098 stars · 334 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

.NET for Apache Spark provides high-performance C# and F# bindings for Apache Spark, exposing DataFrames, SparkSQL, and Structured Streaming to .NET developers. It is a .NET Standard library that runs on Windows, Linux, and macOS and on major cloud Spark services like Azure HDInsight, Databricks, and Amazon EMR.

Use cases

• process big data with spark from c#
• run spark sql queries using .net
• build etl pipelines in c# on spark
• stream processing with spark structured streaming in .net
• run .net spark jobs on databricks or emr
• analyze large datasets with f# and spark

When to choose

• your team is .NET-based and needs distributed big data processing
• you want to reuse existing .NET skills and libraries on Spark
• you deploy Spark workloads on Azure HDInsight, Databricks, or EMR

When to avoid

• you primarily use Python or Scala, where native Spark APIs are richer
• you need the newest Spark features immediately, as .NET bindings lag behind
• your project needs a very active community; the project is in maintenance mode

Facets

library · maturity maintenance

etl streaming data-science machine-learning analytics big-data data-science microservices analytics cross-platform windows dotnet cloud apache-spark spark-sql structured-streaming dataframe csharp fsharp databricks azure-hdinsight amazon-emr net-standard data-engineering linux macos docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="dotnet/spark")