spark-jobserver/spark-jobserver

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

2836 stars · 970 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Spark Job Server is a RESTful service for submitting, managing, and monitoring Apache Spark jobs, jars, and job contexts. It supports persistent Spark contexts, named RDDs, authentication via Shiro or Keycloak, and deployment on YARN, Mesos, and EMR clusters.

Use cases

• submit spark jobs over a rest api
• manage shared spark contexts for related jobs
• run ad-hoc spark jobs without a cluster console
• upload and version spark job jars
• secure spark job submission with authentication and access control
• deploy spark jobs on yarn or mesos

When to choose

• you need a long-running service to submit and monitor Spark jobs via HTTP
• you want persistent Spark contexts shared across multiple jobs for faster startup
• you need job management features like named RDDs, access control, and SSL

When to avoid

• you only need to run occasional Spark jobs and can use spark-submit directly
• you use Spark's newer native REST APIs or a workflow orchestrator like Airflow for job scheduling
• your stack is not JVM/Scala-based and you cannot package jobs as jars

Facets

service · maturity maintenance

http-server api-framework scheduling middleware big-data apis microservices jvm self-hosted apache-spark job-scheduler rest-api spark-jobs job-management data-engineering web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="spark-jobserver/spark-jobserver")