big-data-europe/docker-spark resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

2050 stars · 684 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A set of Dockerfiles and Docker Compose configurations for building Apache Spark Docker images that run a standalone Spark cluster with one master and multiple workers. It supports many Spark/Hadoop/OpenJDK version combinations and lets users build Spark applications in Java, Scala, or Python to run on the cluster.

Use cases

• spin up a standalone apache spark cluster with docker compose
• run spark master and worker containers for big data pipelines
• containerize spark applications in java scala or python
• deploy spark on kubernetes with docker images
• test spark jobs locally without installing spark
• integrate spark into a bde big data europe pipeline

When to choose

• you want a quick containerized spark cluster for development or testing
• you need specific spark hadoop and jdk version combinations as docker images
• you use docker compose or kubernetes to orchestrate spark master and workers
• you want to build spark apps against a consistent containerized environment

When to avoid

• you need a fully managed spark service like databricks or emr
• you require the newest spark version not yet covered by these images
• you prefer official apache spark binaries or spark kubernetes operator for production
• you need a license-cleared dependency since the repo has no explicit license

Facets

infra-config · maturity active

container-runtime deployment developer-tools data-science etl big-data microservices cross-platform apache-spark docker-images spark-cluster docker-compose spark-master spark-worker hadoop standalone-cluster big-data-europe containerization data-engineering containers devops docker kubernetes linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="big-data-europe/docker-spark")