# palantir/pyspark-style-guide

This is a guide to PySpark code style presenting common situations and the associated best practices based on the most frequent recurring topics across the PySpark repos we've encountered.

Repository: https://github.com/palantir/pyspark-style-guide
Canonical: https://ross.abutalabs.com/products/pyspark-style-guide
Language: Python
License: MIT
License Family: permissive
Topics: octo-correct-managed
Last push: 2025-09-08T18:43:48+00:00

## Health v2 (maintenance only)
Score: 51/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 41, release rhythm 35, longevity 100
- inputs: {"age_days": 2148, "days_push": 359, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1260, forks 169 (observed 2026-08-28T04:04:09.926718+00:00)

## What it is
An opinionated style guide for PySpark code from Palantir, presenting common situations and best practices drawn from recurring patterns across many PySpark repositories. It is documentation rather than software, aimed at helping Python developers write maintainable Spark code.

## Use cases
- how to write clean pyspark code
- pyspark column selection best practices
- pyspark code style guide
- best way to reference columns in spark dataframes
- pyspark coding conventions for teams
- learn pyspark idioms for large data pipelines

## When to choose
- you are writing or reviewing PySpark code and want opinionated best practices
- you are standardizing code style across a team's Spark repositories
- you are an experienced Python developer new to Spark's JVM-influenced patterns

## When to avoid
- you need runnable tooling, linters, or formatters rather than prose guidance
- you work with Scala or Java Spark rather than the PySpark API
- you need a general Python style guide rather than Spark-specific advice

## Facets
- artifact type: learning-resource
- maturity: active
- function: documentation, developer-tools
- domain: big-data, developer-tools, tutorials
- platform: python
- tags: pyspark, style-guide, code-style, spark, best-practices, clean-code, data-engineering

## Member repositories
- palantir/pyspark-style-guide (main) score 51

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:09.926718+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T05:04:24.144084+00:00, confidence not recorded.
  - readme: https://github.com/palantir/pyspark-style-guide (fetched 2026-08-28T04:04:09.926718+00:00, sha 2cbd09eb8b45)
- Data as of 2026-08-30T08:39:29.467469+00:00.