s3git/s3git
s3git: git for Cloud Storage. Distributed Version Control for Data. Create decentralized and versioned repos that scale infinitely to 100s of millions of files. Clone huge PB-scale repos on your local SSD to make changes, commit and push back. Oh yeah, it dedupes too and offers directory versioning. observed · 2026-08-28
Health v2 · maintenance only
23/100
- Activity 0
- Release rhythm 8
- Longevity 100
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 3838
- days_rel: n/a
- days_push: 3684
- n_releases_24m: 0
Adoption not part of the score
1461 stars · 63 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
s3git is a Go CLI tool that applies git-like distributed version control to data stored in S3-compatible cloud storage, supporting huge repos with deduplication and directory versioning. It requires no server-side components and offers Go, Python, and Ruby libraries.
Use cases
- version control large datasets in cloud storage
- clone petabyte-scale repos locally to make changes
- deduplicate and version directories of files
- manage build and release artifacts like Kubernetes releases
- devops data pipelines with versioned storage
- store and version photo and video collections
When to choose
- you want git-like workflows for large data in S3
- you need decentralized versioning without server components
- you need deduplication across millions of files
When to avoid
- you need actively maintained software (last release 2016)
- you need git features like branching workflows for source code
- you use non-S3 storage backends
Facets
cli-tool · maturity abandoned
version-control cli file-system compression backup developer-tools cloud-computing files version-control windows cli go git-for-cloud-storage s3 data-versioning deduplication decentralized command-line linux macos
1 source
- readme: https://github.com/s3git/s3git · fetched 2026-08-28 · 0bf7f1abc004
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| s3git/s3git | main | 23 |
For agents
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem