XTXMarkets/ternfs

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

1339 stars · 96 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

TernFS is an exabyte-scale, multi-region distributed file system written in C++, designed for storing large immutable files on commodity hardware and Ethernet networking. It is used in production at XTX Markets for machine learning workloads and supports trillions of files, replication across data centers, and recovery from node failures or bitrot.

Use cases

• store petabyte or exabyte-scale ML training datasets
• build a multi-region replicated file system across data centers
• serve large immutable files to thousands of compute clients
• replace HDFS or similar HPC storage with commodity hardware
• recover accidentally deleted files with a configurable deletion policy

When to choose

• you need to store and read very large mostly-immutable files at extreme scale
• you want multi-datacenter replication to survive a full datacenter failure
• you prefer commodity hardware and Ethernet over specialized storage appliances
• you need strong durability guarantees against bitrot and node failure

When to avoid

• your workload involves frequently modified small files or heavy directory churn
• you need a mature 1.0 filesystem with long-term API stability
• you need POSIX compliance for arbitrary application workloads
• you want a plug-and-play solution rather than operating your own distributed storage cluster

Facets

service · maturity active

file-system microservices microservices machine-learning big-data cpp self-hosted distributed-file-system exabyte-scale multi-region immutable-files erasure-coding posix commodity-hardware storage infrastructure linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="XTXMarkets/ternfs")