colinmarc/hdfs

Health v2 · maintenance only

Adoption not part of the score

1403 stars · 361 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A native Go client for HDFS that connects directly to the namenode via its protocol buffers API, mimicking the stdlib os package interfaces. It also ships a fast commandline client offering POSIX-like verbs (ls, rm, get, put) as a JVM-free alternative to `hadoop fs`.

Use cases

• read and write files on HDFS from a Go program
• list, copy, and delete files on a Hadoop cluster from the shell
• replace slow `hadoop fs` commands with a fast native binary
• script HDFS operations without a JVM installed
• programmatically inspect HDFS file metadata like os.FileInfo

When to choose

• you need HDFS access from Go without shelling out to hadoop or the JVM
• you want fast, scriptable HDFS shell commands with bash completion
• you prefer idiomatic Go APIs mirroring the os package

When to avoid

• your cluster requires Kerberos or advanced Hadoop features not supported by the client
• you need a JVM-based Hadoop ecosystem toolchain (Hive, Spark integration)
• the project lacks an active maintainer and you need guaranteed long-term support

Facets

library · maturity maintenance

database-driver cli file-system big-data developer-tools microservices go cross-platform cli hdfs hadoop namenode protocol-buffers big-data-storage unix-tools command-line linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="colinmarc/hdfs")