XiongjieDai/GPU-Benchmarks-on-LLM-Inference resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1935 stars · 76 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A curated benchmark dataset comparing LLM inference speeds (tokens/s) across many NVIDIA GPUs and Apple Silicon chips using llama.cpp on LLaMA-family models. Results are presented as tables and Jupyter notebooks covering quantized and full-precision model sizes from 8B to 70B parameters.

Use cases

• which gpu should i buy for running llama locally
• compare inference speed of rtx 4090 vs a100 for llm
• can a macbook m1 max run llama 70b
• how fast is llama 3 8b on a 3090
• best gpu for local llm inference on a budget
• how many gpus do i need to run a 70b model
• apple silicon vs nvidia for llm inference

When to choose

• choosing hardware for local LLM inference with llama.cpp
• comparing consumer and datacenter GPUs for token generation speed
• checking whether a given GPU has enough VRAM for a model size and quantization

When to avoid

• benchmarking training performance rather than inference
• comparing inference frameworks other than llama.cpp
• needing up-to-date results for the newest GPU generations

Facets

dataset · maturity maintenance

benchmarking llm-inference gpu-computing large-language-models gpu-computing hardware performance llama-cpp nvidia-gpus apple-silicon tokens-per-second hardware-comparison runpod gpu macos linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="XiongjieDai/GPU-Benchmarks-on-LLM-Inference")