skyzh/tiny-llm resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

4523 stars · 369 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A hands-on course for systems engineers to build an LLM inference and serving system from scratch on Apple Silicon using MLX, implementing a Qwen3 model, KV cache, and a mini vLLM-style server. It also covers building a coding agent, with exercises implemented in Python, C++, or Metal.

Use cases

• learn how LLM inference works end to end
• build a tiny vLLM-style serving system
• implement attention, RoPE, and KV cache from scratch
• write Metal kernels for LLM decode on Apple Silicon
• understand continuous batching and paged KV cache
• study LLM serving without reading a huge CUDA codebase

When to choose

• you are a systems engineer with PyTorch basics wanting to understand LLM serving internals
• you have an Apple Silicon Mac and no CUDA GPU
• you want a guided, test-verified path like CMU's Needle project

When to avoid

• you need a production-ready inference server
• you want to learn on NVIDIA CUDA hardware
• you are looking for a high-level framework rather than a course

Facets

learning-resource · maturity active

llm-inference machine-learning gpu-computing developer-tools large-language-models tutorials developer-tools machine-learning python llm-serving mlx apple-silicon qwen vllm kv-cache continuous-batching metal-kernels course hands-on macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="skyzh/tiny-llm")