slvDev/esp32-ai resource

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

4175 stars · 553 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A project demonstrating a 28.9M parameter language model running entirely on an ESP32-S3 microcontroller at ~9.88 tokens per second, using Per-Layer Embeddings (from Gemma 3n) to keep most weights in flash. It includes trained models (TinyStories, Barista) and serves as a reference for fitting LLMs onto memory-constrained hardware.

Use cases

• run an LLM on an ESP32 microcontroller
• generate text offline on embedded hardware
• learn how Per-Layer Embeddings let large models fit in tiny memory
• experiment with on-device AI without cloud connectivity
• deploy a small language model on a microcontroller with 512KB SRAM

When to choose

• you want to run or study LLM inference on ESP32-S3 or similar microcontrollers
• you need fully offline text generation on embedded hardware
• you want a working example of flash-resident embedding tables and tiered memory layouts

When to avoid

• you need a model that answers questions, follows instructions, or writes code
• you want production-quality general-purpose LLM inference
• your target hardware lacks PSRAM and multi-megabyte flash

Facets

learning-resource · maturity active

llm-inference embedded machine-learning embedded-systems large-language-models hardware embedded iot python esp32 microcontroller tinyml on-device-inference per-layer-embeddings gemma-3n tiny-stories edge-ai

Member repositories

For agents

markdown · JSON · MCP: product_card(name="slvDev/esp32-ai")