microsoft/aici

Health v2 · maintenance only

Adoption not part of the score

2076 stars · 86 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

AICI (Artificial Intelligence Controller Interface) is a Microsoft Research prototype that lets developers write Wasm-based controllers to constrain and direct LLM output token-by-token during inference. It integrates with engines like llama.cpp and HuggingFace Transformers, enabling constrained decoding, prompt editing, and multi-generation coordination.

Use cases

• constrain LLM output to valid JSON or grammar formats
• run guided decoding with regex or CFG constraints
• coordinate multiple parallel LLM generations
• dynamically edit prompts and generated text during decoding
• build custom decoding strategies on top of llama.cpp or HuggingFace
• implement multi-agent LLM conversations with stateful control

When to choose

• you need fine-grained, programmatic control over token-by-token LLM generation
• you want sandboxed controllers that run alongside the inference engine
• you need portability of decoding logic across multiple LLM serving engines

When to avoid

• you only need constrained decoding - use the actively maintained LLGuidance instead
• you want a production-ready, stable LLM serving framework
• you need a simple prompt library without runtime decoding control

Facets

framework · maturity experimental

llm-inference wasm prompt-engineering rag large-language-models artificial-intelligence machine-learning developer-tools rust wasm self-hosted constrained-decoding llm-serving wasm-controllers inference-engine microsoft-research token-generation linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/aici")