rai

Level CSource

CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime.

⭐ 16DevOps & DeploymentRustRepo ↗Updated 2026-09-19

Installation

View source repo →

Related servers