aichat: One Rust CLI for 20+ LLM Providers

My notes

Summary

aichat is an open-source Rust CLI that unifies access to 20+ LLM providers (OpenAI, Claude, Gemini, Ollama, Groq, Bedrock, Mistral, Deepseek, OpenRouter, and more) behind one interface. It bundles a shell assistant that turns natural language into shell commands, a REPL chat mode, RAG over local files and URLs, custom roles and macros, and a lightweight built-in HTTP server exposing OpenAI-compatible chat, embeddings, and rerank endpoints plus a browser playground and a side-by-side “LLM Arena”.

Key Insight

• A single binary (cargo install aichat, brew install aichat, or a prebuilt release) replaces separate CLIs and SDK glue for each provider. One config, one interface, provider chosen per-role or per-call.
• Shell assistant mode adapts generated commands to the detected OS and shell automatically, which is useful as a lower-friction alternative to memorizing flags.
• Input handling is flexible: stdin, local files and directories, remote URLs, and even the output of external shell commands can all be piped in as context, in both one-shot CMD mode and REPL mode.
• “AI Agent = Instructions (Prompt) + Tools (Function Calling) + Documents (RAG)” is the tool’s explicit framing. It treats agents as a composition of three primitives rather than a separate abstraction, backed by a companion repo (sigoden/llm-functions) for building the function-calling tools.
• aichat --serve spins up a local OpenAI-compatible server (chat completions, embeddings, rerank) on port 8000, plus a web playground and an “Arena” endpoint for comparing two models side by side in-browser. Useful for local testing without touching cloud consoles.
• Roles bundle a prompt and model config together, letting you switch personas (each with its own default model and provider) without rewriting prompts each time.