AnythingLLM
Use RouteMux as the LLM provider behind AnythingLLM workspaces.
AnythingLLM has a Generic OpenAI provider type designed for exactly this.
Configure
| Field | Value |
|---|---|
| LLM Provider | Generic OpenAI |
| Base URL | https://api.routemux.com/v1 |
| API Key | your RouteMux key (sk-...) |
| Chat Model Name | a slug from GET /v1/models |
| Token context window | match the model's real context — see its page under Models |
Set the context window honestly
AnythingLLM uses this number to decide how much retrieved context to pack into a prompt. Setting it higher than the model actually supports produces upstream errors on long documents; setting it lower silently truncates your retrieval quality.
Embeddings are separate
AnythingLLM's embedding provider is configured independently of the chat provider. If the
embedding model you pick is not in GET /v1/models, point embeddings elsewhere rather than
assuming the chat key covers it.
Cost notes
RAG workloads resend retrieved chunks on every turn, so token usage scales with chunk size and top-k, not just with conversation length. Usage per request is visible in the console logs.