Large Language Models

Large language models are AI systems capable of understanding and generating human language by processing vast amounts of text data.

Types of LLMs in AnythingLLM

AnythingLLM allows you to get as specific or general as you want with your LLM selection. You can even have multiple LLMs configured at the same time all in the same application!

System LLM

This is the default LLM AnythingLLM will interface with. This is the LLM configuration that will be used when a workspace or agent-specific agent LLM has not been defined.

Workspace LLM

AnythingLLM allows you to set workspace-specific LLMs, this will override the system LLM but only when chatting with the specific workspace. This allows you to have many workspaces that each have their own provider, model, or both!

Agent LLM

AnythingLLM supports AI-agents. When it comes to agents, not all LLMs were created equal. Some LLMs directly support tool calling for better ai-agent functionality. The model is the model that is explicitly used for use with agents.

Supported LLM Providers

AnythingLLM supports many LLMs out of the box with very little, if any setup.

The LLM is the foundational integration that will determine how your workspace or agents respond to your questions and prompts.

You can modify your LLM provider, model, or any other details at any time in AnythingLLM with no worry.

We allow you to connect to both local and cloud-based LLMs - even at the same time!

Local Language Model Providers

Built-in (default)→Ollama→LM Studio→Local AI→KobaldCPP→

Cloud Language Model Providers

OpenAI→Azure OpenAI→Anthropic→Cohere→Google Gemini Pro→Hugging Face→Together AI→OpenRouter→Perplexity AI→Mistral API→Groq→OpenAI (generic)→