Your business documents hold the answers your team and customers keep asking for — but they’re buried in PDFs and shared drives. AnythingLLM turns those files into a private, ChatGPT-style assistant that answers from your own documents, with citations. This tutorial shows you how to install AnythingLLM, connect a language model, and build a document chatbot your whole team can use.
What It Is
AnythingLLM is an all-in-one, open-source AI application for chatting with your documents and running AI agents, designed to be private and self-hosted by default. Its pitch is simple: “Stop renting your intelligence. Own it.” Out of the box you get document ingestion, built-in agents, multi-user support, vector databases, and document pipelines with no extra configuration. As checked on October 18, 2026, it has 67,262 stars on GitHub (github.com/Mintplex-Labs/anything-llm) and is licensed under the MIT License.
Key capabilities from the README: a built-in local LLM engine that runs models on your CPU or GPU with no extra setup, dynamic model routing, automatic and user-managed memories, scheduled tasks, a no-code AI agent builder, MCP compatibility, multi-modal support, and multi-user instances with per-user permissions (Docker version). It works with all popular LLM providers — OpenAI, Anthropic, Google Gemini, Ollama, LM Studio, LocalAI, DeepSeek, Mistral, Groq, and many more — plus multiple vector databases (LanceDB by default, with Pinecone, Qdrant, Weaviate, Chroma, and others). Documents supported include PDF, TXT, DOCX, and more, with drag-and-drop uploads and source citations in answers.
Who It’s For / SMB Use Case
AnythingLLM fits small businesses that want AI answers grounded in their own content, without sending sensitive files to a third party. Practical uses include:
- Customer-facing doc chatbot: Embed a chat widget on your website (Docker version) that answers product and policy questions from your documentation, with citations.
- Staff knowledge base: Let employees ask questions about HR policies, SOPs, and training materials in plain language.
- Sales enablement: Give the sales team an assistant trained on datasheets, pricing, and case studies so answers are consistent.
- Research helper: Upload contracts, reports, or research and ask for summaries, comparisons, and key clauses.
AnythingLLM is not the right pick if you want a fully managed service with zero maintenance — it shines when you self-host, which means someone needs to keep the instance updated. The desktop app lowers the bar, but multi-user and website-embed features need the Docker deployment.
Installation
You have two main paths. The desktop app (Mac, Windows, Linux) is the easiest: download it from anythingllm.com and install like any app. For teams, the Docker deployment is recommended because it enables multi-user support and the embeddable website chat widget.
Docker install (recommended for teams)
Clone the repository and start the stack from the docker folder:
git clone https://github.com/Mintplex-Labs/anything-llm.git cd anything-llm/docker docker compose up -d
Open your browser at http://localhost:3001 and complete the setup. The README also documents deployments for AWS, GCP, DigitalOcean, Railway, Render, and others if you prefer a cloud server. Note the README’s privacy section: AnythingLLM collects anonymous telemetry by default; set DISABLE_TELEMETRY=true in your .env (or disable it in the app under Privacy) to opt out.
Desktop install (simplest for one person)
Download the installer for your OS from anythingllm.com, run it, and you’re ready — the desktop build includes the local LLM engine, so you can run models on your own machine with no extra setup.
How to Use AnythingLLM
Step 1: Choose your language model
Go to Settings → LLM and pick a provider. For maximum privacy, choose the built-in local engine or Ollama to run models on your own hardware. For maximum quality, connect an API key from OpenAI, Anthropic, or Google. You can also set an embedder model (the native embedder works by default) and a vector database — LanceDB is the default and needs no setup.
Step 2: Create a workspace
A workspace is a themed collection of documents plus its own chat settings — for example “Customer Support Docs” or “HR Handbook”. Create one per topic or team so answers stay focused. Each workspace can have its own system prompt, temperature, and chat history settings.
Step 3: Upload your documents
Drag and drop PDFs, DOCX files, or TXT files into the workspace. AnythingLLM processes and embeds them automatically; the document manager shows what’s ingested. Quality matters more than quantity: upload current, well-organized documents and remove outdated versions, because the AI can only be as accurate as what you feed it.
Step 4: Chat with citations
Ask questions in the chat UI. Answers come with source citations pointing to the document sections used, so you can verify every claim — train your team to click through citations on anything important. Try the agent mode for tasks that need tools, like browsing or multi-step lookups.
Step 5: Share it (Docker version)
Invite team members with per-user permissions, or use the embeddable chat widget to put the assistant on your website. There’s also a full developer API for custom integrations. Monitor usage and keep documents fresh — schedule a monthly review of what’s uploaded.
Pricing / Limitations
AnythingLLM is free and open source under the MIT License — no license fees, no seat fees, no feature gates in the code. Your costs are hosting (a server or just your own computer for the desktop app) and any cloud LLM API usage if you choose a hosted model instead of a local one. Mintplex Labs also offers a hosted instance for those who don’t want to self-host; check their site for current pricing.
Limitations: answer quality depends on your documents and your chosen model — a small local model on weak hardware will be slower and less capable than a flagship cloud model. Multi-user features and the website widget require the Docker deployment, not the desktop app. Anonymous telemetry is on by default (opt out with DISABLE_TELEMETRY=true). And like all RAG systems, it can still misread documents, so keep citations visible and humans in the loop for high-stakes answers.
Bottom Line for SMB
AnythingLLM is the simplest way to get a private, document-grounded AI assistant without building infrastructure or sending files to a vendor. Start with one document set that generates constant questions — your FAQ or product docs — prove the answers are good, then expand to teams and your website. If you prefer running models fully offline first, read our Ollama tutorial, and for background on how AI agents fit into a small business, see our AI agents guide.
Frequently Asked Questions
Does AnythingLLM send my documents to the cloud?
Not if you self-host with a local model — documents stay on your machine or server. If you connect a cloud LLM provider (like OpenAI or Anthropic), your prompts and retrieved document chunks are sent to that provider to generate answers, so choose a local model for sensitive content.
What’s the difference between the desktop app and Docker version?
The desktop app is the easiest single-user setup with a built-in local engine. The Docker version adds multi-user support with permissions, the embeddable website chat widget, and easier server deployment — it’s the right choice for teams.
How do I turn off telemetry?
Set DISABLE_TELEMETRY=true in your server or Docker .env settings, or open the app sidebar, go to Privacy, and disable telemetry there. The README lists exactly what is tracked (installation type, document add/remove events, provider types, chat-sent events — never content or IPs).
