Every small business has the same quiet problem: the answer to a customer’s question lives in a PDF, a pricing sheet, or a senior employee’s head, and nobody can find it twice. A self-hosted AI knowledge base lets staff ask in plain language and get answers grounded in your own documents, without sending them to a public chatbot.
This guide builds one with Open WebUI and Pipelines. Open WebUI is a self-hosted, extensible AI platform that can run entirely offline and works with Ollama and OpenAI-compatible APIs, according to its official documentation. Pipelines is its OpenAI-compatible plugin framework for custom logic, retrieval, and filters.
What you are actually building
Think of three layers. Open WebUI is the chat interface your team uses. Pipelines sits between that interface and the model, so you can shape how requests are handled. Your knowledge base is the set of approved documents the system can draw on. When a question arrives, the system retrieves relevant passages and answers from them, rather than guessing from general training data alone. You can then update a document or remove an outdated price list without retraining a model.
If you have not installed Open WebUI yet, start with our guide to using Open WebUI as a self-hosted chat interface. For local models, see our Ollama guide.
Prerequisites
- A computer or small server that can stay on, with Docker installed. A spare office PC, mini-PC, or modest virtual private server is enough for a pilot.
- One model: a local model through Ollama, or an OpenAI-compatible API you already use.
- Five to ten short, current documents, such as procedures, FAQs, product sheets, and delivery policies. Remove drafts, duplicates, and anything containing passwords or sensitive personal data.
- Admin access, a backup habit, and one person who owns document updates.
Step-by-step setup
Step 1: Install Open WebUI and Pipelines
Docker keeps the two services separate and easy to update. Run Open WebUI first, then Pipelines. The official quick start maps Open WebUI to port 3000 on the host, and Pipelines to port 9099, with a shared pipelines volume.
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
docker run -d -p 9099:9099 --add-host=host.docker.internal:host-gateway -v pipelines:/app/pipelines --name pipelines ghcr.io/open-webui/pipelines:mainOpen http://localhost:3000, create your admin account, and confirm the interface loads. If port 3000 is already in use, stop the conflicting service or change the host port.
Step 2: Connect Pipelines to Open WebUI
In Open WebUI, go to Settings, then Connections, then OpenAI API. Add a connection with the Pipelines API URL http://localhost:9099 and the default API key 0p3n-w3bu!. If Open WebUI itself runs in Docker, use http://host.docker.internal:9099 instead, because localhost inside its container does not point to your host machine. Save, then check that the connection appears without an error.
Step 3: Choose and test your model
Pick one model for the pilot. A local model keeps prompts on your own machine, while a cloud API sends prompts to that provider, so choose according to your privacy policy. Ask three simple questions about your business and confirm the answers are sensible before you add any documents.
Step 4: Build one knowledge collection
In Open WebUI, create a Knowledge collection with a clear name, such as Support Playbook, and upload only approved documents from your prerequisites list. Give each document a clear title and date, and keep one version of each procedure. Attach the collection to your test chats, then ask questions in the words customers actually use, not the exact headings in the files. If an answer uses an old document, remove or correct that document rather than adding a longer prompt.
Step 5: Add a retrieval pipeline, only if you need it
Pipelines are powerful, but they are not always needed. The Pipelines documentation itself warns against using Pipelines simply to add another provider, because Open WebUI already supports many providers directly. Use a pipeline when you need custom retrieval, a filter, or a workflow the standard interface cannot do. In Admin Settings, open the Pipelines tab, select a pipeline from a source you trust, and adjust its valves, which are the settings exposed by the interface. Pipelines run arbitrary code, so never install a random pipeline from an untrusted link. Start with one well-understood example, change one setting at a time, and re-run your test questions after each change.
Step 6: Test like a sceptical customer
Write ten real questions before you invite the wider team in. Include edge cases, such as an expired promotion, a discontinued product, and a question your documents do not answer. A good knowledge base says it does not know, or points to the right human, rather than inventing a confident answer. For every wrong answer, fix the document, not just the prompt. Then re-test the same question.
Step 7: Lock it down and keep it current
Give access only to staff who need it, use strong admin passwords, and do not expose the interface to the public internet unless you have a clear reason and proper authentication in front of it. Back up the Open WebUI and Pipelines volumes, and keep a simple monthly routine: remove expired documents, add new procedures, and re-run your ten test questions. A stale knowledge base is worse than no knowledge base, because staff will trust it anyway.
Tips that save small teams time
- Start with one collection and one team. Expand only after the pilot answers reliably.
- Write short documents with one topic each, and put the answer near the top.
- Keep a changelog for price, policy, and procedure documents, so outdated answers are easy to trace.
- Never upload passwords, private customer records, or documents you do not have permission to share.
FAQ
Do I need a graphics card? Not for a small pilot. A cloud API, or a small local model, can run without one. A graphics card helps larger local models run faster, but it is not a requirement to learn the workflow.
Is my data sent to an outside AI company? It depends on the model you choose. A fully local setup keeps prompts on your machine, while a cloud API sends prompts to that provider. Check the provider you connect before uploading sensitive documents.
Can Pipelines replace Open WebUI? No. Pipelines is a plugin framework for OpenAI-compatible clients, and Open WebUI is the interface in this guide. You still need a client to use a pipeline.
What should I do when an answer is wrong? Check the source document first, correct or remove it, and test again. If the document is correct but the answer is still wrong, simplify the document and add the exact question to your test list.
Sources
- Open WebUI documentation: https://docs.openwebui.com/
- Open WebUI Pipelines on GitHub: https://github.com/open-webui/pipelines
