Your business knowledge lives in PDFs, contracts, manuals, and spreadsheets, but a generic chatbot cannot read any of it. LlamaIndex is the open-source framework that connects your documents to a language model, so you can ask questions and get answers grounded in your own files. This tutorial shows you how to install it and build a simple question-answering index over your documents.
What It Is
LlamaIndex is a data framework for building LLM applications over your own data. It provides data connectors to ingest your existing sources, ways to structure that data as indexes, and a retrieval and query interface that feeds relevant context to a language model. The project is published under the MIT License and, as checked on October 17, 2026, has 52,536 stars on GitHub (github.com/run-llama/llama_index). Its official description is: “LlamaIndex is the document processing platform for AI.” One important note from the README: the company’s current focus is its LlamaParse document platform, while the open-source framework remains available as an open toolkit. Check the official documentation for the latest guidance before starting a large project.
Who It’s For / SMB Use Case
LlamaIndex is a developer tool. It fits small businesses that have a developer or agency and want AI answers based on their own documents rather than generic knowledge. Practical uses include:
- Ask your documents: Ask questions over SOPs, contracts, product manuals, or training files and get answers drawn from those files.
- Support drafts: Let support staff search policies and past answers in plain language instead of digging through folders.
- Onboarding helper: Give new hires a question-answering tool over the employee handbook and process docs.
- Local-model option: Pair LlamaIndex with a locally hosted model through Ollama to keep documents on your own machine.
LlamaIndex is not a finished search product you can hand to staff with no setup. It is a framework a developer wires into your tool. If nobody on your team writes code, a hosted document Q&A product will be faster.
Installation
LlamaIndex needs Python 3. You can install the starter package, which includes core LlamaIndex plus a selection of integrations, or install core plus only the integrations you need. The README’s example usage installs core with specific integrations:
pip install llama-index-core pip install llama-index-llms-openai pip install llama-index-llms-ollama pip install llama-index-embeddings-huggingface
If you use an OpenAI model, set your API key as an environment variable. The README example sets OPENAI_API_KEY in code; in a real project, set it in your environment instead of hard-coding it. There are over 300 LlamaIndex integration packages, according to the README, so install only the ones your project needs. If the README does not state something clearly, check the official docs at developers.llamaindex.ai.
How to Use LlamaIndex
Step 1: Put your documents in one folder
Create a folder, for example data, and copy in the documents you want to search: PDFs, Word files, or plain text. Start with a small, clean set of current documents. Remove duplicates and outdated versions first, because the index will happily answer from stale files if you leave them in.
Step 2: Build a vector store index from your documents
This example from the README loads a folder of documents and builds an index using OpenAI:
import os
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index = VectorStoreIndex.from_documents(documents)
Replace "YOUR_DATA_DIRECTORY" with your folder path and "YOUR_OPENAI_API_KEY" with your key, preferably loaded from an environment variable. SimpleDirectoryReader loads the files; VectorStoreIndex turns them into a searchable index.
Step 3: Ask questions through a query engine
Turn the index into a query engine and ask a question, as shown in the README:
query_engine = index.as_query_engine()
query_engine.query("YOUR_QUESTION")
Start with questions you already know the answers to, and verify the answers against the source documents. If answers are vague, add more specific documents, or narrow the folder to the most relevant files.
Step 4: Save the index and reload it later
By default, data is stored in memory. To persist the index to disk under ./storage, use:
index.storage_context.persist()
To reload it later, use the snippet from the README:
from llama_index.core import StorageContext, load_index_from_storage # rebuild storage context storage_context = StorageContext.from_defaults(persist_dir="./storage") # load index index = load_index_from_storage(storage_context)
Rebuilding the index from scratch on every run wastes time and API calls, so persist it once your document set is stable.
Step 5: Run it locally with Ollama (optional)
To avoid sending documents to a hosted provider, the README shows using Ollama as the LLM with a local embedding model. The key lines set the LLM, the tokenizer, and the embedding model:
from llama_index.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama
from transformers import AutoTokenizer
# set the LLM
Settings.llm = Ollama(
model="llama-3.1:latest",
request_timeout=360.0,
)
# set tokenizer to match LLM
Settings.tokenizer = AutoTokenizer.from_pretrained(
"meta-llama/Llama-3.1-8B-Instruct"
)
# set the embed model
Settings.embed_model = HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5"
)
documents = SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index = VectorStoreIndex.from_documents(
documents,
)
This requires Ollama running with the named model available. Check the official docs for the current local setup steps if anything here fails.
Pricing / Limitations
The LlamaIndex framework is free and open source under the MIT License. Your costs are model API usage when you use a hosted provider, developer time, and storage for your index. LlamaParse, the company’s document parsing platform, is a separate product with its own plans; check the official site for current pricing. The main limitations are that LlamaIndex needs a developer, that answer quality depends on document quality and on your choice of LLM and embedding model, and that the company’s focus has shifted toward LlamaParse, so check the docs for the latest direction before building a large system on the framework. Never expose internal documents through an unprotected public endpoint.
Bottom Line for SMB
LlamaIndex is the most direct path from “our knowledge is in folders” to “staff can ask questions and get grounded answers.” Start with one clean folder of documents, build the index, verify answers against the sources, and only then expand. For the broader picture on agents and support tooling, read our guide to AI agents explained for small business and our AI customer support macros tutorial.
Frequently Asked Questions
Do I need to know how to code to use LlamaIndex?
Yes. LlamaIndex is a Python framework for developers. Non-technical teams should use a hosted document Q&A product instead, or hire a developer.
Is LlamaIndex free?
The framework is free and open source under the MIT License. You still pay for hosted model API usage if you use one, and LlamaParse has its own plans.
How is LlamaIndex different from just uploading files to a chatbot?
Uploading files works for one-off chats. LlamaIndex builds a reusable, queryable index over your documents that a developer can embed in an app, persist, reload, and combine with local or hosted models.
