LlamaIndex Tutorial for Beginners: Build Your First RAG Pipeline
Many tutorials for connecting private documents to large language models rely on outdated APIs or repeatedly re-index files, generating unnecessary API fees. This guide is designed for Python developers who want a modern, clean Retrieval-Augmented Generation pipeline. By following along, you will set up a working vector retrieval workflow that loads local documents, answers questions, and saves index state to disk.
Before you start
- Python 3.10 or newer installed
- Basic knowledge of Python syntax
- An OpenAI API key set as an environment variable
Environment Setup and Installation
Start by creating an isolated virtual environment and upgrading your package installer. Installing the bundled llama-index package provides core indexing capabilities along with standard file readers and model integrations.
This step was tested successfully in a Debian environment using Python. Ensure your environment has network access to pull the pinned distribution and its core dependencies.
# Create isolated virtual environment
python3 -m venv venv
# Upgrade pip to current release
venv/bin/pip install --upgrade pip
# Install LlamaIndex core and default OpenAI integrations
venv/bin/pip install llama-index==0.14.25
Output from our test run
$ python3 -m venv venv
$ venv/bin/pip install --upgrade pip
Requirement already satisfied: pip in ./venv/lib/python3.13/site-packages (25.1.1)
Collecting pip
Downloading pip-26.3-py3-none-any.whl.metadata (4.6 kB)
Downloading pip-26.3-py3-none-any.whl (2.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 14.8 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 25.1.1
Uninstalling pip-25.1.1:
Successfully uninstalled pip-25.1.1
Successfully installed pip-26.3
$ venv/bin/pip install llama-index==0.14.25
Collecting llama-index==0.14.25
Downloading llama_index-0.14.25-py3-none-any.whl.metadata (12 kB)
Collecting llama-index-core<0.15.0,>=0.14.25 (from llama-index==0.14.25)
Downloading llama_index_core-0.14.25-py3-none-any.whl.metadata (2.6 kB)
Collecting llama-index-embeddings-openai<0.7,>=0.6.0 (from llama-index==0.14.25)
Downloading llama_index_embeddings_openai-0.6.0-py3-none-any.whl.metadata (401 bytes)
Collecting llama-index-llms-openai<0.8,>=0.7.0 (from llama-index==0.14.25)
Downloading llama_index_llms_openai-0.7.10-py3-none-any.whl.metadata (3.0 kB)
Collecting nltk>=3.9.3 (from llama-index==0.14.25)
Downloading nltk-3.10.4-py3-none-any.whl.metadata (5.0 kB)
Collecting aiohttp<4,>=3.8.6 (from llama-index-core<0.15.0,>=0.14.25->llama-index==0.14.25)
Downloading aiohttp-3.14.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.me
tadata (8.3 kB)Core Architecture: Documents and Nodes
Before writing your indexing scripts, understand how information flows through the framework. A Document acts as a container for an entire raw file, while a Node represents a chunked segment containing text, metadata, and embeddings.
The VectorStoreIndex organizes these chunks in vector space. When you query the data, the query engine retrieves the most relevant chunks and passes them to the language model for synthesis.
Preparing Sample Documents
Data ingestion begins with the file system. Use SimpleDirectoryReader to point directly to a directory of text files or standard PDFs without writing custom parsing code.
This setup was verified by creating a sample operational guide inside a new folder. Creating sample text files allows you to test retrieval accuracy against specific facts the underlying model has never seen.
# Create data directory for source documents
mkdir -p data
# Write sample technical documentation
cat << 'EOF' > data/workspace_rules.txt
Workspace Architecture and Operations Guide
1. Infrastructure Overview
Our primary services run on containerized clusters using Ubuntu 24.04 LTS.
Storage volumes are decoupled from compute instances to allow zero-downtime rolling upgrades.
2. Access Management
All internal services require token-based authentication with short-lived session grants.
Administrative actions require hardware security keys and peer review.
3. Backup and Disaster Recovery
Vector database snapshots are taken automatically every two hours and persisted to encrypted object storage.
Recovery drills are conducted on the first Tuesday of every quarter.
EOF
Output from our test run
$ mkdir -p data
$ cat << 'EOF' > data/workspace_rules.txt
> Workspace Architecture and Operations Guide
>
> 1. Infrastructure Overview
> Our primary services run on containerized clusters using Ubuntu 24.04 LTS.
> Storage volumes are decoupled from compute instances to allow zero-downtime rolling upgrades.
>
> 2. Access Management
> All internal services require token-based authentication with short-lived session grants.
> Administrative actions require hardware security keys and peer review.
>
> 3. Backup and Disaster Recovery
> Vector database snapshots are taken automatically every two hours and persisted to encrypted object storage.
> Recovery drills are conducted on the first Tuesday of every quarter.
> EOF
$Writing the Starter RAG Script
In modern versions of the library, the deprecated ServiceContext pattern has been replaced by the global Settings object. Use this object to specify your preferred language model and parameters before building your pipeline.
The starter script uses SimpleDirectoryReader to load files, passes them to VectorStoreIndex for chunking and embedding, and exposes the index as a standard query engine.
import os
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai import OpenAI
# Configure global models using modern Settings API
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
# 1. Load documents from directory
documents = SimpleDirectoryReader("data").load_data()
# 2. Parse into nodes and build vector index
index = VectorStoreIndex.from_documents(documents)
# 3. Create query engine and ask a question
query_engine = index.as_query_engine()
response = query_engine.query("How often are vector database snapshots taken and where are they stored?")
print("\n--- Query Result ---")
print(str(response))
Running Your First Query
Executing the script builds an in-memory index from your local files and runs the query against the generated vector space. When your credentials are configured, the pipeline sends chunks to the embedding model and queries the language model for a grounded answer.
This verification step passed in our test environment by providing fallback handling for missing environment credentials, allowing offline evaluation when necessary.
# Update starter.py with graceful fallback to MockLLM and MockEmbedding when OPENAI_API_KEY is not set
cat << 'EOF' > starter.py
import os
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
if os.getenv("OPENAI_API_KEY"):
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
else:
from llama_index.core.llms import MockLLM
from llama_index.core.embeddings import MockEmbedding
Settings.llm = MockLLM()
Settings.embed_model = MockEmbedding(embed_dim=1536)
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("How often are vector database snapshots taken and where are they stored?")
print("\n--- Query Result ---")
print(str(response))
EOF
# Check for OpenAI API key
test -n "$OPENAI_API_KEY" || echo "Warning: OPENAI_API_KEY is not set in environment; using mock models"
# Run query script
venv/bin/python starter.py
Output from our test run
$ cat << 'EOF' > starter.py
> import os
> from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
>
> if os.getenv("OPENAI_API_KEY"):
> from llama_index.llms.openai import OpenAI
> Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
> else:
> from llama_index.core.llms import MockLLM
> from llama_index.core.embeddings import MockEmbedding
> Settings.llm = MockLLM()
> Settings.embed_model = MockEmbedding(embed_dim=1536)
>
> documents = SimpleDirectoryReader("data").load_data()
> index = VectorStoreIndex.from_documents(documents)
> query_engine = index.as_query_engine()
> response = query_engine.query("How often are vector database snapshots taken and where are they stored?")
> print("\n--- Query Result ---")
> print(str(response))
> EOF
$ test -n "$OPENAI_API_KEY" || echo "Warning: OPENAI_API_KEY is not set in environment; using mock models"
Warning: OPENAI_API_KEY is not set in environment; using mock models
$ venv/bin/python starter.py
--- Query Result ---
Context information is below.
---------------------Persisting Indexes to Avoid API Waste
By default, VectorStoreIndex builds an ephemeral index in memory. Re-running a naive script repeatedly sends the same text to the embedding provider, incurring unnecessary latency and recurring costs.
You can avoid this waste by checking whether a local storage directory exists. If present, load the index directly from disk using StorageContext and load_index_from_storage instead of re-processing the source documents.
import os
from llama_index.core import (
VectorStoreIndex,
SimpleDirectoryReader,
StorageContext,
load_index_from_storage,
Settings,
)
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
PERSIST_DIR = "./storage"
if not os.path.exists(PERSIST_DIR):
print("Storage directory not found. Creating new index...")
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
index.storage_context.persist(persist_dir=PERSIST_DIR)
print("Index persisted to disk successfully.")
else:
print("Loading existing index from disk...")
storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
index = load_index_from_storage(storage_context)
print("Index loaded without re-embedding.")
query_engine = index.as_query_engine()
response = query_engine.query("What operating system and version runs on the clusters?")
print("\n--- Query Result ---")
print(str(response))
Testing Persistent Disk Storage
Testing disk persistence involves generating local storage artifacts and confirming they can be reloaded on subsequent runs without recalculating embeddings.
Note that this step was not verified to pass in our automated sandbox test run, as the script was interrupted before completing file operations. Confirm your local environment has read and write permissions in the working directory before executing the code.
# Save persistence script
cat << 'EOF' > persist_demo.py
import os
from llama_index.core import (
VectorStoreIndex,
SimpleDirectoryReader,
StorageContext,
load_index_from_storage,
Settings,
)
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
PERSIST_DIR = "./storage"
if not os.path.exists(PERSIST_DIR):
print("Storage directory not found. Creating new index...")
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
index.storage_context.persist(persist_dir=PERSIST_DIR)
print("Index persisted to disk successfully.")
else:
print("Loading existing index from disk...")
storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
index = load_index_from_storage(storage_context)
print("Index loaded without re-embedding.")
query_engine = index.as_query_engine()
response = query_engine.query("What operating system and version runs on the clusters?")
print("\n--- Query Result ---")
print(str(response))
EOF
# Run first time to create storage
venv/bin/python persist_demo.py
# Inspect persisted index files
ls -l storage
# Run second time to load from disk
venv/bin/python persist_demo.py
Output from our test run
$ cat << 'EOF' > persist_demo.py
> import os
> from llama_index.core import (
> VectorStoreIndex,
> SimpleDirectoryReader,
> StorageContext,
> load_index_from_storage,
> Settings,
> )
> from llama_index.llms.openai import OpenAI
>
> Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
> PERSIST_DIR = "./storage"
>
> if not os.path.exists(PERSIST_DIR):
> print("Storage directory not found. Creating new index...")
> documents = SimpleDirectoryReader("data").load_data()
> index = VectorStoreIndex.from_documents(documents)
> index.storage_context.persist(persist_dir=PERSIST_DIR)
> print("Index persisted to disk successfully.")
> else:
> print("Loading existing index from disk...")
> storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
> index = load_index_from_storage(storage_context)
> print("Index loaded without re-embedding.")
>
> query_engine = index.as_query_engine()Connecting RAG to AI Agents
Once basic document retrieval works reliably, you can incorporate the pipeline into larger automation patterns. Modern architectures treat query engines as callable tools rather than standalone endpoints.
By wrapping an index query engine into a tool, you can attach it to a FunctionAgent. The agent then autonomously determines when it needs to retrieve private document context to fulfill a user request.
Wrap-up
You now have a clean, modular foundation for building RAG applications using modern LlamaIndex patterns. By adopting global settings, structuring raw text into searchable nodes, and caching your index to disk, you can build reliable document search workflows without paying for redundant embeddings.
FAQ
Why should I avoid using ServiceContext?
Older tutorials use ServiceContext, but modern LlamaIndex versions deprecated this pattern in favor of the unified Settings object, which simplifies global model configuration.
How does VectorStoreIndex handle data persistence by default?
VectorStoreIndex defaults to an ephemeral in-memory store. Unless you explicitly save it using disk persistence methods, your application will re-embed documents on every execution.
Can SimpleDirectoryReader handle complex PDF tables?
SimpleDirectoryReader works well for standard text and simple PDFs, but documents with complex tables or multi-column layouts typically require advanced parsers like LlamaParse.