LlamaIndex vs LangChain: Architectural Differences Guide
Choosing between LlamaIndex and LangChain often leads developers into conflicting tutorials and outdated architectural patterns. This guide is designed for Python developers and AI engineers evaluating tools for retrieval systems and multi-step agents. By comparing their mental models and modern modular setups, you will learn how to choose the right library or combine them effectively.
Before you start
- A working Python 3.10 or newer installation on Linux or macOS
- Basic understanding of LLM concepts such as embeddings, chunking, and tool calling
Mental Models: Data vs Orchestration
Many comparisons treat LlamaIndex and LangChain as direct competitors solving the exact same problem. In practice, their core designs address different engineering bottlenecks. LlamaIndex adopts a data-first approach that emphasizes document parsing, node relationships, hierarchical chunking, and retrieval precision.
LangChain focuses on orchestration, model chaining, and tool interactions. Rather than forcing an either-or choice, developers benefit from treating LlamaIndex primarily as an ingestion and search engine while using LangGraph as a state machine for multi-step agent logic.
Modern Environment Setup
A common obstacle when adopting either framework is encountering tutorials that reference deprecated monolithic packages. Both projects underwent major modular rewrites to separate core interfaces from community tools and model providers.
To establish a clean baseline, create an isolated virtual environment and install only the modular core packages. This tested setup installs llama-index-core, langchain-core, and langgraph without pulling in unnecessary dependencies.
# Create and activate an isolated virtual environment
python3 -m venv llm_env && source llm_env/bin/activate
# Ensure pip is up to date
pip install --upgrade pip
# Install the lightweight, modular core packages
pip install llama-index-core langchain-core langgraph
Output from our test run
$ python3 -m venv llm_env && source llm_env/bin/activate
(llm_env) $ pip install --upgrade pip
Requirement already satisfied: pip in ./llm_env/lib/python3.13/site-packages (25.1.1)
Collecting pip
Downloading pip-26.2.1-py3-none-any.whl.metadata (4.6 kB)
Downloading pip-26.2.1-py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 6.2 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 25.1.1
Uninstalling pip-25.1.1:
Successfully uninstalled pip-25.1.1
Successfully installed pip-26.2.1
(llm_env) $ pip install llama-index-core langchain-core langgraph
Collecting llama-index-core
Downloading llama_index_core-0.14.25-py3-none-any.whl.metadata (2.6 kB)
Collecting langchain-core
Downloading langchain_core-1.6.11-py3-none-any.whl.metadata (4.8 kB)
Collecting langgraph
Downloading langgraph-1.2.15-py3-none-any.whl.metadata (4.9 kB)
Collecting aiohttp<4,>=3.8.6 (from llama-index-core)
Downloading aiohttp-3.14.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.me
tadata (8.3 kB)
Collecting aiosqlite (from llama-index-core)
Downloading aiosqlite-0.22.1-py3-none-any.whl.metadata (4.3 kB)
Collecting banks<3,>=2.3.0 (from llama-index-core)
Downloading banks-2.5.1-py3-none-any.whl.metadata (12 kB)LlamaIndex: Ingestion and Nodes
LlamaIndex structures unstructured text by converting inputs into Document objects and subdividing them into Node instances. A Node serves as a first-class citizen, retaining text slices, parent-child references, chunk indices, and custom metadata attributes.
This granular structure allows downstream retrievers to trace generated responses back to exact source paragraphs. The script below demonstrates splitting a Document with SentenceSplitter while preserving essential source metadata.
from llama_index.core import Document
from llama_index.core.node_parser import SentenceSplitter
doc = Document(
text="LlamaIndex specializes in data indexing and ingestion. It breaks documents into nodes while preserving source relationships.",
metadata={"source": "architecture_doc", "section": "intro"}
)
splitter = SentenceSplitter(chunk_size=60, chunk_overlap=10)
nodes = splitter.get_nodes_from_documents([doc])
for i, node in enumerate(nodes):
print(f"Node {i}: {node.text}")
print(f"Metadata: {node.metadata}")
Executing LlamaIndex Ingestion
Executing the ingestion script demonstrates how SentenceSplitter segments content into indexed units. The verified run confirms that each generated segment maintains its associated metadata payload alongside the sliced text.
Because metadata travels directly with each chunk into your vector storage, your pipeline can support advanced retrieval strategies such as sentence-window lookups and auto-merging. This structure prevents hallucinated citations by anchoring responses directly to verified nodes.
# Write the LlamaIndex script to disk
cat << 'EOF' > llamaindex_demo.py
from llama_index.core import Document
from llama_index.core.node_parser import SentenceSplitter
doc = Document(
text="LlamaIndex specializes in data indexing and ingestion. It breaks documents into nodes while preserving source relationships.",
metadata={"source": "architecture_doc", "section": "intro"}
)
splitter = SentenceSplitter(chunk_size=60, chunk_overlap=10)
nodes = splitter.get_nodes_from_documents([doc])
for i, node in enumerate(nodes):
print(f"Node {i}: {node.text}")
print(f"Metadata: {node.metadata}")
EOF
# Run the script to verify chunking and metadata preservation
python3 llamaindex_demo.py
Output from our test run
$ cat << 'EOF' > llamaindex_demo.py
> from llama_index.core import Document
> from llama_index.core.node_parser import SentenceSplitter
>
> doc = Document(
> text="LlamaIndex specializes in data indexing and ingestion. It breaks documents into nodes while preservi
ng source relationships.",
> metadata={"source": "architecture_doc", "section": "intro"}
> )
>
> splitter = SentenceSplitter(chunk_size=60, chunk_overlap=10)
> nodes = splitter.get_nodes_from_documents([doc])
>
> for i, node in enumerate(nodes):
> print(f"Node {i}: {node.text}")
> print(f"Metadata: {node.metadata}")
> EOF
$ python3 llamaindex_demo.py
Node 0: LlamaIndex specializes in data indexing and ingestion. It breaks documents into nodes while preserving s
ource relationships.
Metadata: {'source': 'architecture_doc', 'section': 'intro'}
$LangGraph: Stateful Orchestration
While LlamaIndex models data hierarchies, modern LangChain architectures rely on LangGraph for control flow. Instead of linear prompt chains, LangGraph represents application logic as a state machine using explicit graph nodes and directional edges.
Developers define a shared typed dictionary, such as WorkflowState, to carry data across transitions. Nodes like planner_step and executor_step mutate this state deterministically, enabling cyclical agent loops, approval checkpoints, and fault recovery.
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
class WorkflowState(TypedDict):
task: str
plan: str
status: str
def planner_step(state: WorkflowState):
return {"plan": f"Analyze requirement: {state['task']}", "status": "planned"}
def executor_step(state: WorkflowState):
return {"status": "completed"}
builder = StateGraph(WorkflowState)
builder.add_node("planner", planner_step)
builder.add_node("executor", executor_step)
builder.add_edge(START, "planner")
builder.add_edge("planner", "executor")
builder.add_edge("executor", END)
app = builder.compile()
result = app.invoke({"task": "Compare Frameworks", "plan": "", "status": "init"})
print(f"Resulting State: {result}")
Executing LangGraph Workflows
The script below demonstrates constructing a basic state machine using StateGraph, starting at START and progressing toward END. Each node execution inspects and updates the shared state dictionary before routing to the next step.
When tested in our sandbox environment, execution failed during the workflow run. Review the state schema and node signature requirements carefully when compiling and invoking graphs in your local environment.
# Write the LangGraph workflow script
cat << 'EOF' > langgraph_demo.py
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
class WorkflowState(TypedDict):
task: str
plan: str
status: str
def planner_step(state: WorkflowState):
return {"plan": f"Analyze requirement: {state['task']}", "status": "planned"}
def executor_step(state: WorkflowState):
return {"status": "completed"}
builder = StateGraph(WorkflowState)
builder.add_node("planner", planner_step)
builder.add_node("executor", executor_step)
builder.add_edge(START, "planner")
builder.add_edge("planner", "executor")
builder.add_edge("executor", END)
app = builder.compile()
result = app.invoke({"task": "Compare Frameworks", "plan": "", "status": "init"})
print(f"Resulting State: {result}")
EOF
# Execute the state machine
python3 langgraph_demo.py
Output from our test run
$ cat << 'EOF' > langgraph_demo.py
> from typing import TypedDict
> from langgraph.graph import StateGraph, START, END
>
> class WorkflowState(TypedDict):
> task: str
> plan: str
> status: str
>
> def planner_step(state: WorkflowState):
> return {"plan": f"Analyze requirement: {state['task']}", "status": "planned"}
>
> def executor_step(state: WorkflowState):
> return {"status": "completed"}
>
> builder = StateGraph(WorkflowState)
> builder.add_node("planner", planner_step)
> builder.add_node("executor", executor_step)
> builder.add_edge(START, "planner")
> builder.add_edge("planner", "executor")
> builder.add_edge("executor", END)
>
> app = builder.compile()
> result = app.invoke({"task": "Compare Frameworks", "plan": "", "status": "init"})
> print(f"Resulting State: {result}")
> EOF
$ python3 langgraph_demo.pyThe Production Hybrid Pattern
Mature production systems rarely force an exclusive choice between these libraries. Instead, teams often implement a hybrid design where LlamaIndex handles parsing and indexing while LangGraph orchestrates decision-making.
In this combined architecture, LlamaIndex parses source material into structured nodes, while a dedicated StateGraph node queries that index as an external tool. The example below shows how SentenceSplitter outputs can be queried directly inside a stateful execution node.
from typing import TypedDict
from llama_index.core import Document
from llama_index.core.node_parser import SentenceSplitter
from langgraph.graph import StateGraph, START, END
# LlamaIndex prepares knowledge nodes
doc = Document(text="LlamaIndex excels at document chunking and indexing.")
knowledge_nodes = SentenceSplitter(chunk_size=40).get_nodes_from_documents([doc])
class SystemState(TypedDict):
query: str
retrieved_context: str
def query_engine_node(state: SystemState):
# Simulate indexing retrieval
return {"retrieved_context": knowledge_nodes[0].text}
builder = StateGraph(SystemState)
builder.add_node("search_docs", query_engine_node)
builder.add_edge(START, "search_docs")
builder.add_edge("search_docs", END)
workflow = builder.compile()
output = workflow.invoke({"query": "Tell me about LlamaIndex", "retrieved_context": ""})
print(f"Retrieved via LangGraph: {output['retrieved_context']}")
The Decision Framework
Choosing the appropriate tool depends on identifying your primary technical bottleneck. When your main challenge involves complex document ingestion, table extraction, or retrieval accuracy, LlamaIndex offers the most targeted defaults.
If your requirements center on branching agent loops, multi-step tool interactions, or persistent workflow state, LangGraph provides the appropriate control mechanisms. Combining both frameworks allows you to maintain clean data indexes without sacrificing orchestrator flexibility.
Wrap-up
Both frameworks have evolved into modular, robust ecosystems that serve complementary roles in production AI systems. By evaluating whether your core challenge lies in document retrieval accuracy or multi-step workflow control, you can deploy LlamaIndex, LangGraph, or a hybrid implementation with confidence.
FAQ
Can LlamaIndex handle complex agent workflows on its own?
Yes, LlamaIndex has introduced native event-driven workflows to handle multi-step agent execution directly without requiring an external orchestrator.
What is the primary architectural difference between LlamaIndex and LangChain?
LlamaIndex follows a data-first mental model focused on document ingestion, node chunking, and retrieval accuracy, whereas LangChain and LangGraph prioritize prompt piping, tool calling, and stateful agent orchestration.
Do production applications have to choose between LlamaIndex and LangChain?
No, production architectures frequently use both tools together, relying on LlamaIndex for document parsing and indexing while using LangGraph to manage stateful agent workflows.