Pinecone DB: Vector Search and Setup Tutorial

Updated 3 min read

Vector search enables semantic discovery and Retrieval-Augmented Generation workflows, but older tutorials often rely on deprecated packages and removed configuration options. This guide walks through setting up a modern Pinecone DB index using the current Python SDK. By following along, you will deploy a serverless vector database, store sample vectors with custom metadata, and execute filtered similarity searches.

Watch the video on YouTube

Before you start

  • Python 3.10 or newer installed on Linux or macOS
  • Basic familiarity with Python and bash commands
  • A free Pinecone account with an API key from app.pinecone.io

Install the Pinecone SDK

A common pitfall when starting with Pinecone is installing the legacy pinecone-client library. The current official release uses the pinecone package name directly. Setting up an isolated Python virtual environment ensures your dependencies do not conflict with system packages.

Run the commands below to create a virtual environment, activate it, and install the library. A quick verification command checks that the package loads properly. Note that this installation sequence timed out during automated testing in our sandbox environment, so ensure your network connection allows package downloads.

TerminalNot verified in our test run
python3 -m venv .venv && source .venv/bin/activate
pip install pinecone
python3 -c "import pinecone; print('Pinecone SDK version:', pinecone.__version__)"
Output from our test run
$ python3 -m venv .venv && source .venv/bin/activate
(.venv) $ pip install pinecone
Collecting pinecone
  Downloading pinecone-10.3.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.8 kB)
Collecting httpx<1.0,>=0.27 (from httpx[http2]<1.0,>=0.27->pinecone)
  Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Collecting msgspec<0.23,>=0.19 (from pinecone)
  Downloading msgspec-0.22.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.me
tadata (6.0 kB)
Collecting orjson<4,>=3.11 (from pinecone)
  Downloading orjson-3.13.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (42 kB)
Collecting anyio>=4.0 (from pinecone)
  Downloading anyio-4.16.0-py3-none-any.whl.metadata (4.7 kB)
Collecting certifi (from httpx<1.0,>=0.27->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading certifi-2026.10.23-py3-none-any.whl.metadata (2.5 kB)
Collecting httpcore==1.* (from httpx<1.0,>=0.27->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading httpcore-1.0.9-py3-none-any.whl.metadata (21 kB)
Collecting idna (from httpx<1.0,>=0.27->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading idna-3.20-py3-none-any.whl.metadata (7.2 kB)
Collecting h11>=0.16 (from httpcore==1.*->httpx<1.0,>=0.27->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
Collecting h2<5,>=3 (from httpx[http2]<1.0,>=0.27->pinecone)
  Downloading h2-4.4.1-py3-none-any.whl.metadata (3.8 kB)
Collecting hyperframe<7,>=6.1 (from h2<5,>=3->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading hyperframe-6.1.0-py3-none-any.whl.metadata (4.3 kB)
Collecting hpack<5,>=4.2 (from h2<5,>=3->httpx[http2]<1.0,>=0.27->pinecone)
  Downloading hpack-4.2.0-py3-none-any.whl.metadata (3.3 kB)

Core Concepts and Index Design

Before writing index configuration code, consider the constraints of serverless index architecture. Serverless is now the standard default, scaling automatically without needing dedicated pods.

When provisioning an index, two settings are permanent: dimension and metric. The dimension must match your embedding model output, such as 1536 for common text embedding models. Supported distance metrics include cosine, euclidean, and dotproduct. You can also segment data using namespaces and attach metadata payloads capped at 40 kilobytes per vector.

Initialize the Client and Index

Initializing the client requires importing Pinecone and ServerlessSpec. Calling Pinecone() without positional arguments automatically pulls credentials from the PINECONE_API_KEY environment variable.

You do not need to supply an environment or project string. The script below demonstrates checking whether an index already exists using list_indexes before calling create_index. Once created, connecting to the index returns a client object ready for vector operations.

create_index.py
import os
from pinecone import Pinecone, ServerlessSpec

# Initialize client from PINECONE_API_KEY environment variable
pc = Pinecone()

index_name = "quickstart-index"

# Check if the index already exists before creating
existing_indexes = [idx["name"] for idx in pc.list_indexes()]
if index_name not in existing_indexes:
    pc.create_index(
        name=index_name,
        dimension=1536,
        metric="cosine",
        spec=ServerlessSpec(cloud="aws", region="us-east-1"),
    )
    print(f"Created serverless index: {index_name}")
else:
    print(f"Index {index_name} already exists.")

# Connect to the index
index = pc.Index(index_name)
print("Index status:", index.describe_index_stats())

Upsert Vectors and Metadata

Vector records are inserted using the upsert method. Each record contains an id, a list of floating-point values representing the embedding, and an optional metadata dictionary for titles, tags, or reference links.

Serverless indexes operate on a near-real-time model. While upserts are immediately acknowledged, records may take a brief moment to settle across all search replicas. If your pipeline executes queries right after upserting, introduce a brief delay to avoid missing newly added records.

upsert_data.py
from pinecone import Pinecone

pc = Pinecone()
index = pc.Index("quickstart-index")

# Vectors with IDs, 1536-dimensional values, and metadata
sample_vectors = [
    {
        "id": "vec-001",
        "values": [0.05] * 1536,
        "metadata": {"title": "Introduction to Vectors", "topic": "search", "year": 2024},
    },
    {
        "id": "vec-002",
        "values": [0.08] * 1536,
        "metadata": {"title": "Scaling Python Pipelines", "topic": "engineering", "year": 2023},
    },
]

# Upsert vectors into the default or a specific namespace
response = index.upsert(vectors=sample_vectors, namespace="knowledge-base")
print(f"Upserted {response['upserted_count']} vectors successfully.")

Query with Metadata Filters

Similarity searches use the query method by passing a target vector and specifying top_k. By default, the database returns only matching record identifiers and similarity scores.

Setting include_metadata to True includes the stored key-value attributes in the response. You can also supply structured metadata filters using operators like $eq or $in to restrict matches to specific categories or tags.

query_index.py
from pinecone import Pinecone

pc = Pinecone()
index = pc.Index("quickstart-index")

query_vector = [0.05] * 1536

# Query nearest neighbors with metadata filtering
results = index.query(
    vector=query_vector,
    top_k=2,
    include_metadata=True,
    namespace="knowledge-base",
    filter={"topic": {"$eq": "search"}},
)

for match in results.get("matches", []):
    score = match["score"]
    title = match["metadata"]["title"]
    print(f"ID: {match['id']} | Score: {score:.4f} | Title: {title}")

Run the Vector Workflow

You can combine index creation, data loading, and querying into a single standalone script. The script checks for the required environment variable, creates a lightweight four-dimensional index if absent, upserts two records into a namespace, and runs a filtered search.

Run the commands below to save and execute the end-to-end workflow. Note that running the execution step in our automated sandbox environment did not pass because the test container lacked a valid live API key.

TerminalNot verified in our test run
# Write the complete end-to-end demo script
cat << 'EOF' > run_pinecone.py
import os
import sys
import time
from pinecone import Pinecone, ServerlessSpec

api_key = os.environ.get("PINECONE_API_KEY")
if not api_key:
    print("Note: PINECONE_API_KEY is not set. Export your key to run against live cloud.")
    print("Example: export PINECONE_API_KEY='pcsk_...'")
    sys.exit(0)

pc = Pinecone(api_key=api_key)
index_name = "demo-search-index"

existing = [idx["name"] for idx in pc.list_indexes()]
if index_name not in existing:
    pc.create_index(
        name=index_name,
        dimension=4,
        metric="cosine",
        spec=ServerlessSpec(cloud="aws", region="us-east-1"),
    )
    print(f"Created index: {index_name}")

index = pc.Index(index_name)
index.upsert(
    vectors=[
        {"id": "item-1", "values": [0.1, 0.2, 0.3, 0.4], "metadata": {"tag": "ai"}},
        {"id": "item-2", "values": [0.9, 0.8, 0.7, 0.6], "metadata": {"tag": "web"}},
    ],
    namespace="tutorial",
)

time.sleep(2)  # Wait briefly for serverless eventual consistency

res = index.query(
    vector=[0.1, 0.2, 0.3, 0.4],
    top_k=1,
    include_metadata=True,
    namespace="tutorial",
)

for match in res["matches"]:
    print(f"Match ID: {match['id']} | Score: {match['score']:.4f} | Tag: {match['metadata']['tag']}")
EOF
# Execute the workflow script
python3 run_pinecone.py
Output from our test run
$ cat << 'EOF' > run_pinecone.py
> import os
> import sys
> import time
> from pinecone import Pinecone, ServerlessSpec
>
> api_key = os.environ.get("PINECONE_API_KEY")
> if not api_key:
>     print("Note: PINECONE_API_KEY is not set. Export your key to run against live cloud.")
>     print("Example: export PINECONE_API_KEY='pcsk_...'")
>     sys.exit(0)
>
> pc = Pinecone(api_key=api_key)
> index_name = "demo-search-index"
>
> existing = [idx["name"] for idx in pc.list_indexes()]
> if index_name not in existing:
>     pc.create_index(
>         name=index_name,
>         dimension=4,
>         metric="cosine",
>         spec=ServerlessSpec(cloud="aws", region="us-east-1"),
>     )
>     print(f"Created index: {index_name}")
>
> index = pc.Index(index_name)
> index.upsert(

Modern Document APIs and Inference

Recent releases introduce hosted inference and document-level indexing alongside standard vector APIs. Hosted inference allows generating dense embeddings directly through Pinecone, eliminating the need for external embedding infrastructure.

Document indexes let you configure schemas that handle chunking and text indexing automatically. These additions streamline Retrieval-Augmented Generation architectures by handling data prep and vectorization within the managed service.

Wrap-up

You now have a clean foundation for using Pinecone DB with the modern Python SDK. With serverless indexing, namespace partitioning, and metadata filtering in place, your pipeline is prepared for semantic search or context retrieval in AI applications.

FAQ

What package should I install for the Pinecone Python SDK?

Install pinecone using pip. The older pinecone-client package is deprecated.

Why does a query return empty results immediately after an upsert?

Serverless indexes provide near-real-time consistency, meaning new vectors can take a second or two to become queryable across all replicas.

Can I change the dimension or metric of an index later?

No, vector dimension and similarity metrics are permanent upon index creation. If your embedding model changes, create a new index.

What is the maximum size for vector metadata?

Pinecone caps metadata payloads at 40 kilobytes per vector record.

Sources

Subscribe for more tested tutorials