Embeddings and vector stores

Embedding model choice, FAISS, Chroma and pgvector, indexing and updating, and filtering on metadata before similarity search.

Choosing an embedding model

from langchain_openai import OpenAIEmbeddings
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.embeddings import OllamaEmbeddings

# hosted: strong, cheap, and a network dependency
openai_emb = OpenAIEmbeddings(model="text-embedding-3-small", dimensions=512)

# local: no per-call cost, no data leaving the machine
local_emb = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2",
    encode_kwargs={"normalize_embeddings": True, "batch_size": 64},
)

ollama_emb = OllamaEmbeddings(model="nomic-embed-text", base_url="http://localhost:11434")

vectors = local_emb.embed_documents(["first document", "second document"])
query_vector = local_emb.embed_query("what is this about?")
print(len(vectors), len(vectors[0]), len(query_vector))

# batch size matters more than model choice for ingestion throughput
texts = [f"chunk {i}" for i in range(1000)]
batches = [texts[i:i + 128] for i in range(0, len(texts), 128)]
print(len(batches), "batches of up to 128")

• Query and document embeddings must come from the same model. Mixing two models produces vectors in unrelated spaces and similarity becomes noise.
• The dimensions parameter on hosted models reduces storage and cost with a small quality loss. Measure on your own retrieval set before enabling it.
• Normalise embeddings when using cosine or inner-product indexes. Most managed stores normalise for you; local FAISS does not unless you ask.
• Embedding model changes require a full re-index. Version the embedding model name alongside the index and treat a change as a migration.

Stores: FAISS, Chroma and pgvector

from langchain_community.vectorstores import FAISS, Chroma
from langchain_postgres import PGVector

# FAISS: in-process, fastest to start, no server
faiss_store = FAISS.from_documents(chunks, local_emb, distance_strategy="MAX_INNER_PRODUCT")
faiss_store.save_local("faiss_index")
loaded = FAISS.load_local("faiss_index", local_emb, allow_dangerous_deserialization=True)

# Chroma: persistence and metadata filtering with no infrastructure
chroma_store = Chroma(
    collection_name="handbook",
    embedding_function=local_emb,
    persist_directory="./chroma",
)
chroma_store.add_documents(chunks)

# pgvector: the vectors live beside your relational data
pg_store = PGVector(
    embeddings=local_emb,
    collection_name="handbook",
    connection="postgresql+psycopg://user:pass@localhost:5432/app",
    use_jsonb=True,
)
pg_store.add_documents(chunks)

# metadata filtering happens BEFORE similarity, so it changes the candidate set
results = chroma_store.similarity_search_with_score(
    "how do I request a refund?",
    k=6,
    filter={"section": "billing", "version": "2026"},
)
for doc, score in results:
    print(round(float(score), 4), doc.metadata.get("h2"), doc.page_content[:70])

# an upsert pattern: stable ids make re-ingestion idempotent
chroma_store.add_documents(chunks, ids=[stable_id(c) for c in chunks])
chroma_store.delete(ids=stale_ids)

Updating an index

from langchain_core.documents import Document
import datetime

def reindex(store, source_path, new_chunks):
    """Replace every chunk from one source without touching the rest."""
    existing = store.get(where={"source": source_path})
    if existing["ids"]:
        store.delete(ids=existing["ids"])

    stamped = [
        Document(
            page_content=c.page_content,
            metadata={**c.metadata,
                      "source": source_path,
                      "indexed_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
                      "embedding_model": "all-MiniLM-L6-v2"},
        )
        for c in new_chunks
    ]
    store.add_documents(stamped, ids=[stable_id(c) for c in stamped])
    return len(stamped)

# never re-embed documents whose content has not changed
def needs_update(store, source_path, content_hash):
    existing = store.get(where={"source": source_path}, include=["metadatas"])
    hashes = {m.get("content_hash") for m in existing.get("metadatas", [])}
    return content_hash not in hashes

• Delete by source before inserting a revised document. Inserting without deleting leaves the old version retrievable forever, and stale answers are worse than no answers.
• Store the embedding model name and the ingestion timestamp in metadata. When a score drops, the first question is which vector space answered the query.
• Re-embedding a large corpus is the expensive operation: hash the content and skip unchanged documents. In practice, most re-indexes touch a small fraction.
• Test the index with real queries after every ingestion. A count of documents proves the write succeeded, not that retrieval works.

FAQ

Related

Document loaders and text splitters Retrievers in depth

Last refreshed 2026-09-18.