Debugging LocalRAG's Embedding Pipeline: When ChromaDB Returned Empty Results

The symptom was simple: queries against my LocalRAG system returned empty results, even though the vector store showed documents were being ingested. I’d built the pipeline to pull documents from a knowledge base, chunk them, embed with nomic-embed-text, store in ChromaDB, and retrieve with a Mistral 7B running through Ollama. The retrieval step would execute without error, but the top 10 results would all have zero similarity scores.

I started by checking the ingestion script. The document loading worked fine — I could see 1,247 nodes being created from my markdown files. The issue had to be in how I was storing or retrieving the embeddings. I added debug logging to the VectorStoreIndex.from_documents() call and watched the embedding vectors being generated. They looked correct: 768-dimensional float32 arrays with reasonable magnitude.

The problem surfaced when I inspected the ChromaDB collection directly. I opened a Python shell and ran:

import chromadb
client = chromadb.PersistentClient(path="./chromadb")
collection = client.get_collection("local_rag")
results = collection.get(limit=5, include=["documents", "embeddings", "metadatas"])
print(results["embeddings"][0][:10])  # First 10 dimensions
print(results["metadatas"][0])

The embeddings were there, but the metadata was empty. Every document had an empty dict {} for metadata. I’d forgotten that LlamaIndex’s SimpleDirectoryReader was passing metadata through a chain that my custom metadata handler wasn’t preserving.

I traced through the code and found the culprit in my CustomMetadataTransformer class. I’d overridden the transform_node method and was only copying the text field, discarding everything else:

def transform_node(self, node: Node) -> Node:
    # Bug: only copying text, losing metadata
    return TextNode(text=node.text)

The fix required preserving the metadata dictionary:

def transform_node(self, node: Node) -> Node:
    new_node = TextNode(text=node.text)
    new_node.metadata = node.metadata.copy()  # Fix: copy metadata
    return new_node

But that wasn’t the end of it. Even after fixing the metadata copy, queries still returned empty results. I dug deeper into the similarity computation and realized ChromaDB was treating the embeddings as normalized vectors when they shouldn’t be. My nomic-embed-text embeddings weren’t L2-normalized, but I’d set hnsw:space to cosines in the collection configuration.

I recreated the collection with the correct space:

collection = client.create_collection(
    "local_rag",
    metadata={"hnsw:space": "l2"}
)

After re-ingesting the documents with the corrected metadata transformer and proper distance metric, queries started returning relevant results. The first successful query matched a chunk about Python type hints against my question about typing.Optional, with a similarity score of 0.73.

The lesson: metadata in vector databases isn’t just for filtering — it’s part of the retrieval pipeline’s integrity. When I built the initial version of LocalRAG, I focused on the embedding and retrieval logic and skipped validating that metadata was flowing through unchanged. The system appeared to work because ChromaDB accepted the documents; it only failed at query time when the filtering logic expected metadata fields that didn’t exist.

I’m still deciding whether to make metadata validation a required step in the ingestion pipeline or just add better logging when metadata is empty. The next experiment is trying nomic-embed-text-v1.5 with the newer embedding format to see if it improves retrieval quality on my technical documentation corpus.