TeselaDB
DocumentationDownload

Semantic search with $vectorSearch

Store embeddings next to your documents and search by meaning with a vector index.

With $vectorSearch you search documents by what they mean, not by the exact words. Vectors are stored as another field of the document and the index lives on the same nodes as the data: you do not need another database.

This guide uses Python, PyMongo and the all-MiniLM-L6-v2 model from sentence-transformers, which gives 384-dimension vectors. Any embedding model works; set numDimensions to its size.

Shell
pip install pymongo sentence-transformers

1. Store the documents with their vector

Python
from pymongo import MongoClient
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")  # 384 dimensions
client = MongoClient("mongodb://app:<password>@node-01:27017/?tls=true&tlsCAFile=./ca.pem")
articles = client.kb.articles

docs = [
    {"title": "Add a node", "body": "Create an enrollment token and start the binary with --join."},
    {"title": "Drain a node", "body": "teseladb admin node drain moves its replicas before you shut it down."},
]
for doc in docs:
    doc["embedding"] = model.encode(doc["title"] + ". " + doc["body"]).tolist()
articles.insert_many(docs)

2. Create the vector index

Python
from pymongo.operations import SearchIndexModel

articles.create_search_index(SearchIndexModel(
    name="vector",
    type="vectorSearch",
    definition={"fields": [
        {"type": "vector", "path": "embedding", "numDimensions": 384, "similarity": "cosine"}
    ]},
))

3. Search by meaning

Python
query = model.encode("how do I take a machine out of the cluster?").tolist()
results = articles.aggregate([
    {"$vectorSearch": {"index": "vector", "path": "embedding", "queryVector": query,
                       "numCandidates": 100, "limit": 3}},
    {"$project": {"title": 1, "score": {"$meta": "vectorSearchScore"}}},
])
for r in results:
    print(round(r["score"], 3), r["title"])

The question does not contain the word «drain», but the closest result is the article that explains it.

4. Filter and tune

  • Add a filter field to the index ({"type": "filter", "path": "category"}) and pass filter in $vectorSearch to search only one category.
  • Raise numCandidates to improve accuracy at the cost of time, or use exact: true to compare with every vector. With exact, the result is the same on any replica.
  • An inserted document shows up in the search as soon as the write is committed.