Skip to main content

Bring Your Own Vectors (BYOV) — External Embeddings

By default, RushDB generates embeddings for you using the configured server-side embedding model. With BYOV you generate the vectors yourself — using any model, any provider, even a locally fine-tuned one — and push them alongside your records. RushDB stores and indexes them, and you search with a pre-computed queryVector instead of a raw query string.

Use BYOV when:

  • You have a fine-tuned or domain-specific embedding model
  • Your compliance requirements prohibit sending raw text to a third-party embedding API
  • You want to co-locate embedding cost with your own infrastructure

How it differs from managed embeddings​

ManagedBYOV (external)
Who generates vectorsRushDB (server-side)You (client-side)
Search parameterquery: "text"queryVector: number[]
Index sourceTypemanaged (default)external
Shorthand—external: true
DimensionsSet by server RUSHDB_EMBEDDING_DIMENSIONSSet per index by you

Both index types can coexist on the same label and property.

Step 1: Create an external embedding index​

from rushdb import RushDB
import os

db = RushDB(os.environ['RUSHDB_API_KEY'], base_url='https://api.rushdb.com/api/v1')

db.ai.indexes.create({
    'label': 'ARTICLE',
    'propertyName': 'body',
    'external': True,           # shorthand for sourceType: 'external'
    'similarityFunction': 'cosine',
    'dimensions': 1536
})

An external index starts in awaiting_vectors status — it has no backfill work to do because RushDB never calls your embedding provider. It becomes ready as soon as vectors are stored.

Step 2: Push records with inline vectors​

Use the vectors parameter on create (or createMany for batches) to deliver embeddings alongside the data in one call.

import openai
import os

openai_client = openai.OpenAI(api_key=os.environ['OPENAI_API_KEY'])

def embed(text: str) -> list[float]:
    resp = openai_client.embeddings.create(model='text-embedding-3-small', input=text)
    return resp.data[0].embedding

articles = [
    {'title': 'Intro to Graph Databases', 'body': 'Graph databases store data as nodes and edges...'},
    {'title': 'Vector Search Explained',  'body': 'Vector search finds semantically similar documents...'}
]

# Option A: batch import with create_many
vectors = [[{'propertyName': 'body', 'vector': embed(a['body'])}] for a in articles]
db.records.create_many(label='ARTICLE', data=articles, vectors=vectors)

# Option B: one record at a time
for article in articles:
    db.records.create(
        label='ARTICLE',
        data=article,
        vectors=[{'propertyName': 'body', 'vector': embed(article['body'])}],
    )

Step 3: Search with a pre-computed query vector​

Instead of query: "text" you pass queryVector: number[]. RushDB infers dimensions from the vector length.

query_vector = embed('how do graph databases handle relationships?')

results = db.ai.search({
    'propertyName': 'body',
    'queryVector': query_vector,   # dimensions inferred from vector length
    'labels': ['ARTICLE'],
    'limit': 5
})

for article in results.data:
    print(f"{article.get('title')} — score: {article.score:.3f}")

Combining queryVector with structured filters​

queryVector works with where exactly like query — the where clause executes first, then semantic ranking runs over the filtered set.

results = db.ai.search({
    'propertyName': 'body',
    'queryVector': embed('reducing database latency'),
    'labels': ['ARTICLE'],
    'where': {
        'status': 'published',
        'category': 'infrastructure',
        'publishedAt': {'$gte': '2025-01-01'}
    },
    'limit': 10
})

Disambiguation: two indexes on the same property​

If you create both a cosine and euclidean index on the same label+property, specify similarityFunction in the vectors entry to target the right one.

vectors=[
    {"propertyName": "body", "similarityFunction": "cosine",    "vector": cosine_embedding},
    {"propertyName": "body", "similarityFunction": "euclidean", "vector": euclidean_embedding}
]
Vector array entriesMatching indexesOutcome
1 entry, no similarityFunction1 matchOK — writes to that index
1 entry, no similarityFunction2 matches422 Ambiguous — specify similarityFunction
1 entry, no similarityFunction0 matches404 No external index found
1 entry, with similarityFunction—Matched by both dimension + function

Next steps​

  • Hybrid Retrieval — where filters + semantic ranking in one call
  • GraphRAG — use BYOV embeddings in a retrieval-augmented generation pipeline with graph traversal
  • RAG Evaluation — benchmark precision before and after swapping embedding models