Search with vector embeddings (original) (raw)

This page shows you how to use Firestore to perform K-nearest neighbor (KNN) vector searches using the following techniques:

Before you begin

Before you store embeddings in Firestore, you must generate vector embeddings. Firestore does not generate the embeddings. You can use a service such as Vertex AI to create vector values, for example, text embeddings from your Firestore data. You can then store these embeddings back in Firestore documents.

To learn more about embeddings, see What are embeddings?

To learn how to get text embeddings with Vertex AI, see Get text embeddings.

Store vector embeddings

The following examples demonstrate how to store vector embeddings in Firestore.

Write operation with a vector embedding

The following example shows how to store a vector embedding in a Firestore document:

Python
Node.js

import { Firestore, FieldValue, } from "@google-cloud/firestore";

const db = new Firestore(); const coll = db.collection('coffee-beans'); await coll.add({ name: "Kahawa coffee beans", description: "Information about the Kahawa coffee beans.", embedding_field: FieldValue.vector([1.0 , 2.0, 3.0]) });

Go
Java

import com.google.cloud.firestore.CollectionReference; import com.google.cloud.firestore.DocumentReference; import com.google.cloud.firestore.FieldValue; import com.google.cloud.firestore.VectorQuery;

CollectionReference coll = firestore.collection("coffee-beans");

Map<String, Object> docData = new HashMap<>(); docData.put("name", "Kahawa coffee beans"); docData.put("description", "Information about the Kahawa coffee beans."); docData.put("embedding_field", FieldValue.vector(new double[] {1.0, 2.0, 3.0}));

ApiFuture future = coll.add(docData); DocumentReference documentReference = future.get();

Compute vector embeddings with a Cloud Function

To calculate and store vector embeddings whenever a document is updated or created, you can set up a Cloud Run function:

Python

@functions_framework.cloud_event def store_embedding(cloud_event) -> None: """Triggers by a change to a Firestore document. """ firestore_payload = firestore.DocumentEventData() payload = firestore_payload._pb.ParseFromString(cloud_event.data)

collection_id, doc_id = from_payload(payload)

Call a function to calculate the embedding

embedding = calculate_embedding(payload)

Update the document

doc = firestore_client.collection(collection_id).document(doc_id) doc.set({"embedding_field": embedding}, merge=True)

Node.js

/**

// Get the current value of the document's content field. const currentDocumentSnapshot = event.data.after as QueryDocumentSnapshot; const currentContent = currentDocumentSnapshot.get("content");

// Don't update the embedding if the content field did not change if (previousContent === currentContent) { return; }

// Call a function to calculate the embedding for the value // of the content field. const embeddingVector = calculateEmbedding(currentContent);

// Update the embedding field on the document. await currentDocumentSnapshot.ref.update({ embedding: embeddingVector, }); }

Go

// Not yet supported in the Go client library

Java

// Not yet supported in the Java client library

Create and manage vector indexes

Before you can perform a nearest neighbor search with your vector embeddings, you must create a corresponding index. The following examples demonstrate how to create and manage vector indexes with the Google Cloud CLI and the console. Vector indexes can also be managed with the Firebase CLI and Terraform.

Create a vector index

Google Cloud console

To manually create a new index from the Google Cloud console:

  1. In the Google Cloud console, go to the Databases page.
    Go to Databases
  2. Select the required database from the list of databases.
  3. In the navigation menu, click Indexes, and then click the Manualtab.
  4. Click Create Index.
    To index a vector field for vector searches, select Create vector index.
  5. Enter a Collection ID. Enter a vector field path and the number of vector embedding dimensions. Add the names of any additional fields you want to index and an index mode for each field.
    Click Save Index.

Your new index will show up in the list of manual indexes and Firestore will begin creating your index. When your index is done creating, you will see a green check mark next to the index.

gcloud

Before you create a vector index, upgrade to the latest version of the Google Cloud CLI:

gcloud components update

To create a vector index, use gcloud firestore indexes composite create:

gcloud firestore indexes composite create
--collection-group=collection-group
--query-scope=COLLECTION
--field-config field-path=vector-field,vector-config='vector-configuration'
--database=database-id

where:

The following example creates a composite index, including a vector index for field vector-field and an ascending index for fieldcolor. You can use this type of index topre-filter data before a nearest neighbor search.

gcloud firestore indexes composite create
--collection-group=collection-group
--query-scope=COLLECTION
--field-config=order=ASCENDING,field-path="color"
--field-config field-path=vector-field,vector-config='{"dimension":"1024", "flat": "{}"}'
--database=database-id

List all vector indexes

Google Cloud console

  1. In the Google Cloud console, go to the Databases page.
    Go to Databases
  2. Select the required database from the list of databases.
  3. In the navigation menu, click Indexes, and then click the Manualtab.
    The indexes table lists all indexes for the database. Vector indexes include a vector field with a icon.
gcloud

To list all indexes and retrieve index IDs:

gcloud firestore indexes composite list --database=database-id

Replace database-id with the ID of the database.

You can use th index ID to view more details about an index:

gcloud firestore indexes composite describe index-id --database=database-id

where:

Delete a vector index

Google Cloud console

  1. In the Google Cloud console, go to the Databases page.
    Go to Databases
  2. Select the required database from the list of databases.
  3. In the navigation menu, click Indexes, and then click the Manualtab.
  4. In the list of your manual indexes, click the More button for the index you want to delete. Click Delete.
  5. Confirm that you want to delete this index by clicking Delete Index from the alert.
gcloud

gcloud firestore indexes composite delete index-id --database=database-id

where:

Make a nearest-neighbor query

You can perform a similarity search to find the nearest neighbors of a vector embedding. Similarity searches require vector indexes. If an index doesn't exist, Firestore suggests an index to create using the gcloud CLI.

The following example finds 10 nearest neighbors of the query vector.

Python
Node.js

import { Firestore, FieldValue, VectorQuery, VectorQuerySnapshot, } from "@google-cloud/firestore";

// Requires a single-field vector index const vectorQuery: VectorQuery = coll.findNearest({ vectorField: 'embedding_field', queryVector: [3.0, 1.0, 2.0], limit: 10, distanceMeasure: 'EUCLIDEAN' });

const vectorQuerySnapshot: VectorQuerySnapshot = await vectorQuery.get();

Go
Java

import com.google.cloud.firestore.VectorQuery; import com.google.cloud.firestore.VectorQuerySnapshot;

VectorQuery vectorQuery = coll.findNearest( "embedding_field", new double[] {3.0, 1.0, 2.0}, /* limit */ 10, VectorQuery.DistanceMeasure.EUCLIDEAN);

ApiFuture future = vectorQuery.get(); VectorQuerySnapshot vectorQuerySnapshot = future.get();

Vector distances

Nearest-neighbor queries support the following options for vector distance:

Choose the distance measure

Depending on whether or not all your vector embeddings are normalized, you can determine which distance measure to use to find the distance measure. A normalized vector embedding has a magnitude (length) of exactly 1.0.

In addition, if you know which distance measure your model was trained with, use that distance measure to compute the distance between your vector embeddings.

Normalized data

If you have a dataset where all vector embeddings are normalized, then all three distance measures provide the same semantic search results. In essence, although each distance measure returns a different value, those values sort the same way. When embeddings are normalized, DOT_PRODUCT is usually the most computationally efficient, but the difference is negligible in most cases. However, if your application is highly performance sensitive, DOT_PRODUCT might help with performance tuning.

Non-normalized data

If you have a dataset where vector embeddings aren't normalized, then it's not mathematically correct to use DOT_PRODUCT as a distance measure because dot product doesn't measure distance. Depending on how the embeddings were generated and what type of search is preferred, either the COSINE or EUCLIDEAN distance measure produces search results that are subjectively better than the other distance measures. Experimentation with either COSINE or EUCLIDEAN might be necessary to determine which is best for your use case.

Unsure if data is normalized or non-normalized

If you're unsure whether or not your data is normalized and you want to useDOT_PRODUCT, we recommend that you use COSINE instead.COSINE is like DOT_PRODUCT with normalization built in. Distance measured using COSINE ranges from 0 to 2. A result that is close to 0 indicates the vectors are very similar.

Pre-filter documents

To pre-filter documents before finding the nearest neighbors, you can combine a similarity search with other query operators. The and andor composite filters are supported. For more information about supported field filters, see Query operators.

Python
Node.js

// Similarity search with pre-filter // Requires composite vector index const preFilteredVectorQuery: VectorQuery = coll .where("color", "==", "red") .findNearest({ vectorField: "embedding_field", queryVector: [3.0, 1.0, 2.0], limit: 5, distanceMeasure: "EUCLIDEAN", });

const vectorQueryResults = await preFilteredVectorQuery.get();

Go
Java

import com.google.cloud.firestore.VectorQuery; import com.google.cloud.firestore.VectorQuerySnapshot;

VectorQuery preFilteredVectorQuery = coll .whereEqualTo("color", "red") .findNearest( "embedding_field", new double[] {3.0, 1.0, 2.0}, /* limit */ 10, VectorQuery.DistanceMeasure.EUCLIDEAN);

ApiFuture future = preFilteredVectorQuery.get(); VectorQuerySnapshot vectorQuerySnapshot = future.get();

Retrieve the calculated vector distance

You can retrieve the calculated vector distance by assigning adistance_result_field output property name on the FindNearest query, as shown in the following example:

Python
Node.js

const vectorQuery: VectorQuery = coll.findNearest( { vectorField: 'embedding_field', queryVector: [3.0, 1.0, 2.0], limit: 10, distanceMeasure: 'EUCLIDEAN', distanceResultField: 'vector_distance' });

const snapshot: VectorQuerySnapshot = await vectorQuery.get();

snapshot.forEach((doc) => { console.log(doc.id, ' Distance: ', doc.get('vector_distance')); });

Go
Java

import com.google.cloud.firestore.VectorQuery; import com.google.cloud.firestore.VectorQueryOptions; import com.google.cloud.firestore.VectorQuerySnapshot;

VectorQuery vectorQuery = coll.findNearest( "embedding_field", new double[] {3.0, 1.0, 2.0}, /* limit */ 10, VectorQuery.DistanceMeasure.EUCLIDEAN, VectorQueryOptions.newBuilder().setDistanceResultField("vector_distance").build());

ApiFuture future = vectorQuery.get(); VectorQuerySnapshot vectorQuerySnapshot = future.get();

for (DocumentSnapshot document : vectorQuerySnapshot.getDocuments()) { System.out.println(document.getId() + " Distance: " + document.get("vector_distance")); }

If you want to use a field mask to return a subset of document fields along with a distanceResultField, then you must also include the value of distanceResultField in the field mask, as shown in the following example:

Python
Node.js

const vectorQuery: VectorQuery = coll .select('name', 'description', 'vector_distance') .findNearest({ vectorField: 'embedding_field', queryVector: [3.0, 1.0, 2.0], limit: 10, distanceMeasure: 'EUCLIDEAN', distanceResultField: 'vector_distance' });

Go
Java

import com.google.cloud.firestore.VectorQuery; import com.google.cloud.firestore.VectorQueryOptions; import com.google.cloud.firestore.VectorQuerySnapshot;

VectorQuery vectorQuery = coll .select("name", "description", "vector_distance") .findNearest( "embedding_field", new double[] {3.0, 1.0, 2.0}, /* limit */ 10, VectorQuery.DistanceMeasure.EUCLIDEAN, VectorQueryOptions.newBuilder() .setDistanceResultField("vector_distance") .build());

ApiFuture future = vectorQuery.get(); VectorQuerySnapshot vectorQuerySnapshot = future.get();

for (DocumentSnapshot document : vectorQuerySnapshot.getDocuments()) { System.out.println(document.getId() + " Distance: " + document.get("vector_distance")); }

Specify a distance threshold

You can specify a similarity threshold that returns only documents within the threshold. The behavior of the threshold field depends on the distance measure you choose:

The following example shows how to specify a distance threshold to return up to 10 nearest documents that are, at most, 4.5 units away using the EUCLIDEAN distance metric:

Python
Node.js

const vectorQuery: VectorQuery = coll.findNearest({ vectorField: 'embedding_field', queryVector: [3.0, 1.0, 2.0], limit: 10, distanceMeasure: 'EUCLIDEAN', distanceThreshold: 4.5 });

const snapshot: VectorQuerySnapshot = await vectorQuery.get();

snapshot.forEach((doc) => { console.log(doc.id); });

Go
Java

import com.google.cloud.firestore.VectorQuery; import com.google.cloud.firestore.VectorQueryOptions; import com.google.cloud.firestore.VectorQuerySnapshot;

VectorQuery vectorQuery = coll.findNearest( "embedding_field", new double[] {3.0, 1.0, 2.0}, /* limit */ 10, VectorQuery.DistanceMeasure.EUCLIDEAN, VectorQueryOptions.newBuilder() .setDistanceThreshold(4.5) .build());

ApiFuture future = vectorQuery.get(); VectorQuerySnapshot vectorQuerySnapshot = future.get();

for (DocumentSnapshot document : vectorQuerySnapshot.getDocuments()) { System.out.println(document.getId()); }

Limitations

As you work with vector embeddings, note the following limitations:

What's next