Building Semantic Search with an Embeddings API

What is semantic search?

Traditional search engines look for exact matching keywords. If you search for "car", you won't find results that only contain "vehicle" or "automobile". Semantic search understands the meaning of words in a specific context by converting text into numerical vectors (embeddings).

Through our API Directory, you will find various models specialized in generating high-quality text embeddings.

From Text to Vectors (Embeddings)

An embeddings model takes a piece of text—whether it's a single sentence or an entire paragraph—and translates it into a list of numbers, for example, a 1536-dimensional vector. Sentences with similar meanings get vectors that are close to each other in a multidimensional space.

The Pipeline in Pseudocode

Setting up a simple semantic search pipeline consists of two main phases: indexing the data and executing the actual search query.

1. Indexing (Preparing Data)

// Pseudocode for indexing documents
for document in all_documents:
    text_chunk = split_into_small_parts(document.content)
    vector = api.get_embedding(text_chunk)
    database.save(vector, text_chunk, metadata)

2. Search (Similarity Search)

// Pseudocode for an incoming search query
function search(user_query):
    query_vector = api.get_embedding(user_query)

    // Calculate the cosine similarity
    results = database.vector_search(
        query_vector,
        limit = 5
    )

    return results

When to Choose Semantic Search Over Keywords?