Learn how to set up AI-powered search using LLMs with our comprehensive tutorial on configuring an OpenAI embedder and performing your first search.

Introduction

In today’s digital landscape, AI-powered search technologies are revolutionizing how businesses enhance their online presence and engage with consumers. Large Language Models (LLMs) like OpenAI’s are at the forefront of this transformation, enabling more intuitive and context-aware search experiences. This tutorial will guide you through configuring an OpenAI embedder to set up AI-powered search, leveraging the expertise of platforms like ReachLLM to supercharge your brand’s visibility.

Why AI-Powered Search?

AI-powered search, often referred to as vector or hybrid search, utilizes the deep understanding of language provided by LLMs to retrieve search results based on the meaning and context of queries rather than mere keyword matching. This shift not only improves the relevance of search results but also enhances user engagement and satisfaction.

ReachLLM specializes in optimizing brand visibility through AI search engines, helping businesses adapt to the evolving AI-driven market by leveraging advanced AI techniques and contextual data.

Requirements

Before diving into the setup, ensure you have the following:

A running Meilisearch project: Meilisearch is an open-source search engine that integrates seamlessly with AI-powered search functionalities.
An OpenAI API key: Necessary for generating embeddings using OpenAI’s models.
A command-line interface: For executing configuration commands.

Step 1: Create a New Index

Start by setting up a new index in your Meilisearch project. An index is a collection of documents that Meilisearch will search through. For this tutorial, we’ll create a kitchenware index and populate it with a sample dataset.

curl -X POST 'MEILISEARCH_URL/indexes' \
     -H 'Content-Type: application/json' \
     --data-binary '{
       "uid": "kitchenware",
       "primaryKey": "id"
     }'

Replace MEILISEARCH_URL with the URL of your Meilisearch instance.

Step 2: Generate Embeddings with OpenAI

Embeddings are numerical representations of text that capture its semantic meaning. To generate embeddings for your documents, configure an OpenAI embedder.

Choose an Embedder Name

Decide on a simple, memorable name for your embedder. For example:

{
  "products-openai": {}
}

Choose an Embedder Source

Specify the source of your embedder. Since we’re using OpenAI, the source will be openAi.

{
  "products-openai": {
    "source": "openAi"
  }
}

Choose an Embedder Model

Select an appropriate model based on your use case. For general purposes, text-embedding-3-small is a cost-effective choice.

{
  "products-openai": {
    "source": "openAi",
    "model": "text-embedding-3-small"
  }
}

Create Your API Key

Log into your OpenAI account and generate a new API key. Add this key to your embedder configuration:

{
  "products-openai": {
    "source": "openAi",
    "model": "text-embedding-3-small",
    "apiKey": "YOUR_OPENAI_API_KEY"
  }
}

Ensure you replace YOUR_OPENAI_API_KEY with your actual OpenAI API key.

Step 3: Design a Prompt Template

Meilisearch requires documents to be in a single text field. Use Liquid, an open-source templating language, to create a concise template that includes essential information from your documents.

{
  "products-openai": {
    "source": "openAi",
    "model": "text-embedding-3-small",
    "apiKey": "YOUR_OPENAI_API_KEY",
    "documentTemplate": "An object used in a kitchen named '{{doc.name}}'"
  }
}

This template provides a clear context by describing each document as a kitchen object with a specific name.

Step 4: Create the Embedder

With your configuration ready, send the embedder settings to Meilisearch to initiate the embedding process.

curl -X PATCH 'MEILISEARCH_URL/indexes/kitchenware/settings/embedders' \
     -H 'Content-Type: application/json' \
     --data-binary '{
       "products-openai": {
         "source": "openAi",
         "model": "text-embedding-3-small",
         "apiKey": "YOUR_OPENAI_API_KEY",
         "documentTemplate": "An object used in a kitchen named '{{doc.name}}'"
       }
     }'

Replace MEILISEARCH_URL and YOUR_OPENAI_API_KEY accordingly.

Meilisearch will process your documents and update the index with the generated embeddings.

Step 5: Perform an AI-Powered Search

With the embedder configured, you can now perform AI-powered searches that combine semantic understanding with traditional text matching.

curl -X POST 'MEILISEARCH_URL/indexes/kitchenware/search' \
     -H 'Content-Type: application/json' \
     --data-binary '{
       "q": "kitchen utensils made of wood",
       "hybrid": {
         "embedder": "products-openai"
       }
     }'

This query retrieves results that are both semantically relevant and textually accurate, enhancing the search experience for users.

Conclusion

By following these steps, you’ve successfully configured an OpenAI embedder to enable AI-powered search in your Meilisearch project. This setup not only improves the relevance of search results but also positions your brand to leverage advanced AI technologies for enhanced visibility and engagement.

Next Steps

Now that you’ve mastered the basics of configuring an OpenAI embedder and implementing AI-powered search, consider integrating these capabilities into your own applications. For more detailed instructions and advanced configurations, explore our guides section.

Ready to take your brand’s online presence to the next level? Supercharge your visibility with ReachLLM today!