Skip to main content

Vector store RAG

Retrieval Augmented Generation, or RAG, is a pattern for training LLMs on your data and querying it.

RAG is backed by a vector store, a vector database which stores embeddings of the ingested data.

This enables vector search, a more powerful and context-aware search.

We've chosen Astra DB as the vector database for this starter flow, but you can follow along with any of Langflow's vector database options.

Prerequisites​

Open Langflow and start a new project​

  1. From the Langflow dashboard, click New Flow.
  2. Select Vector Store RAG.
  3. The Vector Store RAG flow is created.

Build the vector RAG flow​

The vector store RAG flow is built of two separate flows for ingestion and query.

The Load Data Flow (bottom of the screen) creates a searchable index to be queried for contextual similarity. This flow populates the vector store with data from a local file. It ingests data from a local file, splits it into chunks, indexes it in Astra DB, and computes embeddings for the chunks using the OpenAI embeddings model.

The Retriever Flow (top of the screen) embeds the user's queries into vectors, which are compared to the vector store data from the Load Data Flow for contextual similarity.

  • Chat Input receives user input from the Playground.
  • OpenAI Embeddings converts the user query into vector form.
  • Astra DB performs similarity search using the query vector.
  • Parse Data processes the retrieved chunks.
  • Prompt combines the user query with relevant context.
  • OpenAI generates the response using the prompt.
  • Chat Output returns the response to the Playground.

  1. Configure the OpenAI model component.

    1. To create a global variable for the OpenAI component, in the OpenAI API Key field, click the  Globe button, and then click Add New Variable.
    2. In the Variable Name field, enter openai_api_key.
    3. In the Value field, paste your OpenAI API Key (sk-...).
    4. Click Save Variable.

  2. Configure the Astra DB component.

    1. In the Astra DB Application Token field, add your Astra DB application token. The component connects to your database and populates the menus with existing databases and collections.
    2. Select your Database. If you don't have a collection, select New database. Complete the Name, Cloud provider, and Region fields, and then click Create. Database creation takes a few minutes.
    3. Select your Collection. Collections are created in your Astra DB deployment for storing vector data.
    info

    If you select a collection embedded with Nvidia through Astra's vectorize service, the Embedding Model port is removed, because you have already generated embeddings for this collection with the Nvidia NV-Embed-QA model. The component fetches the data from the collection, and uses the same embeddings for queries.

  3. If you don't have a collection, create a new one within the component.

    1. Select New collection.

    2. Complete the Name, Embedding generation method, Embedding model, and Dimensions fields, and then click Create.

      Your choice for the Embedding generation method and Embedding model depends on whether you want to use embeddings generated by a provider through Astra's vectorize service, or generated by a component in Langflow.

      • To use embeddings generated by a provider through Astra's vectorize service, select the model from the Embedding generation method dropdown menu, and then select the model from the Embedding model dropdown menu.
      • To use embeddings generated by a component in Langflow, select Bring your own for both the Embedding generation method and Embedding model fields. In this starter project, the option for the embeddings method and model is the OpenAI Embeddings component connected to the Astra DB component.
      • The Dimensions value must match the dimensions of your collection. This field is not required if you use embeddings generated through Astra's vectorize service. You can find this value in the Collection in your Astra DB deployment.

      For more information, see the DataStax Astra DB Serverless documentation.

If you used Langflow's Global Variables feature, the RAG application flow components are already configured with the necessary credentials.

Run the Vector Store RAG flow​

  1. Click the Playground button. Here you can chat with the AI that uses context from the database you created.
  2. Type a message and press Enter. (Try something like "What topics do you know about?")
  3. The bot will respond with a summary of the data you've embedded.
Search