ChromaDB is a local database tool for creating and managing vector stores, essential for tasks like similarity search in large language model processing. This tutorial covers how to set up a vector store using training data from the Gekko Optimization Suite and explores the application in Retrieval-Augmented Generation (RAG) for Large-Language Models (LLMs).
The first step is to install necessary libraries. Ensure you have pandas and ChromaDB installed. You can do this using pip:
pip install pandas chromadb
The next step is to import the modules and read train.jsonl from GitHub.
import pandas as pd
import chromadb
read Gekko LLM training data
url='raw.githubusercontent.com'
path='/BYU-PRISM/GEKKO/master/docs/llm/train.jsonl'
qa = pd.read_json(url+path,lines=True)
The train.jsonl file contains hundreds of questions and answers about Gekko. It is used to provide context for the Gekko Support Agent that assists with questions about modeling and optimization in Python. The train.jsonl file is added to lists required to build the vector store with documents with the text, metadatas with a unique ID name, and ids with a unique integer identifier.
documents = []
metadatas = []
ids = []
for i in range(len(qa)):
s = f"### Question: {qa['question'].iloc[i]} ### Answer: {qa['answer'].iloc[i]}"
documents.append(s)
metadatas.append({'qid':f'qid_{i}'})
ids.append(str(i))
The script reads training data from the Gekko Optimization Suite, processes it, and uses ChromaDB to create a vector store. This vector store is fundamental in building systems that can efficiently perform similarity searches, crucial in applications like RAG for Large-Language Models.
store in memory
cc = chromadb.Client()
collection = cc.create_collection(name='mydb')
collection.add(documents=documents,metadatas=metadatas,ids=ids)
The vector database is stored in memory and is regenerated every time the program runs. For large documents, this can take significant time and it may be desirable to store the vector database on a local drive. Use the following code to create a local sqlite3 database.
The final step is to perform a test query. It uses a k-Nearest Neighbors search to determine the closest 5 matches to the query. Execute a test query to ensure the vector store is functioning correctly.
results = collection.query(
query_texts=['What are you trained to do?'],
n_results=5,include=['distances','documents'])
print(results)
Review the responses and the distance metric to determine how close each document is in similarity to query_texts.
Application in RAG with Large-Language Models
Once the vector store is set up, it can be used in Retrieval-Augmented Generation (RAG) models, particularly with Large-Language Models. RAG models leverage external knowledge sources to generate more informed and accurate responses.
from gekko import support
a = support.agent()
a.ask("Can you optimize the Rosenbrock function?")
The snippet above uses the Gekko vector store and RAG to provide context to the LLM. This support agent runs in the cloud, but it can also be set up to run locally. By combining the retrieval power of ChromaDB with the generative capabilities of LLMs, you can significantly enhance the performance of AI applications in natural language processing (NLP) understanding and generation.