RAG with Azure AI Search and Azure Open AI in 9 minutes

Рет қаралды 23,058

Күн бұрын

Equipment used
Blue Yeti Microphone [ www.amazon.in/Blue-Recording-... ]
RAG [ Retrieval-Augmented Generation ] concept explained
📍 How we do we ingest data into our RAG architecture?
📍 How we do we query from our RAG architecture?
📍 Detailed use of Azure AI Search and Azure OpenAI in RAG
📍 Demo of our built in RAG application
Code: github.com/ambarishg/AZURE-AI...
Playlist provided is [ • Generative AI ]

Пікірлер: 49

@UmairMateenKhan 2 ай бұрын

Awesome tutorial Ambarish. Keep up the good work.

@paullopez_ai 6 ай бұрын

Great video! Thanks for sharing the code too, greatly appreciated.

@ambarishg 6 ай бұрын

Glad to help!

@KenGarff-zp1mh Ай бұрын

Thanks for the video!

@pedromoraes5197 5 ай бұрын

Great video, Man!! Thanks for sharing.

@ambarishg 5 ай бұрын

Thanks for watching!

@samosertogo 6 ай бұрын

Well done! Brilliant work and very replicable. Thanks a lot!

@ambarishg 6 ай бұрын

Many thanks!

@Melophile_Gaurav 12 күн бұрын

Hey, this was helpful! :)

@davemcshane5292 6 ай бұрын

brilliant video!! Much better than the marketing ones from microsoft :)

@ambarishg 6 ай бұрын

Wow, thanks!

@italoaguiar 6 ай бұрын

thanks! very informative video!

@ambarishg 6 ай бұрын

Glad it was helpful!

@murtuza.chawala 4 ай бұрын

Awesome Video I tried it too wanted to know any way we can do reinforcement learning making the bot train on our feedback

@saideepj6998 Ай бұрын

Thanks for such a detailed content. Is the uploaded data going to private ? will that data be trained with open source LLM's . How to add extra layer of security to make sure that the uploaded data is within my private GPT

@limjuroy7078 4 ай бұрын

Here is my little 2 cents', can you do an end-to-end tutorial on how you do this, including creating Azure AI Service, Azure Storage as well as the coding part? 😅😅😅

@ambarishg 3 ай бұрын

Great suggestion!

@xyz-vv5tg 3 ай бұрын

Please. I need it. I have no idea on Azure or any of its services. I'm just exploring and getting confused

@JohnPatrickGallagher Ай бұрын

I’ll second this, I’m getting Started and it’s a great tutorial so far!

@indrayne1840 4 ай бұрын

Sir I have one doubt, like we are using Azure Cognitive search and the index updates everyday midnight. So we have both new content and old content with us. Now what I want is to retrieve the most recent content first and than old content. How can I implement it?

@shashanksagar4878 Ай бұрын

Do we not require any embeddings and vector database here? Also, is vector search not required? If so when is it required? Thank you

@Momentum_Option_Buyer 27 күн бұрын

1:01 Is Azure search is or acts like a vector database that keeps the knowledge? Please clarify I have seen several videos but confused on this question.

@pooblock4092 Ай бұрын

Does this work with files with size large than 16mb?

@akankshamakkar8538 6 күн бұрын

So here we need 3 services, right? azure openai, azure ai search and azure blob storage?

@mohammadsohail6534 6 ай бұрын

Hi @ambarishg Great video Will the Azure Search AI store the indexed data or it is stored in StorageAcount

@ambarishg 6 ай бұрын

In Azure AI Search

@andonii46 5 ай бұрын

Amazing video! One question, if you have the data source a blob storage, if you delete a pdf file from that blob storage, when you run the indexer is the reference and information about this deleted document deleted from the index as well? thanks!

@ambarishg 5 ай бұрын

The data is stored in the Azure AI Search index. Please note that if we have to remove a document from the index, we have to delete it from the index. Please also note that in the file upload_docs.ipynb, the function remove_from_index can remove documents from the index

@Momentum_Option_Buyer Ай бұрын

You didn't use vector database in this specific video (use case) to check data into vectors?

@ambarishg Ай бұрын

Please watch the following for VECTORS 1. kzfaq.info/get/bejne/eLWAictelbSxaIk.html - Vector , Hybrid, Semantic search with Azure AI Search in 11 minutes 2. kzfaq.info/get/bejne/p7CcZqyVqLHKpnk.html -RAG With Azure AI Search using Vector/ Hybrid / Exhaustive KNN / Semantic Reranker in 15 minutes

@sarak8467 6 ай бұрын

Hi Ambarish, thank you for the very helpful video! I have an enquiry and I hope you could help me with it. So, on the Azure Platform, I have created a a Search Service and an OpenAI Service and connected both in order to be able to ask questions about the data stored on an index. The data is in a Blob Storage container that has multiple folders in it. What I want to do when searching for information, is to filter the folders; I want to choose the folder (either by name or path), so that the search service only returns the content of that folder, and then the openAI service uses that content as a reference to generate an answer to my question. Do you know how I could do that?

@ambarishg 6 ай бұрын

In the code we have added metadata while putting data into the search index. Please add your customized metadata while putting the data into the search index. While doing the Search on Azure AI , we should provide the filter with the proper metadata and it should filter accordingly. Hope this helps. May be I should so a followup video explaining this in detail. Thanks again for your questions.

@sarak8467 6 ай бұрын

@@ambarishg Thank you so muh for your reply! IT would be perfect if you could show us how to do it. If you can, please show us how to do it on Azure and through code. (I use either Python or C#) Thanks again!

@ankan54 7 ай бұрын

have you tried creating a vector index in ACS? will vector search be better than semantic search?

@ambarishg 6 ай бұрын

Good suggestion will try. Usually Hybrid search is better

@ambarishg 6 ай бұрын

@ankan54 . Please have a look at the video kzfaq.info/get/bejne/p7CcZqyVqLHKpnk.html where we discuss the following RAG with Azure AI Search with Azure OpenAI with different search techniques and Langchain with Conversation Chain, Prompt Template and Conversation Buffer 📍 Vector Search 📍 Hybrid Search 📍 Exhaustive KNN Search 📍 Hybrid Search with Semantic reranking 📍 RAG with Azure AI Search with Azure OpenAI 📍 Use Langchain with Conversation Chain 📍 Use Langchain with Prompt Template 📍 Use Langchain with Conversation Buffer

@LumityGaming 6 ай бұрын

Hi Ambarish, amazing video! Thank you so much for taking the time to make it. Would you be able to help me understand where I can locate the config information? I apologize if some are self-explanatory, I dont even know where to begin on a few of them searchservice = values_env['searchservice'] index = values_env['index'] searchkey = values_env['searchkey'] category=values_env['category'] #AZURE STORAGE CONFIGURATION storageaccount = values_env['storageaccount'] container=values_env['container'] storagekey=values_env['storagekey'] localpdfparser=values_env['localpdfparser'] verbose=values_env['verbose'] FILE_PATH = values_env['FILE_PATH'] formrecognizerservice=values_env['formrecognizerservice']

@ambarishg 6 ай бұрын

Please create a .env file. In the .env file , put the key in the format = ; example searchservice= I should have put a .env.example file for clarity. Will put that shortly and explain it.

@ambarishg 6 ай бұрын

Added the .env.sample file in the repository.Hope this helps

@limjuroy7078 4 ай бұрын

What if I want the app "also" has the ability to upload our own PDF file and query the content of the just uploaded file through UI?

@ambarishg 4 ай бұрын

Yes it is possble

@krishnavasu3499 5 ай бұрын

Great video! Very helpful. Instead of azure AI search, if I wanted my retriever to be Qdrant, would it be possible? I have a few files in Qdrant and would like to use Azure open AI to get the output from Qdrant and give the result to the user. If possible, how could I do it?

@ambarishg 5 ай бұрын

Yes you can! kzfaq.info/get/bejne/Y5tjq7yDm9-1cY0.html This shows RAG with QDRANT [ RAG with Opensource Tools - Qdrant and Mistral in 6 minutes ]. You may please modify this for Azure Open AI

@ambarishg 5 ай бұрын

kzfaq.info/get/bejne/npp2Y5yXvM7dmJs.html This shows RAG with LlamaIndex - Qdrant and Azure OpenAI in 9 minutes