Advanced RAG tutorial with Llamaindex & OpenAI GPT: Sentence Window Retrieval vs Basic Chunking

Рет қаралды 5,890

Күн бұрын

Correction: at 1:53, I said that an embedding is a x digit string. That is not correct, it should be a list of x numbers.
Google Colab Code: colab.research.google.com/dri...
Link to Diagram: link.excalidraw.com/readonly/...
Why build your own retrieval augmented generation pipeline when OpenAI's custom GPTs can do it out of the box? Did you know that the OpenAI solutions, as of the making of this video, are not scalable to large knowledge bases? Also, having your own pipeline gives you a lot more control over the design which you will need if you are building an enterprise grade top-notch system.
In this tutorial, we will talk through a number of advanced techniques such as sentence window retrieval, hierarchical automerge retrieval, returning Top K results vs. greedy search, reranking etc.
We will also work through some code and do a real comparison between basic chunking vs. sentence retrieval strategies.

Пікірлер: 32

@mikestaub 6 ай бұрын

This is currently the best RAG tutorial on the internet.

@victorfeight9644 2 ай бұрын

clear effective explanations thank you

@sitedev 6 ай бұрын

Great video with an awesome easy to follow explanation of RAG. Reminds of a recent Andrej Karpathy video.

@unclecode 6 ай бұрын

Fascinating! Your approach to teaching and presenting is poetic. It is well organized, well explained, and well illustrated. Indeed, kudos to you. If I could, I would subscribe to your channel twice!

@danielvalentine132 6 ай бұрын

Great job explaining window and how vector store and doc store relate and where window lives. I’ve been trying to understand this aspect of llamaindex, and you made it very clear!

@hubel-labs 6 ай бұрын

Yeah, it took me a while to figure that out too. Glad it helped you!

@nazihfattal974 6 ай бұрын

As always, well prepared, easy to follow video that delivers a lot of information and value. Thank you!

@arjoai 6 ай бұрын

Wow, that’s a wonderful piece of advice from such a talented professional in the field. Thank you 😊

@MaliRasko 6 ай бұрын

Your explanations and delivery is on point. Thank you for an excellent content and relaxed narration style.

@ginisksam 4 ай бұрын

Llama Index has new version 0.10 - will migrate your codes n learn same time. Thanks for introducing Sentence Window Retrieval. Most basic straight-split and retrieve/chat doesnt produce much meanings on our docs.

@mayurmoudhgalya3840 6 ай бұрын

Another great video!

@natal10 6 ай бұрын

Great video!! Love it

@hubel-labs 6 ай бұрын

Thank you!!

@nunoalexandre6408 6 ай бұрын

Love it!!!!!!!!!!!!!!!!!

@sivi3883 5 ай бұрын

Awesome video explained very clearly! Thanks a ton! If I may ask, what tool do you use for those visual flows. Love it!

@JOHNSMITH-ve3rq 6 ай бұрын

Wow _ darn useful !!

@sayanbhattacharyya95 5 ай бұрын

Really cool video! Is there an "ideal" or "recommended" value of window_size?

@Work_Pavan-mu9ye 6 ай бұрын

Excellent video. Liked the workflow you showed in the beginning. What SW are you using to create this workflow?

@hubel-labs 6 ай бұрын

Excalidraw

@peteredmonds1712 6 ай бұрын

I'm a bit confused on the use case for re-ranking. Doesn't that defeat the purpose the top-k search in that we include all chunks, significantly increasing the number of tokens we use? Is the idea to do re-ranking with a smaller & cheaper LLM before sending the resultant top-K chunks to a more robust LLM?

@hubel-labs 6 ай бұрын

yeah, it took me a while to understand it as well. It does end up using more tokens but I guess that can be, as you said, mitigated with a lower spec LLM. The idea is that embeddings is not a perfect representation of the text - it's a good way to sort through millions of text chunks efficiently but the exact ranking within the top 20, for instance, may not be as good as using the text directly. So if you were to do reranking, you might use the embeddings to return j results (where j is somewhat larger than k) and then rerank those j results using LLM into a final top k that you then pass into your user-facing LLM.

@peteredmonds1712 6 ай бұрын

Small correction: embedding are not a 1536 digit number but of vector of size 1536

@hubel-labs 6 ай бұрын

Yes, you are right …. I was thinking of hashes for some reason! I’ll add a correction!

@goonymiami 6 ай бұрын

Seeing your explaination at around 09:30, it seems like we can only use K windows to serve as knowledge base to answer a prompt. What If the prompt asks information that is contained in more than K windows? Like if I have several documents containing each a bio of a person, and if the user asks to sort those 10 people by age... how can it figure it out? I guess we can use a big value for K, if the cosine similarity engine can take it... but I am guessing providing too much context to the LLM will cost a lot of money?

@hubel-labs 6 ай бұрын

Yeah, it wouldn’t work very well in that scenario. I wonder if perhaps there is a strategy where the LLM can determine the right K based on what it is trying to do. Or instead of setting K to be a fixed number, to return all chunks where the cosine similarity is above a given threshold.