Graph RAG: Improving RAG with Knowledge Graphs

Рет қаралды 43,614

Күн бұрын

Discover Microsoft’s groundbreaking GraphRAG, an open-source system combining knowledge graphs with Retrieval Augmented Generation to improve query-focused summarization. I’ll guide you through setting it up on your local machine, demonstrate its functions, and evaluate its cost implications.
LINKS:
Graph RAG: tinyurl.com/y3f9cbnd
Github: github.com/microsoft/graphrag
GraphRAG flowchart: tinyurl.com/2s4ytcur
RAG flowchart: tinyurl.com/56mwzkk7
Community Creation: tinyurl.com/54n88c2j
💻 RAG Beyond Basics Course:
prompt-s-site.thinkific.com/c...
Let's Connect:
🦾 Discord: / discord
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: / promptengineering
💼Consulting: calendly.com/engineerprompt/c...
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Newsletter, localgpt:
tally.so/r/3y9bb0
00:00 Introduction to GraphRAG and Its Cost Issue
00:44 Understanding Traditional RAG
01:46 Limitations of Traditional RAG
02:22 Introduction to GraphRAG
02:39 Technical Details of GraphRAG
05:46 Setting Up GraphRAG on Your Local Machine
06:22 Running the Indexing Process
12:00 Running Queries with GraphRAG
14:26 Cost Implications and Alternatives
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

Пікірлер: 116

@aswarytiwari8806 29 күн бұрын

Would really appreciate the comparison of all the Graph based RAG frameworks, like LlamaIndex's Property Graph RAG, MS's GraphRAG etc.

@kennethpinpin9053 Ай бұрын

Excellent concise introduction on GraphRAG and why, how, when it is needed. Please compare this with Neo4j and Llama Index.

@abhijitbarman 29 күн бұрын

Thanks for sharing this video.I would love to see a comparison between different Graph RAG solutions.

@aussie84tom 27 күн бұрын

Would love this too, especially neo4j and a focus on running it locally or with GROQ

@shubhamgattani5357 26 күн бұрын

@@aussie84tom Agree!

@TijsZwinkels 24 күн бұрын

+1 - Would be very interesting to see whether these perform comparatively, or whether one is better than the other.

@VivekHaldar Ай бұрын

Love the focus on cost tradeoff. Thanks!

@roguesecurity Ай бұрын

Thanks for sharing this video. It was very informative. I would love to see a comparison between different Graph RAG solutions.

@danieljbukowski Ай бұрын

+1 to this -- very interesting and would love to hear about other approaches using graphs!

@andytan4978 27 күн бұрын

@potatodog7910 11 күн бұрын

Same

@robboerman9378 Ай бұрын

Very useful, also good to share the associated cost. It would be very interesting to see a video comparing the different GraphRAG implementations as you mentioned.

@NimaAmini2000 Ай бұрын

Great job. Please publish more videos about GraphRAG with other competitors.

@positivevibe142 Ай бұрын

Thanks for stating that Microsoft Graph RAG is limited in contextual understanding, I already noticed that but thought that I messed up somewhere in my setup/building. Excellent, informative, and precise details! Thanks again!

@user-wr4yl7tx3w Ай бұрын

this looks great. yes, would be interested to compare against llamaindex.

@danielshurman1061 Ай бұрын

Another great choice of important topics and video to show the current best implementations. Thank you!

@engineerprompt Ай бұрын

Thank you. Always glad to see your comments.

@aa-xn5hc Ай бұрын

Great. Please more on this topic

@rajpootatul02 13 күн бұрын

Thanks for sharing this video. It was highly informative. I'd be interested in seeing a comparison of different Graph RAG solutions.

@dawid_dahl 6 сағат бұрын

Great overview. Thank you!

@CollinParan Ай бұрын

Interested in the Neo4J version

@paulmiller591 Ай бұрын

Great video and very timely. Please do more like these.

@engineerprompt Ай бұрын

thank you, planning on doing that :)

@mehmetbakideniz 16 күн бұрын

very usefull indeed! please publish more videos comparing different frameworks in terms of accuracy and the cost.

@MPReilly2010 Ай бұрын

Excellent! Outstandingly thorough and clear details. Subscribing now.

@engineerprompt Ай бұрын

thank you!

@dewanjeesoma 37 минут бұрын

Thanks for sharing the video.I would love to see a comparison between different Graph RAG solutions. I have one question that for incremental document ingestion how the relationship build with the existing document set?

@Ayush-tl3ny Ай бұрын

Can you please tell how can we change the embedding model to use some open source embedding

@andrewandreas5795 Ай бұрын

Great video! Could this be used with other models like Claude or maybe even OS models?

@jeffsteyn7174 Ай бұрын

Excellent. Going to subscribe for more info. Maybe i missed it but it seems like theres no specialised db used like with neo4j. So effectively we can use any data store?

@DayLearningIT-hz5kj Ай бұрын

Please share more about this topic 👍👍 so the most expensive part is in creating the graph ? Do you think that it is really improved the accuracy of the response ?

@armans4494 Ай бұрын

Great video! Thank you

@mr.anonymus6110 Ай бұрын

comparing them and a short video showcasing it on a local llm would be nice for sure

@DePhpBug Ай бұрын

Been wondering , especially the book.txt , let's say if is a .json object will our RAG works well with json objective when we feed those data into doing embedding etc through the RAG proceess?

@henkhbit5748 Ай бұрын

Thanks, if we can use open-source LLM then the cost will be less or use groq, right?. There are no examples using python code instead of command line?

@ganesh_kumar Ай бұрын

Baring the cost, can we use it for large number of docs and will it have good accuracy, what about the time taken for the complete indexing? And latency??

@dcmumby Ай бұрын

Thanks , what uses would traditional RAG be more effective ? Would love to see more detailed implementation as well as using alternative providers. Also very interested in comparing the competition

@engineerprompt Ай бұрын

In cases where you have well structured QA dataset, e.g. HR policies. In those cases you don't really need advanced techniques like knowledge Graphs.

@MuhammadZubair-n7d Ай бұрын

yes please make a comparison video and do you have videos for using other techniques as well?

@engineerprompt 29 күн бұрын

will do the comparison. I dont' have those videos up but planning on them soon.

@jekkleegrace Ай бұрын

yes please compare with llama index

@mohammmedalmulhem622 24 күн бұрын

Thank you for the great video. Is there a maximum input size for creating a GraphRAG? Every time I try to insert my documents, I get an error with "create_summarized_entities."

@joshlyn9041 16 күн бұрын

Thanks for the video, could you explain how we can go to the actual data sources that GraphRAG cites? like [Data: Reports (63,65)], cheers

@IslandDave007 Ай бұрын

It would be very interesting to see the results of this in global mode compared to dropping the whole book into the prompt of Gemini or Claude if it fit under their token limit. Obviously once you get beyond those limits RAG is required. Also would be great to see this run fully locally against a standard RAG local solution using the exact same LLM.

@ledinhngoc1102 22 күн бұрын

Is there any topic talk about entity ambiguous or entity linking in Graph RAG? How the system handle entities with different form but are synonyms?

@tasfiulhedayet Ай бұрын

Interested to know about the difference between those different frameworks

@engineerprompt Ай бұрын

Will do.

@lesptitsoiseaux Ай бұрын

How can I add metadata that can be valuable to use as entities? I don't mind buying your course if it helps my use case: I am building a recommendation engine for classes based off transcripts (multiple per class).

@rockypunk91 21 күн бұрын

If you index different documents at different point of time. We end up with multiple artifacts in the output folder. How should one do a search over all outputs. Like a production level application

@DEEPANMN Ай бұрын

Is this possible use a networkx graph library instead of LLM generated graph! I have readymade graph on my private dataset

@engineerprompt Ай бұрын

I think its possible. You will need the LLM to generate a query that can traverse the graph

@gregsLyrics Ай бұрын

Always amazing vid and wisdom. I do wish there was a Star-Trek Vulcan mind meld available. There is so much I do not understand and I think I am jumping in way ahead of the learning curve - which makes this task that much more difficult. I have a vast amount of scanned pdf that I think is perfect for this, except the "scanned" part. These documents are not machine readable and I greatly fear OCR (garbage in garbage out). Plus I have tens of thousands of pages of bank statements, credit card statements, investment statements , tax returns that I need to analyze and tie to the data. Currently doing all of this manually with printed pages, post-it notes and highlighters. Will your RAG course get me up to speed and teach a design process with the various tools that I will be able to grasp?

@engineerprompt Ай бұрын

The RAG course is focused on different techniques on how to build better RAG applications. In this course, I don't go too much into the ingestion part. For your problem, you might want to look into multimodal models like Claude or Gemini where you can directly process these scanned pages but the cost can add up quickly

@BACA01 Ай бұрын

For this you'll need a RAG with local vision models VLLMs

@BACA01 Ай бұрын

Can we use it with VLLMs?

@JatinKashyap-Innovision Ай бұрын

Please create the comparison video. Thanks.

@josephroman2690 Ай бұрын

interested in the video of comparison of all neoj4, llamaindex and this one

@mohanmadhesiya3116 Ай бұрын

we want comparing videos

@unclecode Ай бұрын

You know, I wonder if you could check the cost breakdown and token consumption for the indexing and inference phases. How much of the seven dollars is for inference? If it's low, we could just index once, right? But seven dollars is huge, so it doesn't work. Also, I saw the tokens moved were 900,000+ context tokens and around 160,000 generated tokens. There's room for improvement. I don't think we need 900,000 context tokens for that question. Maybe those context tokens are from the initial prompts used for indexing. If that's the case, rerunning it and noting the costs for indexing vs. inference would be useful. By the way, great content as usual. The knowledge graph concept really clicked for me, like the linear flat nave RAG, a longstanding computer science idea. Its connection to ontology, which interests me, is amazing. Remember Stanford's WordNet and VerbNET? They were golden datasets for extracting relationships and similarities. Now, using large language models to generate knowledge bases from text lets us use those good old time algorithms safely.

@engineerprompt Ай бұрын

Just checked, there were only 25 calls to the embedding API and about 570ish calls to GPT4o. So probably 90-95% is coming from GPT4o calls. That's during the index creation process. So this will be needed. In any case, I think this process runs only once to create the graph and then the retrieval part is pretty much like normal RAG. I will try to run the indexing with something like Haiku which will be much lower cost but hopefully will be good enough indexing. I agree with you. I think the LLM community need to look back some of the techniques built earlier for IR and reuse them. Will get much better performance out of these LLM based applications.

@unclecode Ай бұрын

@@engineerprompt Interesting. In that case, a viable production-level solution is to fine-tune a small model (~1B) specifically for generating knowledge graphs, with different flavors for topics like medicine, math, and business. This would create accurate knowledge graphs at a much lower cost. It's a great business opportunity. Honestly, I never thought of naive RAG as a whole pipeline! To answer complex questions, creating a knowledge graph is essential-perfect for a startup. Secondly, if inference is like normal RAG, that's good. But we need to dig into the GraphRAG library to see how much context it fetches from the knowledge graph and injects into the prompt. If it's too much, we'll have an issue. It’s about heuristically picking the minimal context needed to produce answers, so there's an optimization scenario here. Summary: use a small language model for generating a knowledge graph, and optimize how much context the GraphRAG library injects from the knowledge graph to answer questions. Please check the latter one if you find time.

@ilianos Ай бұрын

Regarding the cost implications (last part of the video): I think - a benchmark for different LLMs with GraphRAG and/or - some request router (but specialised for GraphRAG) would be suitable to optimize the right LLM for the right kind of request.

@engineerprompt Ай бұрын

Agree, I might do a video comparing different models when it comes to cost. Will be an interesting comparison

@u.a3 29 күн бұрын

Great video! When you mentioned the costs, was it the cost of implementing the graph rag and creating it, or does that also include the answers retrieved? If every answers costs that much then surely the price is too high. But if it is just a one-time high cost of creating the graph rag and the actual answers don't cost as much then I guess it is not really an issue.

@engineerprompt 29 күн бұрын

This was combined cost of graph rag creation and query part. The major portion is coming from the dataset creation part.

@u.a3 29 күн бұрын

@@engineerprompt clears it up, thank you!!

@sebastianmanns5391 Ай бұрын

Would be suuuuper nice if you could compare Microsoft versus LlamaIndex versus Neo4j implementations :)

@KristianSchou1 Ай бұрын

Have you experimented with a custom documentloader? - I'm working on a RAG system at work, and I've found that the prebuild loaders are severely lacking when it comes to handling text extractiong from pdf-files from "the real world". Would be nice to hear your thoughts

@IamMarcusTurner Ай бұрын

llama parse could from llama index could be a good choice. They noticed the same

@BACA01 Ай бұрын

You would need a RAG with local VLLM.

@bayu_sm Ай бұрын

Thanks for sharing, I like the explanation. Please compare this with Llama Index and Neo4j

@engineerprompt 29 күн бұрын

will do. thanks

@awakenwithoutcoffee Ай бұрын

question thought: can graphRAG work with multi-modal data ? I would actually think why not as a picture can be classified/linked to a specific node.

@engineerprompt Ай бұрын

I think it will be able to. Usually, for multimodal RAG, you use a vision model to generate descriptions for each image and preserve the links. Then you can embed the images descriptions just like you do for text

@awakenwithoutcoffee Ай бұрын

@@engineerprompt gotcha. How and where do you think would the images be stored/retrieved ? Let's say we attach a summary of a picture + metadata about the page/chapter/subject/id: maybe we could store the images in a separate database and retrieve it at runtime. Cheers!

@remyrflt 28 күн бұрын

Does it have to use OpenAI api key to make this work, or other OpenAI-compatible api (like from Perplexity or Together) will work too?

@engineerprompt 27 күн бұрын

This works on OpenAI or compatible APIs.

@GoldenkingGT101 Ай бұрын

Can it be done with ollama?

@CollinParan Ай бұрын

He said you can in the video, by changing the endpoint in your settings file.

@paulmiller591 Ай бұрын

Yes he did mention that was an option

@aa-xn5hc Ай бұрын

What about Ollama ?

@JumpDiffusion 17 күн бұрын

Nowhere in the paper (From local to global..) they mention use of an embedding model.

@lavamonkeymc Ай бұрын

How did u make the flowchart with sonnet?

@engineerprompt Ай бұрын

I provided the paper to it and then asked it to create visual representation of the proposed method. Usually, it takes a couple of iterations for it to do a good job

@MoadKISSAI 15 күн бұрын

Can you please make a comparison with Neo4j?

@ilianos Ай бұрын

One thing that I don't understand is: why is it limited to only 300 tokens? When you consider the current best models nearly all have 128k now...

@aaronhhill Ай бұрын

Yeah, Claude has a 200,000 token limit and Gemini how has a 2,000,000 token limit.

@engineerprompt Ай бұрын

During RAG, you have to chunk your documents into subdocs. The idea is that you only want to use part of the document that are relevant to the user query since each token costs money. You get to choose the chunk size. In the paper they used 300 but you can set it to any value you want. But larger chunks will incure more charges if you are using an api.

@jefframpe5075 Ай бұрын

Greg Kamradt @Dataindependent has a good video on 5 levels of text splitting for RAG, which explains the trade offs between different methods. kzfaq.info/get/bejne/brV6dpViuZi3fGg.htmlsi=Go7eAYu0kkL_exiv

@sriramkommi7684 21 күн бұрын

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte , bro im getting this error while running the indexing pipeline , can u please help me

@qicao7769 Ай бұрын

After see the cost caused by GraphRag, I am going to change the setting of Embadding and to my local LLM.... too expensive

@engineerprompt Ай бұрын

Agreed but you definitely want to watch my next video on the topic

@bigqueries Ай бұрын

pls compare all graph rag

@mogliff3414 29 күн бұрын

I am not able to see RAG beyong basic due to card . Need other platform say paid you tube etc

@engineerprompt 29 күн бұрын

Can you please clarify. Are you not able to signup because of the payment method? please email me and we can sort it out. Thanks

@svenandreas5947 Ай бұрын

ollama is not working for embedding

@engineerprompt Ай бұрын

were you able to use the LLM via Ollama? I think for embeddings there is not a standard API so that's why its not working.

@mrchongnoi Ай бұрын

Thank you for the video very good. Very expensive solution. It seems to me that this solution would not work for a company that has thousands of documents.

@engineerprompt Ай бұрын

Yes, in the current form it's going to be expensive. Now if you were to replace GPT4o with an open weights model or less expensive model, then it could be a different story IF they provide same level of accuracy

@malikrumi1206 Ай бұрын

Why have knowledge graphs become all the rage? How is that any better than a traditional database? Note that PosgreSQL already has a vector store extension, and since it has been out for a while now, I would assume all the others do, too. Since a number of others are requesting a head to head comparison with llamaindex, why not include a comparison with a traditional RDBMS? In one version, the model uses an SQL tool to find (read) the query answer in postgres, and then in the second, it looks for vectors in pg_vector?!

@MehdiAllahyari 27 күн бұрын

Great video for showing how graphrag works. However, as interesting as this graphrag approach may sound, it's not a practical one to use it in production. For a fairly short text/book (65 pages) it cost $7. Think if you want to process 100s of documents, it'll be prohibitively expensive!

@engineerprompt 27 күн бұрын

Agree, I think with time the cost will go down or other approaches will evolve over time based on this.

@MehdiAllahyari 27 күн бұрын

@@engineerpromptyes the cost is already very low and will go even lower. But even if price becomes very low, the overall price Wii be still expensive, because then people will upload more and more text and the same thing happens. For me graphrag really has nothing new. It’s simply a hype. There are better tools for extracting entities and relationships. We have a couple of videos about rag and KG in our channel. It’s called twosetai. Check it out. 😊

@fintech1378 Ай бұрын

cost will come down significantly in the near future, as long as it works then people should start initializing the workflow

@jayco10125 14 күн бұрын

Use the o-mini model

@engineerprompt 14 күн бұрын

would be a good option.

@j4cks0n94 Ай бұрын

That is SO expensive for just a book

@renandiniz2019 29 күн бұрын

Compare microsoft with neo4j

@marconeves9018 Ай бұрын

Are you not familiar with local hosting? Just wondering why you don't opt to showcase these tool integrations by going local instead of using paid APIs-- you closed the video with the premise of this being more expensive but thats greatly reduced if youre hosting it yourself.

@engineerprompt Ай бұрын

check out other videos on my channel :)