No video

LlamaIndex overview & use cases | LangChain integration

  Рет қаралды 24,167

Sophia Yang

Sophia Yang

Күн бұрын

Colab: colab.research...
Blog post: sophiamyang.me...
00:00 intro
01:04 LlamaHub
02:32 basic query functionalities
03:31 document management
04:38 query multiple documents
06:26 Router query engine
08:37 Hypothetical document embeddings
10:12 use LlamaIndex with LangChain
🔔 SUBSCRIBE to my channel: www.youtube.co...
⭐ Stay in touch ⭐
📚 DS/ML Book Club: dsbookclub.gith...
▶ KZfaq: / sophiayangds
✍️ Medium: / sophiamyang
🐦 Twitter: / sophiamyang
🤝 Linkedin: / sophiamyang
💚 #AI

Пікірлер: 58
@ronaldokun
@ronaldokun Жыл бұрын
What a delightful combination of knowledge, objectivity and calm energy. Thank you!
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thanks so much 🙏😊
@matten_zero
@matten_zero 10 ай бұрын
Llama Index for long term memory and Lang Chain for chaining and agents. This all happened within a year or so. What a time to be alive!
@SophiaYangDS
@SophiaYangDS 10 ай бұрын
Yeah both LlamaIndex and LangChain are great. I like to use them together as well 😄
@SimonMariusGalyan
@SimonMariusGalyan 3 ай бұрын
Thank you for the fantastic overview of LlamaIndex functionalities 😊
@matten_zero
@matten_zero 10 ай бұрын
I like how you added additional info in the yellow text overlays. That helps with retaining info. Great job 👏
@SophiaYangDS
@SophiaYangDS 10 ай бұрын
Thanks so much 🙏
@wiktorwysocki516
@wiktorwysocki516 4 ай бұрын
Great you mention that those tools are not exclusive. However probably for simpler use cases one of them will be enough.
@8eck
@8eck Жыл бұрын
Too bad that i can't like twice your videos! Thank you, very informative videos. 🙏
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thanks so much for the kind comment 🙏😊
@taeyangoh7305
@taeyangoh7305 Жыл бұрын
Thank you so much, Sophia!
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thank you 🙏
@ahmedmusawir
@ahmedmusawir 7 ай бұрын
Great content! Thanx ... plz give us more on combining Lamaindex with Langchain ... thanx in advance
@kevon217
@kevon217 Жыл бұрын
Love your videos. Always on my queue! Thanks for sharing!
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thanks 🙏
@jerryoverton7037
@jerryoverton7037 6 ай бұрын
Really useful stuff! Thanks for making this video.
@talhayousuf4599
@talhayousuf4599 11 ай бұрын
Another Informative video, didn't know about this.
@SophiaYangDS
@SophiaYangDS 11 ай бұрын
Thank you 🙏
@8eck
@8eck Жыл бұрын
Please create a breakdown video about different kind of models, chat models, instruction models etc.
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Great ideas thanks 🙏
@kuzeyk5532
@kuzeyk5532 11 ай бұрын
Great tutorial, thank you!
@SophiaYangDS
@SophiaYangDS 11 ай бұрын
Thank you 😊🙏
@sabashioyaki6227
@sabashioyaki6227 Жыл бұрын
Excellent!
@VaibhavPatil-rx7pc
@VaibhavPatil-rx7pc Жыл бұрын
Excellent !!!
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thanks 🙏
@foysalmamun5106
@foysalmamun5106 Жыл бұрын
Thank you lot
@henkhbit5748
@henkhbit5748 Жыл бұрын
Thanks for explaining llmaindex. 👍 Is it also a vectorstore like chromadb?
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Yes it has many vectorstore options as well just like LangChain
@nobody2937
@nobody2937 Жыл бұрын
You look gorgeous 👍👍👍 And very clear and detailed explaination... Thank you ...🌹🌹🌹
@mzty999
@mzty999 7 ай бұрын
try to use llamaindex and huggingface model, but all kinds of issues. It seems model kwargs cannot be passed to hf transformer model?
@happyday.mjohnson
@happyday.mjohnson Жыл бұрын
I used Langchain's dataloader and then converted into a llamaindex. I had to do this because the llamaindex beautifulsoup scraper would return 403's because it wouldn't let me enter the user header. The challenge I am having with both SDKs is they are throwing so much good stuff out but unfortunately the testing seems lacking. YIPPEE!! My question: What if I want to make a local podcast of stuff happening in my town that gets buried. Some things in the town might come from a Wikipedia entry, some will come from our town web site buried in council minutes, etc. some will come from police reports, more investigative. For quality results should everything be indexed? Or when does one just rely on "Do a web search?" Also, what are good methods for fact searching?Funny when I ask why it came up with an answer. ChatGPT 4 told me "Because it makes a good story" That was interesting and made sense that it would do that. But I'd like to have a better handle on fact and fiction. Thank you.
@felipejaramillo124
@felipejaramillo124 Жыл бұрын
Interesting story. You mention you converted to LLama index but then say you had issues with Llamaindex beautiful soup scraper returning 403's. Maybe you were referring to Langchain's issues before you made the switch?
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Great use cases. Thanks for sharing! Sounds like the things you are looking at might not show up in web searches? That's why you got them from sources? Indexing everything you care makes sense to me. And yeah fact checking is hard. I thought LangChain has a fact checking chain, maybe try that. Or just ask LLM if the answers are facts.
@geraldofrancisco5206
@geraldofrancisco5206 Жыл бұрын
clicked fast
@ProfessorMuwaffaq
@ProfessorMuwaffaq Жыл бұрын
Very nice!!. thank you but can you make more videos about how to structure the data in the documents. and how can we tell llama about the data structure in the documents. for example to tell them that the document have Q&A template structure separated by (----) and if you do a video about hoe tow to make the result better. this would be Awesome. Thank you and good luck!!!!!
@juanjesusizquierdo2464
@juanjesusizquierdo2464 Жыл бұрын
Great tutorial, thank you very much! My question is, how can we persist the indices and load them from a storage directory, in the Query multiple documents example? I could not find any example in the llama-index documentation about persisting several indices (from several documents) and load them individually to replicate the SubQuestionQueryEngine example.
@jamesxiao4996
@jamesxiao4996 Жыл бұрын
Hi Sophia, my team and I found this video very helpful during Berkeley's AI/LLM Hackathon last week :D I'm a rising junior studying CS, and most of my experience thus far has been all sorts of programming NOT related to AI/ML/DS. However, I'm now really interested in experimenting through building meaningful tools/apps on top of the ML layer through frameworks such as LlamaIndex, Pinecone etc. I just wanted to ask if you think its necessary to have a strong understanding in AI ML fundamentals before moving forward with these tools. Thanks
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thank you so much 🙏😊 it's amazing you participated in the Berkeley Hackathon. Great opportunities there. All these tools are so new and higher level extracted away from the AI theories. If you'd like to work in the application layer, go for it. You will do great 👍
@user-vz7rn8lb4i
@user-vz7rn8lb4i Жыл бұрын
Good video '
@SophiaYangDS
@SophiaYangDS Жыл бұрын
Thank you 🙏
@tiagomarante7720
@tiagomarante7720 8 ай бұрын
Hi Sophia, can I use langchain to load models from TogetherAI or other API LLMs, and then use that loaded model with LlamaIndex?
@SophiaYangDS
@SophiaYangDS 8 ай бұрын
I think so. it's a good idea to use llamaindex and LangChain together
@TroxerLiveCops
@TroxerLiveCops Жыл бұрын
Hey Sophia, when I tried to query across multiple documents I get this error: "InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a " Do you know why this happens?
@shrey6867
@shrey6867 10 ай бұрын
Hey, is there any way you can query mongodb collection? like something similar to langchain sqlchains
@tantomanontroppo8582
@tantomanontroppo8582 Жыл бұрын
hi, how can i do in context learning from a txt file and lengchain but using falcon 7b parameter instead of open ai? i try to query a txt file with lengchain but not using openai as model interpreter but using falcon 7b
@user-wr4yl7tx3w
@user-wr4yl7tx3w Жыл бұрын
How is it different from LangChain?
@SophiaYangDS
@SophiaYangDS Жыл бұрын
I mentioned their differences at the end of my video and blog post (linked in the description)
@anuvratshukla7061
@anuvratshukla7061 10 ай бұрын
Thanks, I have a python query engine in llama index. I want that engine as a tool in langchain, can you please make a video or let me know here?
@matten_zero
@matten_zero 10 ай бұрын
Personally i perfer Llama Index for memory as i like the syntax better. I think I like building my own agents from scratch but seems like LangChain is best game in town for agents.
@nilewalker7240
@nilewalker7240 7 ай бұрын
I used to think that too, but then I tried developing an application where I needed agents that switched to different prompt templates, depending on the query , and also whenever the application accident I needed to manually clear my ram cache super dumb issues. Then I switched over to haystack and I was able to rebuild everything I had with Lang chain and got a lot more organization and logging control within a few hours.
@gamersworld98
@gamersworld98 Жыл бұрын
2:38 Not able to use it withiu openai api key for basic query functionality Is this flow of?
@SophiaYangDS
@SophiaYangDS Жыл бұрын
You don't need to use OpenAI models. There are various LLM providers you can use and even local models: gpt-index.readthedocs.io/en/latest/how_to/customization/custom_llms.html
@paultoensing3126
@paultoensing3126 3 ай бұрын
How could this ever be applied to creating a RAG for a customized ChatGPT? Closer to the point, as something with a user interface that my grandmother could use? Obviously the average user simply isn’t going to run anything that has code on the page.. they’ll need a slick UI. Something like the OpenAI Custom GPT. How does that slick UI ever become integrated with what you’re doing?
@user-wr4yl7tx3w
@user-wr4yl7tx3w Жыл бұрын
Is LangChain and Llama Index primarily used by data scientist or is there a new job role that uses them? Or is it machine learning engineers?
@SophiaYangDS
@SophiaYangDS Жыл бұрын
They are for creating LLM applications. I don't think roles matter since the same roles might have completely different responsibilities at different places
@ikjb8561
@ikjb8561 Жыл бұрын
This would have been even better if we did not need to use openai
@kylesims7790
@kylesims7790 Жыл бұрын
Promo-SM 😪
How LlamaIndex Brings Data to LLMs
27:09
Arize AI
Рет қаралды 27 М.
No empty
00:35
Mamasoboliha
Рет қаралды 12 МЛН
Gli occhiali da sole non mi hanno coperto! 😎
00:13
Senza Limiti
Рет қаралды 4,6 МЛН
黑天使遇到什么了?#short #angel #clown
00:34
Super Beauty team
Рет қаралды 38 МЛН
LangChain vs. LlamaIndex - What Framework to use for RAG?
16:51
Coding Crash Courses
Рет қаралды 15 М.
LlamaIndex for Dummies: Choosing The Right Index For Your Use Case
6:43
Introduction to LlamaIndex with Python (2024)
39:57
Alejandro AO - Software & Ai
Рет қаралды 10 М.
I Analyzed My Finance With Local LLMs
17:51
Thu Vu data analytics
Рет қаралды 464 М.
Understanding Embeddings in RAG and How to use them - Llama-Index
16:19
Prompt Engineering
Рет қаралды 35 М.
Langchain vs LlamaIndex vs OpenAI GPTs: Which one should you use?
9:00
What's AI by Louis-François Bouchard
Рет қаралды 16 М.
ADVANCED Python AI Agent Tutorial - Using RAG
40:59
Tech With Tim
Рет қаралды 136 М.
LlamaIndex Sessions: 12 RAG Pain Points and Solutions
37:57
LlamaIndex
Рет қаралды 13 М.
Talk to Your Documents, Powered by Llama-Index
17:32
Prompt Engineering
Рет қаралды 81 М.