CHATGPT For WEBSITES: Custom ChatBOT: LangChain Tutorial

No video

CHATGPT For WEBSITES: Custom ChatBOT: LangChain Tutorial

Рет қаралды 100,308

Күн бұрын

In this video, we will create a chatbot based on GPT-3 powered Natural Question Answering BOT on any Website using LangChain. We will combine the concepts of Embedding, Information Retrieval and VectorStore to create a powerful information retrieval system for your website.
LINKS:
Google Colab: colab.research...
OpenAI API Pricing: openai.com/pri...
OpenAI Models: platform.opena...
FAISS: github.com/fac...
Links to Videos:
PDF Files: • ChatGPT for YOUR OWN P...
Open Tools: • Talk to YOUR DATA with...
-------------------------------------------------
☕ Buy me a Coffee: ko-fi.com/prom...
Join the Patreon: patreon.com/PromptEngineering
-------------------------------------------------
Langchain is a powerful open-source Python (and Javascript) framework that allows you to build applications with LLMs and text embeddings. With Langchain, you can easily train GPT on your own data and create a personalized LLM that can answer questions and generate text.
In this tutorial, we will walk you through the process of building an application with Langchain using Streamlit. We will dive into Langchain and its features, including its ability to train GPT on documents and text embeddings.
We will show you how to create a Text Embedding model for your application using Langchain and how to integrate it into your project. You will also learn how to train GPT on PDF documents and fine-tune it to your specific use case.
Throughout the tutorial, we will be building a fully functional application that allows you to upload a PDF file, ask questions about it directly, and have an LLM answer for you. We will use the powerful capabilities of Langchain to create a seamless user experience and showcase the framework's many possibilities.
Langchain is a versatile framework that can be used in many applications, from chatbots to document analysis. We will show you how to leverage Langchain to create powerful applications with Streamlit that can help automate tasks and improve efficiency.
If you're looking to take your natural language processing skills to the next level, this tutorial is a must-watch. Learn how to build powerful applications with Langchain and chat with GPT about your PDF files. So, what are you waiting for? Let's get started with Langchain!
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

Пікірлер: 188

@engineerprompt Жыл бұрын

Want to connect? 💼Consulting: calendly.com/engineerprompt/consulting-call 🦾 Discord: discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: ko-fi.com/promptengineering |🔴 Join Patreon: Patreon.com/PromptEngineering ▶ Subscribe: www.youtube.com/@engineerprompt?sub_confirmation=1

@saltinejones Жыл бұрын

This was a game-changer for me. I'm pretty new to langchain and was having a hard time wrapping my head around it, but your video made everything click. Thanks for sharing your wisdom with the rest of us!

@engineerprompt Жыл бұрын

Glad it's helpful. Thank you 😊

@biraescudero Жыл бұрын

My friend, I watch most of your videos and I am learning a lot. Your explanations are f******* CLEAR and you show complex things in a way that any one (even I) can understand without any doubt!!! Thanks a lot for your work, your videos and patience! Go on! You make the difference!

@engineerprompt Жыл бұрын

Thanks :)

@roua. Жыл бұрын

Great video, I really appreciate that you kept going back to the diagram while writing the code.

@lovol2 Жыл бұрын

Probably the most clear explanation of this. Thanks. Next up, some UI for it 😊

@engineerprompt Жыл бұрын

Soon :)

@victor-balta 10 ай бұрын

I was searching or hours for a video like this. Thanks! It's pretty straightforward, and your diagram helps a lot in understanding the concepts

@lucarestagno Жыл бұрын

This video explanation is just great. I was switching between the websites of different products, like LangChain and QDrant, but I was not able to get the overall picture of such a solution. Thanks for the explanation! It's crystal clear!

@merneadanfoundation6775 Жыл бұрын

Wow, what a clear and concise explanation!!! Thanks for this and keep on giving us this valuable information.

@automanai Жыл бұрын

It is truly amazing. Greetings from China.

@Luca-tw9fk Жыл бұрын

Cool. Would be fun to make a website where the nav is entirely handled through conversation with Chatbot

@JasonFowler74 Жыл бұрын

Thank you for sharing with the world. Your work is packed with information and remains accessible.

@engineerprompt Жыл бұрын

Thanks :)

@buntus_ 16 күн бұрын

thank you so much for this video brother. I learnt a lot.

@underdog578 10 ай бұрын

Great video, a good follow-up might include how to use agents to enhance the chatbot for sites that also have dynamic content

@joser100 Жыл бұрын

Very good content as usual, two points in case you can comment or consider for the future: 1. Just on this context of web-sites (there is many more data to consider), there is normally plenty of info in web-sites that is provided as non-text, think of diagrams or screenshots, etc, in fact some websites are strongly supported by this type of "image" support, I guess we will need multi-modal support for this at some point in time (you can also have links to documents or other type of data of course) , so asking in case there is anything that could help with this part, for a more comprehensive/relevant scraping of sites. 2. Then the point with secured areas of web-sites, and with this the fact that users may have rights to access parts of a site (or not), think of Confluence web documents that are in private domains with a complex set of access rights, any recommendations as to how to possibly handle this type of data? I guess what I am saying is complex, since we would need to implement Auth access management into what embeddings can be accessed, etc

@engineerprompt Жыл бұрын

Thanks for kind words. 1. You are right, most of the LLMs focus on text. There are a few models that have the ability to understand images (think Llava, GPT4-mini, I have videos on both) but at the moment they are computational infeasible to run as part of such a system. 2. Yes, for the propriety dataset, you will need authentication mechanisms. I believe there are some packages which let's you do that.

@tarun4705 Жыл бұрын

You're awesome. This is so helpful.

@engineerprompt Жыл бұрын

Thanks :)

@mallikarjunmongolla4519 6 ай бұрын

Hi, Thank you for this video. Your explanation is great and really helpful in understanding the concepts.

@pranaymohite7777 Жыл бұрын

Awesome explanation, very easy to understand. Thank you

@engineerprompt Жыл бұрын

Glad it was helpful!

@shaifulislam7592 Жыл бұрын

That's really a knowledgable content.

@blackwhalestudio Жыл бұрын

i had to install a lot of other libraries such as: #!pip install unstructured #!pip install pdf2image #!pip install pytesseract #!pip install tiktoken also the very last step when asking a question doesnt work for me. it says InvalidRequestError: you must provide a model parameter could you please help me to resolve this? thank you very much for your video!

@viniciuswm Жыл бұрын

Thank you for this great content. One suggestion is to compare the FAISS with other vector db solutions like Pinecone, etc.

@engineerprompt Жыл бұрын

Interesting idea, will do :)

@diamond2869 7 ай бұрын

very well explained. Thank you!

@MikeBtraveling Жыл бұрын

Love your videos, keep up the great work!

@mayank1334 Жыл бұрын

Amazing tutorial! I wanted to ask, do you also have a video where you build a Chat UI (in Python)

@saikarthik6394 Жыл бұрын

@engineerprompt Please link if you have a video on this. Thanks.

@LibertyRecordsFree Жыл бұрын

Best teacher ever :) Just to make sure I understood, these embeddings with langchain allow to retrieve the most relevant website parts to be pasted as "context" with the question for the chatbot to answer, right?

@engineerprompt Жыл бұрын

Glad you found it helpful. Yes, the chatbot generates answers just by looking at the most relevant parts and doesn't see the rest.

@yalmeme 11 ай бұрын

Can you provide link to excalidraw sheme please?

@kenkioqqo Жыл бұрын

Splendid! Very well explained.

@engineerprompt Жыл бұрын

Glad it was helpful!

@Larimuss Ай бұрын

Thanks great tutorial. Can you do some basic python ai setup tutorials explaining the functions?

@aeonfoxie1260 4 ай бұрын

I really really enjoyed your video tutorials. Insightful and no bullshit. And I like your explanation with diagram . Could you share these diagram too? 🥰

@engineerprompt 4 ай бұрын

Thank you, it will be part of an upcoming course.

@aeonfoxie1260 4 ай бұрын

@@engineerprompt, Thank you. Will be waiting.

@carterjames199 Жыл бұрын

Also what are the differences between openAi embedding models and other open source embedding models? Is it just faster or will I get completely different results?

@LocalizationAcademy Жыл бұрын

Great explanation, thank you!

@khalidbouziane4005 Жыл бұрын

OpenAI Chatgpt is able to keep memory of the previous questions, in this tutorial u have used one of their APIs, my question is did u try to check if the chatbot that u created using langchain is able to keep track of your previous questions and it taked that in consideration while answering a new one.?

@engineerprompt Жыл бұрын

Yes, you can implement that as part of the langchain. For this specific example I didn't cover it but will be doing it in new videos.

@DJPapzin Жыл бұрын

You explain very well

@aqashatehreem7206 Жыл бұрын

Great video. I have been working with langchain for some time now but still this video helped me a lot to fully understand the details. Just one question though! How can we make this work if the data on the website is constantly changing? For example, in the case of an e-commerce website. Do you have any video on that too?

@wilfredomartel7781 Жыл бұрын

Looks great

@rishabh1692 Жыл бұрын

I need to login into the website before getting access to all the urls documents. How do I achieve that using Selenium URL Loader? Or any other way?

@berlinisaiah6892 Жыл бұрын

Thanks for sharing

@reggiechandra9592 4 ай бұрын

Thank you! Your instruction is very clear and great. Can you please share the code?

@alainkaiser2697 Жыл бұрын

this is an amazing video!

@himanshushrivastava7287 Жыл бұрын

Amazing tutorial, Thank you! By following your tutorial, I was able to develop QA bot that can provide answers from a given document. However, this chatbot is limited to the document’s information only. I want to create a chatbot that can integrate its own data with the document’s data. How can I achieve this? Please help!

@arpitjain2809 Жыл бұрын

How will the implementation shared in Google colab change when new contents appear on the website (like a blog). The entire embedding (training data) needs to be run again and the pickle file regenerated. Is that correct? What approach is better if I want to make a model for changing data (like a blog)

@user-vh3vv6vt6u Жыл бұрын

would you please give us link to the diagram you use for this tutorial

@saatwikkaul3585 Жыл бұрын

I'm having a little trouble, at the very last step when the user asks a question I get the error : InvalidRequestError: This model's maximum context length is 2049 tokens, however you requested 2893 tokens (2637 in your prompt; 256 for the completion). Please reduce your prompt; or completion length. How can I bypass this?

@AIforAll-DOMANA 7 ай бұрын

great, thank u

@user-rc6ik9gz6g Жыл бұрын

Thank you, very useful video. How remember the previous chats and consider the new response?

@rashidnizamuddin3700 Жыл бұрын

nice video Thanks so much can you please also create a video on how to host this on an environment and not local so that other people can query it

@user-zj9pq5xc7x 2 ай бұрын

good content. code needs some changes due to updation, but great explanation

@engineerprompt 2 ай бұрын

thanks, its a really old video, probably need to start creating newer versions of the same :)

@atenkhu6092 Жыл бұрын

Hope you do an update video in the future were the ChatBot is actually on a live website answering questions. Or Can you please provide hints on how someone can embed the chatbot let's say on a wordpress website?

@godfreyogbeide2340 7 ай бұрын

Hi, good tutorials

@jesusperdomo8388 Жыл бұрын

I really appreciate the tutorial but it doesn't help much doing them in Colab. It would be much more valuable if you did them in VSC or some text editor installing the packages. For example UnstructuredURLLoader doesn't work at all, there is a "libmagic is unavailable but assists in filetype detection on file-like objects" error that no one seems to have been able to fix. I appreciate if someone has the solution.

@oxytic Жыл бұрын

Hi bro , kindly do a same thing in open source model kindly do the needful.

@DonJuanchito Жыл бұрын

Thank you for your time in this video. Could you clarify the following for us?: with this set up, does the model look for answers both using Gpt-3.5 general knowledge (same as chatgpt) AND documents from langchain? Or only documents from langchain and not general chatgpt knowledge?

@engineerprompt Жыл бұрын

With this, it will primarily use the provided information for answering along with ChatGPT training data. You can limit that by providing a prompt that asks the model to use the provided context (information). Hope this helps

@ubaisalih2987 7 ай бұрын

it's amazing but something is missing , as this should be a chatbot for a Website , so how to deploy this on a website ?

@sandeepsaha Жыл бұрын

Great video. Would be amazing if you add how one can create a UI for this chat bot

@engineerprompt Жыл бұрын

Check this out for some ideas: kzfaq.info/get/bejne/iK-HlcSdv53XhKM.html

@debasispanda1764 Жыл бұрын

This is a great content .. Thank you. Can you please create a detailed contents regarding execution of BI level queries on top of a Pandas/Spark dataframe using embeddings and models locally (no OpenAI I mean)

@engineerprompt Жыл бұрын

thanks, yeah will be covering this in future.

@bwilliams060 Жыл бұрын

This was good, but the thing I wanted to learn is how to use the site map to feed all of the urls for a website into a document loader using code? Can you provide a code snippet by chance? Thanks!

@yazanrisheh5127 Жыл бұрын

I keep getting this error despite creating a brand new OpenAI account. I'm trying to create a chatbot that can read from a pdf file. How do I fix this error: "You exceeded your current quota, please check your plan and billing details" There's no way I can exceed if I literally just created my account...

@engineerprompt Жыл бұрын

Most probably you don't have money in your account.

@telecomahmedk1 Жыл бұрын

Thanks for your efforts If you can tell me how use openai embedding with gpt4all Please share with me any guide for this

@engineerprompt Жыл бұрын

will be realizing a video on it soon.

@jayhu6075 Жыл бұрын

I like this kind of tutorial with a flowchart to following the steps easily. I saw you use vs code with a Jupyter Note plugin? I this approach better or can I use Jupyter notebook separately ? Hopefully more from this stuff. Many thanks

@LibertyRecordsFree Жыл бұрын

Also interested how you did this notebook style in vscode

@engineerprompt Жыл бұрын

VScode has a plugin for notebook. I usually use notebooks for experimentation but for production, I use scripts. I personally find it helpful to have everything in a single place. But ultimately its just personal choice :)

@engineerprompt Жыл бұрын

there is a plugin for it. You can search for it.

@jayhu6075 Жыл бұрын

@@engineerprompt Thx for the feedback.

@stephenngari775 Жыл бұрын

how would you go about hooking this app to a UI?

@TylerKlug Жыл бұрын

Also curious about this, please!

@engineerprompt Жыл бұрын

soon :)

@mediahawkai Жыл бұрын

Awesome video! However, I am curious as to how to store the conversation history. So that the user can ask follow up questions about the previous response.

@engineerprompt Жыл бұрын

Check this out: kzfaq.info/get/bejne/mt5_adOr0c2umY0.html

@EsenEspinosa 6 ай бұрын

Is there a bug in the vicuña response? The new line characters are inside the source url. ¿Is that avoidable or that output is generated by the llm ?

@BosnianDeveloper Жыл бұрын

Great video but I have a question for you sir.. Can you integrate chat GPT into a website that already has chat systems between users and then integrate chat GPT into one fake profile to act like a chatter and train it to chat like normal person ?

@FCrobot 11 ай бұрын

In the scenario of conversational robots, how to limit the token consumption of the entire conversation? For example, once the consumption reaches 1,000, it will prompt that the tokens for this conversation have been used up.

@SarthakPan 10 ай бұрын

The only problem is that it also gives responses to out of context questions too. Suppose I have a financial firm but someone come and ask what are the upcoming movies in 2023? It will answer that too. How to prevent it?

@Movies.Clips.T Жыл бұрын

Greetings sir, I am really inspired by you thanks for your work, I would like to know that how can I use vicuna, alpaca or Llama instead of Openai to reduce the cos. Really appreciating your efforts my mentor :)

@roberthuff3122 Жыл бұрын

Excellent! Will you explore one or more possible web UIs like Gradio or Streamlt?

@engineerprompt Жыл бұрын

Yes, working on a video, will be making a whole series. :-)

@rohit_700 11 ай бұрын

I have a very simple question. TO use Open ai embiddings model lets say ada-002 do we need subscription?

@Visitirio Жыл бұрын

Thanks alot for sharing this video with us... But there is a problem that the default model is working but gpt-3.5-turbo is not working... Any solution for this... please guide... I'll appreciate that...

@projectsbyryan 9 ай бұрын

How is it different, now that we can customize ChatGPTs?

@RajanKumar-yy2gn 4 ай бұрын

will this langchain library will also work for dynamic webpages?

@abhaybisht8901 Жыл бұрын

Thankyou so much

@ZeeNoorTrip 8 ай бұрын

Hi thanks for the video, just wondering why it doesn't support If I create this code for python environment?

@senethys Жыл бұрын

What about all the other sources that are in the website such as documents and pdfs? Is it just plain text we are talking about here?

@user-fg4ht4fx3d 9 ай бұрын

Many thanks, what about using Redis do you have code.

@binitkunal4627 Жыл бұрын

i am geeting error [Document(page_content='Access Denied You don\'t have permission to access i have used web based loader no error is comeing but useing UnstructuredURLLoader getting error . can please difference b/w web based loader and UnstructuredURLLoader

@raviadi295 9 ай бұрын

When i run this code, the "source" is showing as " " (empty string). Can someone let me know why the source url is not showing?

@mehdiyedroudj4301 Жыл бұрын

Great work dude. Is it possible to use use Vicuna or other instead of openIA ? Thank you

@engineerprompt Жыл бұрын

Yes you can! check this out: kzfaq.info/get/bejne/g9KporKTya-sqaM.html

@rehmozayub9442 8 ай бұрын

Do you have a tutorial on how to take this model online for a Wordpress website?

@user-fm5yy2sx2p Жыл бұрын

Hi, how to use LangChain in multiple sources pdfs, txt, web pages, JSON all this in one knowledge base? Will you create that video tutorial?

@helbertrodriguez6449 Жыл бұрын

Is it possible to have a chatbot like Chatgpt in a website that was trained and ready to answer any question the person might have

@ozzy1987mr Жыл бұрын

seria bueno que comiences a aplicarle la api de ingenieria inversa para abaratar costos, ya que utiliza el modelo gratuito de OpenAI

@ABHISHEKSHARMA-fo4tf Жыл бұрын

Excellent video . this video where i learn so much about langchain. i have question that i have make chatbot for house searching and i store dataset data into pinecone . data like location, bhk, sqft, price now our chatbot is only answering the question whatever he asked but client want the if we cannot provide such information like we cannot provide location or bhk, then bot will ask question about that. can you please tell how i can make this type of chabot?.

@engineerprompt Жыл бұрын

Thank you, I think you will need to create two layers, the first one check for any missing information (with memory) and asks user for it. Then you can have this second layer which takes the updated input and use it for the information retrieval.

@geneoverride3725 Жыл бұрын

Discord would be great.

@fACEB0T Жыл бұрын

Talkin to me?

@geneoverride3725 Жыл бұрын

@@fACEB0T Why do you think so?

@engineerprompt Жыл бұрын

I finally found time to set it up. Still learning the ins and out of it. Hopefully will be sharing this week. Thanks 😊

@geneoverride3725 Жыл бұрын

@@engineerprompt Dont worry, I can help.

@WarongkornTritipakit Жыл бұрын

Thanks!

@agkol92 10 ай бұрын

Hey Hi awesome video. I have very basic coding skills and was using the colab file you have provided. However when running the chain command I am getting an error " f"OpenAIChat currently only supports single prompt, got {prompts}" CAn you please help

@user-rc6ik9gz6g Жыл бұрын

How to remove unwanted data from vector file using pattern matching or any other method.

@eduardoruesta5180 Жыл бұрын

Excellent video!! i have to create a chatbot that only answer questions about the data from a specific client, for example store location, hours, stock, etc. I need to give the bot a kind of personality, not only answer questions, but also respond with a certain "charisma", how can I do that? I thank you for all the information

@engineerprompt Жыл бұрын

Look at adding a system prompt. That way, you will be able to give it a persona.

@eduardoruesta5180 Жыл бұрын

@@engineerprompt thanks for that!! do you know where i can learn how to adding a system prompt? Thanks again!!

@zainabfahim8714 10 ай бұрын

do you have a video where the same task can be done, but instead of OpenAI models, using Falcon?

@engineerprompt 10 ай бұрын

Check out my localgpt videos

@MugilaSADA 4 ай бұрын

how to do web scraping chatbot from URL

@user-vq7gz6gx1x Жыл бұрын

Is it possible with any opensource LLM for the the same?

@chrisshort3328 Жыл бұрын

would b good if u included ur ipynb file as well

@aischool0912 Жыл бұрын

Can we build a hotel booking bot using langchain framework with LLM?

@rehatvirk1807 Жыл бұрын

module not found langchain. How do i solve this

@xflory26x Жыл бұрын

Thanks for the video, I've been trying it with many other websites but I keep getting an error: exeption: Invalid file. The FileType.UNK file type is not supported in partition, do you know whether it only works on certain websites?

@engineerprompt Жыл бұрын

This will work with websites that mostly have text.

@kwamenadadson Жыл бұрын

HI is there a way I can use my own website?

@brajmohanTikarya Жыл бұрын

One suggestion: Use light theme for teaching purpose. White gives more clarity and visibility. You can compare both(black and white) and validate.

@geneoverride3725 Жыл бұрын

No. Light theme sucks. Dark is pretty good.

@lovol2 Жыл бұрын

I prefer dark

@geneoverride3725 Жыл бұрын

@@lovol2 light hurts the eye...eeew

@fliu5282 Жыл бұрын

No. Here is the problem, a lot of Chatbot I can't customize the Avatar. I can't customize the "look" on the lower right "avatar" when launched, I can't customize anything except data input. This is sucks. I try all 10 diff Chatbots none can do what I want to do and look and feel...all chatbot looks "stupid" i cant to have my own look with a message so visitor will click.Plus the chatbot reply I want to emulate OPEN Ai in a slow motion and not a photo graphic snap shot reply. It does not emulate human motion. Anyways...

@carterjames199 Жыл бұрын

What if you want the model to return paragraphs as source material instead of the urls? Is there a way to make it return the specific text chunks that is got it’s answer from?

@engineerprompt Жыл бұрын

Yes, you can easily do that. You will have to use a different qa_chian in langchain. Will cover this in another video.

@magalhaesbrunop 9 ай бұрын

My code is given this error AttributeError: module 'openai' has no attribute 'Embedding' Some one could help me? I already tried Python versions 3.10 and 3.11

@engineerprompt 9 ай бұрын

I think the LangChain implementation has changed. I will try to rectreate this with updated code

@arpitjain2809 Жыл бұрын

The code sample you shared doesn't work. Please check and see if you can update Colab notebook.

@engineerprompt Жыл бұрын

What's the error? I runs on my end. You will need to provide your own API key.