Analysing Chatflows using LangSmith - FlowiseAI Tutorial #7

Рет қаралды 7,333

Күн бұрын

#flowiseai #flowise #openai #langchain
Observability in LLM application is a critical component to creating production-ready applications. We can use LangSmith to analyse every step in the chain.
🙏 Support My Channel:
Buy me a coffee ☕ : www.buymeacoffee.com/leonvanzyl
📑 Useful Links:
LangSmith: www.langchain.com/langsmith
💬 Chat with Like-Minded Individuals on Discord:
/ discord
🧠 I can build your chatbots for you!
www.cognaitiv.ai
🕒 TIMESTAMPS:
00:00 - Introduction
01:10 - Create Langsmith account
01:47 - Adding Langsmith to Flowise
02:50 - LangSmith API Key
03:40 - Viewing Langsmith traces
05:19 - Output Parser example
07:14 - Metadata tab
07:29 - Conversation with memory
11:32 - RAG example

Пікірлер: 57

@leonvanzyl Ай бұрын

The Langsmith option has moved. It can now be found under Settings -> Configuration -> Analyse Chatflow.

@gautammandewalker8935 4 ай бұрын

Amazing! More detailed video on Lang smith and other options will be great.

@leongodwin69 4 ай бұрын

This is so good. I love your detailed, well paced, and easy to understand instructions. I hope this series goes on for dozens more episodes, getting deeper and deeper, exploring more and more use cases with even greater integrations. I want to use the skills you are helping me develop in business scenarios

@leonvanzyl 4 ай бұрын

Thank you very much for the compliments! The intention is to create many more episodes.

@ahmedkhalidak4515 4 ай бұрын

Thank you Leon. I think that last use case should help with adjusting the Chunks values for the Splitters

@leonvanzyl 4 ай бұрын

You're 💯 correct!

@JoaquinTorroba 4 ай бұрын

Thanks Leon! Amazing content.

@leonvanzyl 4 ай бұрын

Thank you!

@ward_jl 4 ай бұрын

Leon, are you planning on doing a video on LangGraph as well? Developing complex agents using LangChain, LangSmith, and LangGraph would be very interesting!

@leonvanzyl 4 ай бұрын

Absolutely!

@RolandoLopezNieto 3 ай бұрын

Interesting request

@iokinpardoitxaso8836 4 ай бұрын

Amazing video-tutorial as always ❤. Maybe you could go deeper on Datasets and How to launch-execute Evaluations? On new video Langsmith - part 2? 😊 That would be much apreciated.

@leonvanzyl 4 ай бұрын

You're right. There is so much more to cover on Langsmith. I'll work on a separate Langsmith series perhaps.

@mohamedgabr9954 4 ай бұрын

Thank you. Great work . Can you please make another detailed video about evaluating LLM response using Langsmith? And how we can enable the chatbot users to give feedback about the response in the resulting chatbot? And how to make this feedback using 3rd party apps .... For example python applications?

@lancerkind 4 ай бұрын

Brilliant work! I am in need of this detail. Are there other things we should know about debugging with Flowise? Such as log files? Or do you think that langsmith tyoe tools provide all that’s needed.

@leonvanzyl 4 ай бұрын

Thank you 👍. You can view the console to see Flowise related messages, like exceptions. Langsmith is my preferred tool for analysing the chatflows.

@Trashpanda_404 2 ай бұрын

Using DigitalOcean, would you use Docker to build Flowise, Lamma 3, and then LangSmith? Flowise already uses LangChain, so no need for that? Then in another docker you could run LangGraph, and maybe another for ActivePieces? Now it's getting complicated. Oh yeah, you would also need another for something like a PostGreSQL. smh... Then take some tylenol, create a agent to sing you a lullaby, and take a nap. lmao... Thanks for all of the videos brother! You are a beast and a legend!

@yanzhang1181 4 ай бұрын

Love your LangChain tutorial. Any Flowise LlamaIndex tutorial to create production-ready apps?

@leonvanzyl 4 ай бұрын

I'm waiting for Llama Index to come out of Beta 👍

@EmoteB 4 ай бұрын

First of all, I want to thank you for your work in creating video guides. I'm using Flowise and it's giving me very good results, but I have a small issue that I'm not quite sure how to solve. When I use agents with custom tools, the agent sometimes doesn't use the tools. I work a lot in the prompt to make them use the tools (to bring and modify information from a database), but as the context grows, the use of the tool ends up being lost and the agent ends up inventing information. I'm not sure if there's any solution to this with chain nodes since I can't use custom tools with chains. It would be great if you could make a video at some point about using agents and chains together or if there's any strategy to this issue I'm having. Best regards.

@leonvanzyl 4 ай бұрын

Thank you! Agents video will release soon. Hopefully that will help. All you can really do is lower the temperature, to 0 if need be. Use GPT-4 or greater. Add instructions to the system message to tell the model to use the tool.

@rushitparmar2042 Ай бұрын

Hi, I installed flowise ai recently and I am not getting the option “analyze chatflow” like you showed in this vedio. How can I connect lamgsmith to analyze chat flow ?

@OstrichCode 3 ай бұрын

Thanks Leon! It's powerful. I can analyze Chatflow using LangSmith with localhost:3000. However, the 'Analyse Chatflow' option disappeared after I deployed Flowise with Render.

@leonvanzyl 3 ай бұрын

Ensure you deploy the latest version of Flowise to Render. In the latest version of Flowise they moved the Analyze chatflow option to a nested option under settings.

@OstrichCode 3 ай бұрын

It has been moved to the Chatflow Configuration. Thank you so much@@leonvanzyl

@MahmoudShakershakercs 2 ай бұрын

When I use PDF in RAG , I want the user to ask, "In which page number did you get this data ?" the model responds to him with a page number , How can I do it ?

@topodin 4 ай бұрын

In the final LLM response, how do you assemble a common response from several sequentially constructed LLMChains if each previous response is an invitation to the next LLMChain? I use Prompt Chaining from the marketplace (Use output from a chain as prompt for another chain)

@topodin 4 ай бұрын

or how to save the chain of reasoning into the final assembled answer if autoGPT is used?

@leonvanzyl 4 ай бұрын

All the outputs from the previous chains can be added to the final prompt. Simply create placeholders / variables for each of the previous chains in the final prompt template and map their values. You can then give the prompt and instruction to format all these values in whatever way you want. For interest sake, what's the use case?

@topodin 4 ай бұрын

@@leonvanzyl Thanks for the tip. I was just guessing that's how it's done. I wonder if it will result in re-spending tokens in the final build. The variant in which I use it is the preparation of marketing research related to the identification of pain points in the audience and further development of this idea in the text of advertisements, which takes into account these pain points.

@SE-fk8ey 4 ай бұрын

I have a problem! When i try to activate the 'analyze flow' function by entering my Langsmith API credentials, after saving, the chat stops to work. With 'chat stops to work' i mean that i can't input anything when i press the chat icon. The entirety of the input field is gray. Nothing else seems to have changed in my chain and prompt... I'm running the exact same canvas setup as you by the way.

@leonvanzyl 4 ай бұрын

Lol, I noticed that too lately. Simply go back to the dashboard and then go into your chatflow 👍

@abiolasamuel8092 4 ай бұрын

When building a custom GPT in ChatGPT4 there's a feature called Conversation Starters. Not sure whether Flowise have same feature.

@leonvanzyl 4 ай бұрын

Would have been awesome! Hopefully they implement something like that soon.

@iokinpardoitxaso8836 4 ай бұрын

It is ready on Flowise. Just click on top right Setting button for any chatflow and under "Starter Prompts" you should be able to add them.

@abiolasamuel8092 4 ай бұрын

@@iokinpardoitxaso8836 that was quick.

@abiolasamuel8092 4 ай бұрын

@@iokinpardoitxaso8836 Is there a way to personalize the default message in the chat window? The default message is "Hi there! How can I help?". Any way to change that?

@stephenwhiting8508 3 ай бұрын

Leon love, love, love these videos. When I try to upload a file from S3, I keep getting the error "Unable to load file in the unstructured loader" I am using their paid version and its a hosted version of flowise.

@stephenwhiting8508 3 ай бұрын

@leonvanzyl any idea how to solve?

@stephenwhiting8508 3 ай бұрын

Solved it

@leonvanzyl 3 ай бұрын

Hi there. The S3 loader is definitely a bit more complicated than the other loaders. You need access to an unstructured.io API (either self hosted, or with unstructured.io cloud). You also need to set up access to the S3 bucket. This probably requires a dedicated video.

@leonvanzyl 3 ай бұрын

Awesome, what was the issue?

@stephenwhiting8508 3 ай бұрын

@@leonvanzyl Thanks Leon. Also a fellow SA, but living in Canada now.

@bassamel-ashkar4005 4 ай бұрын

Leon. I did exactly as you did but I'm using OpenAI assistant node and custom tool. I get the following error when I try to send a message. it doesn't seem to be coming from the AI bur rather from a default message from flowise. API key must be provided when using hosted LangSmith API. I have added the correct langsmith API and analysis is turned on. Any idea why this error could be happening?

@bassamel-ashkar4005 4 ай бұрын

seems like its a bug. flowise team is working on it.

@leonvanzyl 4 ай бұрын

Thanks for the info

@123xxxify Ай бұрын

Why we dont have the Chain Analyze option in the settings menu

@leonvanzyl Ай бұрын

Flowise like to move things around. It's under settings and config.

@IdPreferNot1 4 ай бұрын

Shows how why my token cost exploded with "pseudo memory" feature simply passing entire chat each time. It will be interest to see how this, and more importantly, explicit RAG may be anachronistic in a years time. Here's to a 10MM token context window next year!

@leonvanzyl 4 ай бұрын

Image the cost of passing in a million tokens which each request 😁.

@IdPreferNot1 4 ай бұрын

Stateless LLM's need to change... but just realized that if its no longer stateless, its probably AGI!! @@leonvanzyl

@mohitpatil2236 2 ай бұрын

In setting there is no option show of "ANalyse chatflow" for me

@mohitpatil2236 2 ай бұрын

sorry for this comment i got ans

@leonvanzyl 2 ай бұрын

It was moved to settings / config / analysis

@TheBestgoku 4 ай бұрын

how are they going to charge people? i don't get it. I am using their system(flowise) for a few simple bots on my servers. how will lanchaing charge me??? what am i not undersatnding?

@leonvanzyl 4 ай бұрын

I think you are referring to LangSmith. Langchain and Flowise is free to use. You do not have to use Langsmith though. It's an optional tool that simply assists with debugging.

@TheBestgoku 4 ай бұрын

@@leonvanzyl thank you for explaining, i was just amazed , like what technology do these guys have that is going to charge me when i dont even have a bank linked. i understnd that its if u ink the api then they will