PandasAI, OpenAI and Streamlit - Analyzing File Uploads with User Prompts

Рет қаралды 23,972

Күн бұрын

In this video, we'll use Streamlit to build a small UI that allows users to upload a file of CSV data, and will create a text input and buttons that allow users to submit a prompt that can then be used to analyze and interrogate the data.
The prompt will be passed to the PandasAI library, which will take care of sending that prompt along with the data to OpenAI for analysis via their LLMs.
Blog post: bugbytes.io/posts/streamlit-p...
☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲:
To support the channel and encourage new videos, please consider buying me a coffee here:
ko-fi.com/bugbytes
📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀:
00:00 Intro
01:58 Loading API Key with python-dotenv
02:58 Building a UI with Streamlit
09:42 Using PandasAI and OpenAI to respond to user prompt
16:03 Generating Matplotlib Charts with PandasAI
𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮:
📖 Blog: bugbytes.io/posts/streamlit-p...
👾 Github: github.com/bugbytes-io/
🐦 Twitter: / bugbytesio
📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
Blog post: bugbytes.io/posts/streamlit-p...
Streamlit: docs.streamlit.io/
PandasAI: github.com/gventuri/pandas-ai
Titanic Data: github.com/datasciencedojo/da...
python-dotenv: pypi.org/project/python-dotenv/
#python #llm #pandas #artificialintelligence #datascience #streamlit

Пікірлер: 91

@shuntera Жыл бұрын

This is awesome. Have never heard of streamlit before and my mind is racing with what I can do with this on top of Pandas and ChatGPT

@bugbytes3923 Жыл бұрын

Thanks! Lots of possibilities with Streamlit and these other tools.

@pereganranwei2153 Жыл бұрын

Great resource would love more content relating to deploying Open AI ML solutions on streamlit

@techitint.9100 3 ай бұрын

Well explained and straight to the point, thanks so much for your video.

@bugbytes3923 2 ай бұрын

Thank you!

@zac1427 Жыл бұрын

thank you! I had this idea in my head and found this video on youtube!

@bugbytes3923 Жыл бұрын

You're welcome! Thanks for watching.

@graczew Жыл бұрын

Good stuff as always. 👍👍👍

@bugbytes3923 Жыл бұрын

Thanks a lot, Krzysztof!

@user-lc2dh6zx5p Жыл бұрын

I was actually going to ask how to display graphs on the web before i finished the video, but i did manage to figure it out on my own. You have to set PandasAI(llm,save_charts=True), then have code in the display anser logic that detects if the number of files in the folder where the images are stored changes, and display the latest image. You would need to do something else like checking for the specific filenames if you had concurrent users in your app but for now it's enough. answer=pandas_ai.run(st.session_state.df,question) if st.session_state.count

@TheLbachtschin Жыл бұрын

Great job! Thanks!!

@bugbytes3923 Жыл бұрын

You're welcome - thank you for watching!

@Ego_Maritm Жыл бұрын

Great Job bro💪

@bugbytes3923 Жыл бұрын

Thanks !

@danquixote6072 Жыл бұрын

This was really good. Would certainly be keen to know how to extend the functionality of this - for example, allow other users to input their own Open AI keys. Would also like to see how this is done in Flask too.

@bugbytes3923 Жыл бұрын

Thanks! Some good ideas there.

@SOHAILKHAN-iu8fu Жыл бұрын

One of the best channel on youtube

@bugbytes3923 Жыл бұрын

Thanks very much!

@WarLordOmar Жыл бұрын

true, love it thank you

@gustavomonne4921 Жыл бұрын

Awseme video, thanks

@bugbytes3923 Жыл бұрын

Thanks for watching!

@roberthuff3122 Жыл бұрын

Great! Very well done. I don't see the project in the GitHub repo.

@onnot701 Жыл бұрын

this is great!

@bugbytes3923 Жыл бұрын

Thanks Onno!

@jpcam4781 11 ай бұрын

Bloody brilliant! Is it trivial to scale this to 'read' a pdf with multiple tables and converting the tables into csvs? I am thinking of a use case where the user uploads a pdf of financial statements? Have you try your example using st.session_state? And conversation buffer memory or does this not work with pandasai?

@ciprianchamps Жыл бұрын

hi @BugBytes , i have this error when i try to upload the file: "AxiosError: Request failed with status code 403" and i put correctly the API-keyand the libs, any idea?

@mohammedabdelgalil5860 Жыл бұрын

Thank you so much for the help! I was not able to get it to work on streamlit cloud though only locally any info there? I would truly appreciate it

@WhySoBroke 4 ай бұрын

Great video!! Have you found any updated way or easier to run Pandas AI to analyze data? Many thanks in advance!

@SANJIVRAI6693 Жыл бұрын

Great tutorial, can you show us how to do this with a offline LLM Model where the data is private and processed locally.

@rodrigokitsuro5753 Жыл бұрын

Amazing content! 🦾 As a 'not developer' that is just starting with Python and OpenAI for work, this video was so helpful, i really apreciate it! In order to ad a new text_area for a new prompt, related to the last PandasAI response, which would be the next code lines i should use to achieve this?

@bugbytes3923 Жыл бұрын

Thank you Rodrigo! Glad it was helpful. Given the response, you could store that in a variable, and use the response as part of the prompt for the next call to pandas_ai.run(). The rest of the prompt can be constructed with another text_area from Streamlit. There's also a helpful library called "langchain" that's good for building apps with multiple calls to LLMs. Not sure at the moment how well it integrates with PandasAI, but I'll do a video on langchain soon too. Thanks again!

@rodrigokitsuro5753 Жыл бұрын

@@bugbytes3923 Thank you a lot again!! I'll be waiting this video! 🙌

@PratikBhande-qq1ht 23 күн бұрын

Can you please help me where I can upload multiple excel or csv files and ai agent will correlate between them in order to give output

@Diana-zo2ut Ай бұрын

Thanks, Can we do this with images?

@Ashmanpan 5 ай бұрын

Possible to do pandasAi with opensourse models?

@SuperLiberty2008 10 ай бұрын

how to set up function description? you have pop-up clue for every function

@mocanada304 Жыл бұрын

Hi, Thanks for the amazing video. I tried very hard to give you an answer while asking ChatGPT for help. The issue I see and ChatGPT 4 acknowledges too, is that the answer provided by chatgpt is not in the format that one can use to display the graph within the streamlit directly. Maybe you need to add a check condition if the data is plottable then use st.pyplot() ? Below is the code snippet for displaying charts using streamlit in the browswer. --- import streamlit as st import pandas as pd import numpy as np import matplotlib.pyplot as plt # Create a random DataFrame for demonstration df = pd.DataFrame({ 'A': np.random.rand(50), 'B': np.random.rand(50) + 1 }) # Create a figure and axes fig, ax = plt.subplots(figsize=(10, 5)) # Plot a bar chart using the DataFrame ax.bar(df.index, df['A'], label='A') ax.bar(df.index, df['B'], bottom=df['A'], label='B') ax.legend(loc='upper left') ax.set_title("Bar Chart") ax.set_xlabel("Index") ax.set_ylabel("Value") # Display the chart with Streamlit st.pyplot(fig)

@bugbytes3923 Жыл бұрын

Thanks! And yeah, good idea! I'll have a play around with it, and see if I can get it working.

@ordaflash 8 ай бұрын

please can you give some more details about your solution ? i cannot handle how from the response generated, we could print it as a plot in the example you have given, you creates a new chart. but in our case we need to display the chart generated by the AI no ? Thank you !

@visheshgupta2048 Жыл бұрын

That is Awesome, I also created something similar, but I was wondering if we can plot this visualization inside the streamlit application instead of separately. Is there any other way ?

@bugbytes3923 Жыл бұрын

Thanks! Not found a way to do this yet, but I'm sure it's possible.

@visheshgupta2048 Жыл бұрын

@@bugbytes3923 okay thanks for your reply

@binoyxbabu 8 ай бұрын

@@bugbytes3923could you pls find that way and do another video or comment here...that wil be awesome

@binoyxbabu 7 ай бұрын

I have found it and it worked well (use pyplot figure method)

@yenaung7643 Ай бұрын

Which pandasai version do you use?

@EmptyNonsens Жыл бұрын

Private LLM tut would be amazing

@bugbytes3923 Жыл бұрын

Will look into it soon!

@anupgeorge2261 7 ай бұрын

Super

@bugbytes3923 7 ай бұрын

Thanks a lot!

@RedCloudServices Жыл бұрын

I built a streamlit app with a Folium map can you make a video which interacts with a Folium map? that would be an interesting use case.

@bugbytes3923 Жыл бұрын

Good idea! Will look into that one.

@vashisht1 11 ай бұрын

On pandasai Github page under security the guy has given some info if you want to keep your data privacy. I think one can use that..

@evansmwakughu8562 4 ай бұрын

Hello BugBytes, can you create a video instead of connecting to open AI you use open source llms? That could be great thank you.

@techystu5823 11 ай бұрын

Everything was working fine until I added the from pandasai import PandasAI and then i get an error on my streamlit page says No module named 'bs4'. Can you please help

@bugbytes3923 11 ай бұрын

Not sure why that is, but you can try installing that library with the command: pip install BeautifulSoup4

@latestmoviesforall 8 ай бұрын

how can i create a virtual environment? using windows. when i creat the two file main.py and .env, i don't seen the ( venv-llm) PS: any help plz. thanks for the knowledge you shared

@bugbytes3923 8 ай бұрын

Hey - you can try the following: 1. python -m venv ./venv-llm 2. ./venv-llm/Scripts/activate 3. pip install (enter the package names you want to use)

@latestmoviesforall 8 ай бұрын

thank you@@bugbytes3923 , I put the code in a new terminal in the C path at the end , and it gives me this message : "Python not found. Run without arguments to proceed." what should I do please ?

@JiteshPatil518 2 ай бұрын

Hi...Getting below error while running the app ImportError: cannot import name ‘PandasAI’ from ‘pandasai’ (C:\Users\RONALD AINEBYONA\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandasai_init_.py)

@abhishekbhattacherjee8873 2 ай бұрын

same error facing. were you able resolve this.

@jaysonmagdamit4784 Жыл бұрын

how can i create a virtual environment? using mac - Thanks!

@bugbytes3923 Жыл бұрын

Hey Jayson - you can run the following commands (change 'my-venv' to any name you like): -- python3 -m venv my-venv source my-venv/bin/activate -- The first command will create the environment using Python's built-in "venv" module. The second command activates the environment - once activated, you can install external packages into this environment.

@MahalingamBalasubramanian 6 ай бұрын

It seems this code is not supporting graphs, I am getting below response Generating an answer, please wait... The bar chart for the top 3 Organization Names by Number of employees is displayed on the axes with coordinates (0.125,0.11) and a size of 0.775x0.77.

@anouarben779 Жыл бұрын

Are OpenAI API keys free or what, Mine requires credits is it the same for everyone?!

@unl0ck998 Жыл бұрын

OpenAI API is not free

@iwhiteside 7 ай бұрын

Hi Bugbytes I am new to python your videos are great I am getting an error from your code PandasAI is not defined I am sure I did something wrong

@AshWickramasinghe 7 ай бұрын

the library got updated and the new version isn't using the PandasAI

@binoyxbabu 9 ай бұрын

Bro, i really liked your video but could you reply or do another video that how could we bring graphs also to our streamlit page rather showing it in a seperate window like in this video. I think with that its 100% and a complete product....i know you said that its possible but matter of finding it how 😊

@binoyxbabu 9 ай бұрын

@bugbytes3923 fyr

@vinaysr6617 4 ай бұрын

Some of these have been revised. PadasAI is imported as smartdataframe

@bugbytes3923 4 ай бұрын

Thanks for pointing that out. Might try and do a follow-up!

@abhishekbhattacherjee8873 2 ай бұрын

can you send the revised code using the Smartdataframe

@barutistudio1397 Жыл бұрын

Can we get the code?

@adamtorabi5972 7 ай бұрын

I am interested in the code too

@glory_vn7085 9 ай бұрын

For Generating Charts，we can make a tricky: llm = OpenAI() df = SmartDataframe(df, config={"llm": llm}) df.chat('Plot the histogram of countries showing for each the gdp') from PIL import Image image = Image.open('exports/charts/temp_chart.png') st.image(image, caption='') ######### 'exports/charts/temp_chart.png', this can be found from "print(df.last_code_generated)"

@ordaflash 8 ай бұрын

please can you give more details about your solution ?

@jamesstover718 6 ай бұрын

This code works, but with latest pandasai the filename now that I get is a long unique file name. So, I had to add code to extract the relative path to my app with the unique filename and then display it. Wish this was handled better by the pandasai library, but it works. I have an issue though when plotting some charts like a horizontal bar chart where the png image file is chopping off the front of the y-axis labels. Always something to battle.

@karthikvb1293 5 ай бұрын

Even though it's rendering on UI, Still it also opens in new window. How to stop that?

@zac1427 Жыл бұрын

Guys ! I found a way to show plot in web app ! if st.button("Generate"): if prompt: with st.spinner("Generating response..."): answer = pandas_ai.run(st.session_state.df,prompt) fig_number = plt.get_fignums() if fig_number: st.pyplot(plt.gcf()) else: st.write(answer)

@bugbytes3923 Жыл бұрын

Nice! Thanks for sharing that!

@visheshgupta2048 Жыл бұрын

Hi, I used your code, and it is working on the local machine. But this code does not satisfy requirements after deploying it. Do you have any idea about this ?

@zac1427 Жыл бұрын

@@visheshgupta2048 did you use aws ? make sure you let your virtual machine download all the packages in the requirement. It took me a while to fix the bug when I deploy it on AWS

@visheshgupta2048 Жыл бұрын

@@zac1427 no, I used github and streamlit to deploy. Does AWS works? To show graph in the deployed server.

@phaninagandla4983 9 ай бұрын

Hi, I am receiving the following error when using the above code: " AttributeError: module 'matplotlib' has no attribute 'get_fignums' " : if possible can you share the entire code you?

@betenu1 Жыл бұрын

The women and children were more likely to survive because they were put on the lifeboats first.

@bugbytes3923 Жыл бұрын

True - makes sense!

@pidswing5334 Жыл бұрын

Hey BugBytes! The responses provided by my model were very raw for example, when asked about the fare price, I received the response, "32.204207968574636". Can you help me with why is this the case?

@shekarshekar4988 Жыл бұрын

Hi Mate, In the below comments,@zac1427 found a way to display the charts on the frontend. But I have played around with the code he/she mentoined. But It hasn't worked. import matplotlib.pyplot as plt if st.button("Submit"): if a: with st.spinner("Please Wait....."): b=Pandas_ai.run(df, prompt = a) fig_number = plt.get_fignums() if fig_number: st.pyplot(plt.gcf()) else: st.write(b) I am receiving the below warning and couldn't not able to display charts. UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so it cannot show the figure. Please let us know if it's worked or if you have found any other way.