OpenAI Vision API Crash Course - Chat with Images (Node)

Рет қаралды 2,600

Күн бұрын

#openai
We can use the OpenAI Vision API for more than just chatting with images. We can extract unstructured data as well and use it in our applications.
🙏 Support My Channel:
Buy me a coffee ☕ : www.buymeacoffee.com/leonvanzyl
💬 Chat with Like-Minded Individuals on Discord:
/ discord
🧠 I can build your chatbots for you!
www.cognaitiv.ai
🕒 TIMESTAMPS:
00:00 - Introducing Vision API
01:53 - Project setup
06:43 - Image from URL
09:10 - Passing local image

Пікірлер: 41

@leonvanzyl 3 ай бұрын

Any cool project ideas that use the Vision API?

@DarkKnight-uk7mq 3 ай бұрын

A comic book reader

@PubgSpeed-wl8yo 3 ай бұрын

Hi what should i do with this python, how can i pass this to the client? It would be nice if you could show me how to connect it to a web site, or a web application, and the fact that it pulls data from a picture is not new).

@meister4831 2 ай бұрын

An RAG support chatbot leveraging all the technical illustrations and screenshots from the manual...

@bryanrokhose 3 ай бұрын

Leon, we would like the python version 😊

@DarkKnight-uk7mq 3 ай бұрын

Yes we dos

@ward_jl 3 ай бұрын

Amazing content as always. Looking forward to learning about more use cases and how to use this in Flowise!

@StephenGroenewald 3 ай бұрын

Man your timing is perfect. I have 200 custom GPT and starting to release them and some I am moving to assistants for deployment in FlowWiese, VoiceFlow etc and want to do direct code so this is perfect.

@DarkKnight-uk7mq 3 ай бұрын

Another great one

@leonvanzyl 3 ай бұрын

Thank you

@mlavinb 2 ай бұрын

Great content! thanks for sharing

@leonvanzyl 2 ай бұрын

You're welcome 🤗

@AndrewChildsza 3 ай бұрын

I just used this to create a auto narration of some HellDivers 2 gameplay... It's awesome, thanks!

@leonvanzyl 3 ай бұрын

That is an awesome use-case!

@bwilliams060 3 ай бұрын

Thanks Leon, yes a python version would be great!

@PepettosClub 3 ай бұрын

other ideas : - an analysis of a pdf or image and the data is sent to google sheet or google doc (e.g. a quote). - analysis in an image folder (google drive) with the creation of an image description or comment and hashtag, and it's sent to Pinterest or Instagram - from a photo of an item, he writes an ad for a site like Craigslist or Ebay

@user-iz7er7ks4x 3 ай бұрын

awesome! Leon, is there any way you can tell us how to connect Flowise and email?

@leonvanzyl 3 ай бұрын

Thanks! I have very similar videos on my channel where we create custom tools that use Make.com. Make will also allow you to send emails.

@cesar333 3 ай бұрын

I would like to see how i can upsert several images into a vector database and have a QA bot analyze the question and return what it thinks from the image.

@alankerrigan Ай бұрын

Is it more expensive to send it the image as opposed to asking it to analyse the image via a URL? Great videos, btw!

@leonvanzyl Ай бұрын

Same cost

@AndrewChildsza 3 ай бұрын

Would be cool to analyse security camera footage...

@leonvanzyl 3 ай бұрын

This API can work with videos as well. I've seen an example of this in the OpenAI docs.

@ia_para_Negocios 3 ай бұрын

very good video. but I think the most people are intimidated with languague code. I does same result using typebot or N8n asking to vision and get same results. its easy to understand and have no code implementation only http requests.... by the way different ways to get same result but the important here in this video is the knowledge. thanks so much to sharing. sorry for my English i am from Brasil

@abiolasamuel8092 3 ай бұрын

Would be great if this can be integrated with FW to analyze uploaded files via the chatbot.

@leonvanzyl 3 ай бұрын

It's already available in Flowise. Just enable "upload image" on the ChatOpenAI node 😎

@abiolasamuel8092 3 ай бұрын

@@leonvanzyl Oh I see. I did actually enabled the feature but since it isn't working in AWS I thought maybe you were referring to another feature. Looks like I'll move to Render eventually.

@bisrat_girma Ай бұрын

deserve a subsribe

@leonvanzyl Ай бұрын

Thank you

@snehasissnehasis-co1sn 3 ай бұрын

Ollama...... flowise......... what'sapp chatbot with local model ... create workflow possible??????

@leonvanzyl 3 ай бұрын

If you're using a local chatbot, services like WhatsApp will not be able to call your local endpoints.

@florentflote 3 ай бұрын

@eivanmtzleal9547 3 ай бұрын

my thanks for showing examples with JS, there is very little information with JS...

@leonvanzyl 3 ай бұрын

You're welcome 🤗

@smen66 3 ай бұрын

So far I have had problems extracting languages other than English from images.

@leonvanzyl 3 ай бұрын

Check out the "limitations" section in the API docs: "Non-English: The model may not perform optimally when handling images with text of non-Latin alphabets, such as Japanese or Korean." platform.openai.com/docs/guides/vision

@smen66 3 ай бұрын

@@leonvanzyl Yep. But German is in Latin alphabet and i couldn't get good results yet.

@caiokaspary1 3 ай бұрын

Why don't you use type = 'json_object'? This way, you will receive a valid JSON in 100% of cases. In your way, trusting only the prompt, sometimes we can. Receive non-valid json. Does that make sense to you? I am unsure what I am talking about, but I am commenting based on my experience. Also, 2 things: 1. I'm not a coder, but I understand a bit. A friend referred me to your channel because I wanted to learn flowise. 2. The most watched playlist from your channel is Flowise AI (2024) Tutorial Why don't you do videos like that, 'Chat with Images' inside Flowise? It would be amazing. I also miss videos with custom APIs being called in Flowise.

@leonvanzyl 3 ай бұрын

Thank you for the feedback