I built a *streaming* AI chat app

I built a streaming AI chat app

Рет қаралды 121,146

Жыл бұрын

Learn how to use the new Vercel AI SDK to quickly build streaming AI chat apps with APIs like OpenAI, Anthropic, HuggingFace, LangChain, and more.
Upgrade to get the full SvelteKit Course at fireship.io/pro
Use promo code FKIT to save 35%
Vercel AI Announcement vercel.com/blog/introducing-t...

Пікірлер: 148

@yashkhd1100 Жыл бұрын

It's incredible how many hats he is able to wear at the same time. I mean making high quality super dense tech videos, making top notch Courses, keeping track of all happenings and also creating such incredible side projects. Please do make a video about ur time management...I'm sure it's gonna worth ton to lot of people...!!

@chouaibdjerdi9092 Жыл бұрын

Yes exactly, How can someone keep up with todays tech updates and even do projects and tuto about it barely can keep up with what has been done 10 years ago HAHA

@bigs2469 Жыл бұрын

I run a 20k sub software dev channel on a separate account - It's hard to explain, but the drive you get from "I need to build this for content" is insane. I can hardly focus on my non-channel side projects because I don't have that fire under me, but when I'm building something for a video I just get carried away. Perhaps Fireship has a similar mentality.

@mwansa430 Жыл бұрын

Maybe he made an AI that is able to replicate his voice and he uses it to make the videos

@JBroMCMXCI Жыл бұрын

In a previous video he showed his workflow using all generative AI tools so I think it takes very little time to make these videos, just time to develop the workflow

@glmchn Жыл бұрын

U got bamboozled, it's just an AI making the content since day 1 🤯

@JoshuaGenders Жыл бұрын

just started working with sveltekit, so nice

@SirDamatoIII Жыл бұрын

Man I am excited for the SvelteKit course! Jeff for President!

@AdrianMark Жыл бұрын

Fantastic work man. Can't wait for the Vercel course to be out on Fireship pro

@mohaimanulislammahin4095 Жыл бұрын

0:01 train is from my country. Love from Bangladesh

@carlosfortega3470 Жыл бұрын

in the time he programs, edits, and uploads the video, I made the basic chat app. lmao

@Eurotool Жыл бұрын

In that time I made nothing

@YuriG03042 Жыл бұрын

are you actually comparing yourself to someone with over a decade of experience? bruh moment

@heroe1486 11 ай бұрын

@@YuriG03042Could play a role but those are mostly hello world examples, things that take time are usually little details you need when developing a real app, you can ignore those for video tutorial purposes

@abeechr Жыл бұрын

Dear Jeff and any of you super coders out there, How would this work at scale? If deployed, how many simultaneous users could stream data given the standard paid api? And what might the code look like to deal with hitting that limit?

@chukaraslan9051 9 ай бұрын

Easy. Use multiple accounts each with their own api keys and go brrrrrrrrr

@abeechr 8 ай бұрын

@@chukaraslan9051 So create an api key for each of my users?

@dongums Жыл бұрын

thank youuuu

@other_perfection Жыл бұрын

did you know that you can do it without Vercel AI SDK using OpenAI SSE (Server-Sent Events) Streaming API? it's not much more complicated

@marvinkr Жыл бұрын

Got content or recs on how to pull that?

@VaibhavShewale Жыл бұрын

time to make one for myself!

@greendsnow 11 ай бұрын

Vercel jumped on it because it takes 5+ seconds to complete a stream, during which they charge you for each 50 ms of edge function your using. So 100 edge units for each answer. USD 2 for 1 million units.

@chukaraslan9051 9 ай бұрын

function makeMoreOpenAIAccountsForUnlimitedAPIKeysAndFckVercel () { return "nothing to see here) }

@muhammad.mohsen Жыл бұрын

The "fkit" course 😭😭💀you're too good man!!

@YuriG03042 Жыл бұрын

FirebaseKit stack is too strong

@TheSteelcry Жыл бұрын

For anyone trying to follow along at home for your own app, in case you get a 400 code (malformed response) and no other info the possible causes can be: - Invalid or maxed out API key - Bad prompt with illegal characters or too long - in my case I passed in const messages to the request instead of const {messages} (oops!)

@salmanbehen4384 Жыл бұрын

Frick all this... We need a video on how you keep yourself updated with all this and manage your time.

@ScytherDOTA Жыл бұрын

Newbie question from someone with no backend knowledge: What would I need to look up for if I would want to make a AI chatbot such as that in React? Any tips for starters?

@stunseedfortnite Жыл бұрын

Fortnite

@stunseedfortnite Жыл бұрын

Amongus

@stunseedfortnite Жыл бұрын

Cheeseburger

@shapelessed Жыл бұрын

First of all - Don't build it in React... It's overhyped, overcomplicated and just plain slow. Meanwhile Svelte is like drugs, once you try it you don't wanna go back.

@marusdod3685 Жыл бұрын

@@shapelessed svelte is react but shit. give me jsx expression any day of the week

@radumihaidiaconu Жыл бұрын

i just did it in NuxtJS

@Jip1912 Жыл бұрын

What is the benefit of using this over just using the ChatGPT API? Is the only benefit that its streaming so typing the result instead of waiting for the whole result before showing it?

@DemPilafian Жыл бұрын

The *_"typing effect"_* makes it sentient.

@the_ws Жыл бұрын

Well I built that weeks ago using simple vanilla js (with streaming)

@pepperpeterpiperpickled9805 Жыл бұрын

does Open AI gather your data?

@jorgesaxon3781 Жыл бұрын

Im not gonna say first that would be cliche, me being early means nothing, so Ill just say fireship if you see this you should make a video on hugging face transformers

@beyondfireship Жыл бұрын

Soon... HF transformers are awesome

@fagnersales532 Жыл бұрын

❤

@devgenesis6436 Жыл бұрын

I want to built this for my webapp..i mean it would be nyc if the user ask question on my web app and based on the data on the web app the chatbot responds..like if the app is about stack overflow for ex..the chatbot responds to the data according to user so he dont have to go through all the articles...can someone help me

@mrala Жыл бұрын

can we use open source local models?

@MrAmack2u Жыл бұрын

Any idea how to maintain the streaming text while using the new function calling api openai just reelased? I had to disable stream to get it to work properly.

@cholst1 Жыл бұрын

Not an answer, but a sidenote, functions are fucking incredible.

@shateq Жыл бұрын

Vercel's ai hype train 😂😂

@agesnipes Жыл бұрын

So in this example, you're using readable streams over HTTP? Are there any advantages/disadvantages to this approach compared to websockets?

@skyhappy Жыл бұрын

nope

@potatohater5599 Жыл бұрын

Pretty sure it uses a lot more bandwidth, since I think each HTTP request has a lot more boilerplate, like headers.

@its.arjun.s 10 ай бұрын

Unlike having to keep a connection alive with WebSockets, HTTP is much more simpler, and better in cases where client->server streaming isn't required. Also HTTP is probably more familiar anyway. SSEs have always been a great option

@KnorxThieus 3 ай бұрын

@@potatohater5599No, HTTP SSE uses a single connection

@Action2me 8 ай бұрын

When I use this code, the roles are rendered as "undefined". What am I doing wrong?

@dhruvshetty3244 10 ай бұрын

can we use this in next js?

@YHK_YT Жыл бұрын

What if I want to run my own modem and have it train locally?

@ChadThieleChaddeus 4 ай бұрын

This isn’t possible with Firebase cloud functions is it? (The streaming part)

@random__weeb Жыл бұрын

OpenAI has revoked all free monthly limits for API Calls and has become a paid service now. Can someone suggest which other platform is free?

@ea_naseer Жыл бұрын

hugging face

@noooOne462 7 ай бұрын

i deployed an openai chat app into an Ubuntu server but as soon I make it https the stream stops working, any clue why is It happening

@daleryanaldover6545 11 ай бұрын

Thanks I have the app built now, I just need an API KEY 😵

@voidmind Жыл бұрын

I wonder if customer support bots based on ChatGPT, used by big corporations, could be manipulated to get unlimited prompts on things not related to a real support conversation

@agorushkin Жыл бұрын

Good question... huh. ChatGPT api afaik, has a system message - this messages basically tells the bot what it is. But hey, I'll have to look into that, sounds really interesting

@bittersweet4074 Жыл бұрын

They should do additional training on that part probably. Otherwise people just gonna write an entire book with it.

@Batman-lg2zj 11 ай бұрын

@@bittersweet4074and that’s bad thing

@chukaraslan9051 9 ай бұрын

@@Batman-lg2zjnot nessicarily. As a matter of fact I may or may not have achieved huge success with the aforementioned method and may or may not choose to make the repo public in the near future

@inowatchvideos Жыл бұрын

This was definitely his AI voice

@zzonawav Жыл бұрын

thanks Fireship .. this github repository is definitely broken lololol

@muntakim.data.scientist Жыл бұрын

Bangladeshi Train for a sec 0:01

@muddi900 Жыл бұрын

So is the stream is just library streaming the full response? Because the API does not provide a stream of the responses.

@unconv Жыл бұрын

The API does it already. You just need to add "stream: true" to the request.

@chroxify8913 Жыл бұрын

its important to note that it might be a good idea to add rate limiting to your endpoint incase you arent elon musk and can afford it either way

@facciolite Жыл бұрын

I am Elon Musk.

@arthuradriansens3291 Жыл бұрын

Is there a free alternative to this?

@manemobiili Жыл бұрын

I was in the middle of working and i put up this video as whitenoise Got instantly more overwhelmed.

@spyllite Жыл бұрын

@unidosemprendedores Жыл бұрын

Now make it work with firebase

@ricardocnn Жыл бұрын

Waiting for Vue and Nuxt support.

@compton8301 Жыл бұрын

Waiting? Why don't you give it a short?

@leerob Жыл бұрын

Vue support just landed!

@ricardocnn Жыл бұрын

@@leerob awesome!

@7heMech Жыл бұрын

Fireship really doesn't rest. (but he did rest, if you see what I did there 😉😉)

@johnsausage Жыл бұрын

Rest assured that he will rest at some point in time 😂

@hey_james Жыл бұрын

I absolutely love your videos man! But as some one with sensitivity to sibilance, your s’s sound very piercing. It would be amazing if you could tame the sibilance a tad with a de-esser 😅😅👌

@vectoralphaAI Жыл бұрын

When you say app in all your videos do you mean mobile app or website? Because websites are not mobile apps.

@notalanjoseph Жыл бұрын

Neither.. he means web application

@bthundergun Жыл бұрын

Can anyone here give me an idea what the best way would be to have a LLM that's trained on my knowledge base?

@chukaraslan9051 9 ай бұрын

Get a life That being said, look into langchain and RAGs

@gourav7315 Жыл бұрын

where is nextjs video 😅

@Dev-Siri Жыл бұрын

Beyond Fireship IS the new Fireship

@Happyday-nn6rh Жыл бұрын

how can customized data for answer ????

@unconv Жыл бұрын

By sending a system message

@Happyday-nn6rh Жыл бұрын

@@unconv i want use my data example a pdf and just see this file and answer , dont answer another question ...

@vietnam3196 11 ай бұрын

Can you do python

@PlasticEARTH 3 ай бұрын

Streamberry

@Nameless-jp2hj Жыл бұрын

His voice is definitely ai generated

@AZisk Жыл бұрын

Schwarzenegger

@boreddad420 Жыл бұрын

npm i ai

@ruocaled 11 ай бұрын

they just keep building SDKs on top of SDKs... guys, there are like 3 api endpoints of openai, these companies tries so hard to get a share and get you hooked to their eco-systems.

@nz69 Жыл бұрын

69th

@flor.7797 Жыл бұрын

This is how AI super viruses will be created 😢

@rubenuijtde Жыл бұрын

Was this recorded with your AI voice clone?

@kp-here5169 Жыл бұрын

First comment :)

@ropoxdev Жыл бұрын

No I am

@cholaempire Жыл бұрын

@@ropoxdev no you are not

@ropoxdev Жыл бұрын

@@cholaempire yes I am

@JeffBarron1 Жыл бұрын

cool ill use your chat bot instead of openai so i dont have to pay for it 😂 jk

@yolla_4 Жыл бұрын

@Beyond Fireship btw the earth is flat.

@smplchmp Жыл бұрын

triggered by all these serif fonts. otherwise, brilliant as usual 🖖

@ropoxdev Жыл бұрын

Hi (first)

@chromus9149 Жыл бұрын

first

@ropoxdev Жыл бұрын

No I am

@JBroMCMXCI Жыл бұрын

Isn’t it a bit tasteless to make jokes about India and trains?

@YuriG03042 Жыл бұрын

it's not a joke about either india or their trains? it's about AI and hype

@JuanFmTech Жыл бұрын

The typing effect gets old for the user waiting on long replies, just don’t use it

@riyaadhabrahams Жыл бұрын

But then you will have to show nothing or a loading spinner until the request is done. Which I think is a worse experience

@zokalyx Жыл бұрын

@@riyaadhabrahams exactly, as far as I know, it's not an effect, it's just the actual speed of the generation.

@ahsokaincognito Жыл бұрын

That's the actual inference process, hence streaming. It's not a fake delay

@underscore. Жыл бұрын

dumb comment

@MichaelEHastings Жыл бұрын

Love the subtle fl@t urth truths that your smarter audience picks up on. Th

@YuriG03042 Жыл бұрын

truths LMAO smarter audience

@kritikusi-666 Жыл бұрын

Can you do a tutorial on how (www.twitch.tv/trumporbiden2024 this) is done? Not sure what they are using for the backend. The AI model is uncensored. Be cool to know the process.