How Did Llama-3 Beat Models x200 Its Size?

Рет қаралды 100,608

Ай бұрын

Sign up Shipd now to start earning while coding! tally.so/r/3jBo1Q
And check out Datacurve.ai if you're interested: datacurve.ai/
In this video, I compiled the latest Llama-3 news and information that you might have missed. Llama-3 is actually very impressive, and I am going to find my jaws because I accidentally dropped it somewhere.
xAI News
[Grok-1] x.ai/blog/grok-os
[Grok-1.5 Vision] x.ai/blog/grok-1.5v
[Code] github.com/xai-org/grok-1
Llama-3 News
[Blog] ai.meta.com/blog/meta-llama-3/
[Huggingface] huggingface.co/collections/me...
[NVIDIA NIM] nvda.ws/3Jn5pxb
This video is supported by the kind Patrons & KZfaq Members:
🙏Andrew Lescelius, alex j, Chris LeDoux, Alex Maurice, Miguilim, Deagan, FiFaŁ, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner
[Discord] / discord
[Twitter] / bycloudai
[Patreon] / bycloud
[Music 1] massobeats - swing
[Music 2] massobeats - lush
[Music 3] massobeats - glisten
[Profile & Banner Art] / pygm7

Пікірлер: 277

@bycloudAI Ай бұрын

Sign up Shipd now to start earning while coding! tally.so/r/3jBo1Q And check out Datacurve.ai if you're interested: datacurve.ai/ On a side note, I am also looking for some like-minded people that are down to work together. For video scripting to maybe revive the AI newsletter with me, feel free to hit me up on Discord if you’re interested!

@marinepower Ай бұрын

Llama 3 is 8B instead of 7B because the increased vocabulary size -- llama 8B has a feature dimension of 4096. Therefore, the initial embedding layer goes from 32000*4096 to 128000*4096, and the final prediction layer goes from 4096*32000 to 4096*128000. Aka a difference of 800M parameters.

@samsonthomas6797 Ай бұрын

This kind of knowledge 🤍🤍

@Ginto_O Ай бұрын

Bigger vocabulary means the tokens are longer?

@Ginto_O Ай бұрын

Never mind he talks about it in video sorry

@lio1234234 Ай бұрын

I much prefer this method of a greater vocabulary size as it results in higher efficiency of long contexts, scaling beyond what the 7b is in terms of efficiency past a given point.

@lelouch1722 Ай бұрын

@@Ginto_O Depends on the tokenization method, but it can be the case. In some methods like Wordpiece, high frequency words are kept as one token will low frequency are split into subwords. If you increase vocab size then you allow for more tokens and hence more "full word" token at the same time.

@xman8908 Ай бұрын

I usually Hate Facebook but this time there doing a really good thing by pioneering open sourced ai

@carkawalakhatulistiwa Ай бұрын

We happy because the use 100 million for traning ai for public

@nangld Ай бұрын

They are accelerating the AI arms race even faster, risking it all coming to billions losing jobs and heavy social unrest being suppressed by robots.

@starblaiz1986 Ай бұрын

I know right? I still can't believe we ended up in the timeline where Meta of all companies are the champions of open source! Like seriously, who went back in time and stepped on a butterfly? GIVE ME THEIR NAMES!! 😅

@daxtron2 Ай бұрын

Only because their initial llama models were leaked lol

@Vifnis Ай бұрын

Facebook becoming "Meta" is still kinda a weird flex, but hey, maybe it'll work out tin the end who knows

@sukantasaha5678 Ай бұрын

Open sourcing is better because it takes away the leverage of models like GPT4 and other closed sourced ones from their competitors. If you can't compete, disrupt the competition.

@abhi36292 Ай бұрын

sad to see Stability ai falling apart tho

@luckyb8228 Ай бұрын

Stable diffusion is not falling apart, SD3 has hit gold in my view. That is the best image generation model right now. SD3 is accessible via API and its gonna make a killing. I don't think we have seen the last of them. As a matter of fact,its only a start. Stable diffusion has has the potential to give SORA a run for its money.. We will see.

@abhi36292 Ай бұрын

@@luckyb8228 i mean the company, didn't by cloud mention that. the apis access could gradually become a closed source software although SD3 demos are amazing i agree

@float32 Ай бұрын

Or you could see it as meta dumping money to hurt the competition.

@RafaGmod Ай бұрын

If the model could train more why would they stop? I thin they may be under the expected budget and wait better results. In this case, opensourcing is a good marketing strategy

@6AxisSage Ай бұрын

This 3rd phase of Zuck has really hit his stride, amazing.

@kiwihuman Ай бұрын

As AI gets better, the zuck appears more human.

@atomiste4312 Ай бұрын

@@kiwihuman i mean, if you're a robot and you want your kind to rule over humans, going open source is the fastest way towards improvment

@juanjesusligero391 Ай бұрын

And this isn't even Zuck's final form.

@guncolony Ай бұрын

Llama 3 represents the pinnacle of civilization by the new human species, homo zuckerbergus

@GearForTheYear Ай бұрын

@@kiwihuman yes, improvements in compute are exponential. It's no mere coincidence.

@andrewlescelius474 Ай бұрын

Congrats on graduating bro 🎓🎉👏 and to clarify, I'm not the "boss man," I only want to support your excellent work. Thank you for all your videos and excited to follow along your adventure 🙂

@finalfan321 Ай бұрын

nice!

@bycloudAI Ай бұрын

you the goat, thank you so much for your kind words!

@user-qr4jf4tv2x Ай бұрын

if you live long enough you see your self become a hero - reptile zuk

@jmvr Ай бұрын

You either die a villain, or you live long enough to see yourself become a hero

@Terenfear Ай бұрын

So I guess that's Zuck's redemption arc, huh.

@TheRhopsody Ай бұрын

Gust Jenius 🎉

@CorridorCrew Ай бұрын

Congrats on graduating and good luck on your foray into doing more KZfaq. Your videos always go beyond surface level news. It’s the reason you’re the only AI channel I watch, and why I watch all the videos you drop. Looking forward to seeing how your channel grows! -Niko

@Vvk2000 Ай бұрын

Damn corridor commented😮

@bruhmoment23123 Ай бұрын

your videos are ass bruh

@fhub29 Ай бұрын

Hi Niko, love corridor

@EddieBurke Ай бұрын

I expected y'all to watch cus he's been making quality vids for a good bit now, but seeing a CorridorCrew comment with like 15 likes is bizarre

@GeorgeG-is6ov Ай бұрын

Corridor?

@user-ex6xc5ox3k Ай бұрын

How the hell is Zuck the good guy in this?

@jameshughes3014 Ай бұрын

character arc of the century for sure.

@blakecasimir Ай бұрын

The lesser evil, perhaps. FB still makes bank selling customer data...

@nangld Ай бұрын

AI is generally not a good thing. It is here to replace you.

@naevan1 Ай бұрын

Stop thinking in these terms for multibillioners please

@StevenAkinyemi Ай бұрын

@@nangld And what are you going to do about it? Cry more?

@samsonthomas6797 Ай бұрын

Mistral 7B was released based on Llama 2 architecture, i can't wait after 2-5 months what Mistral will release based on this new way of training models by Meta AI

@Slav4o911 Ай бұрын

Llama 3 based models will absolutely beat GPT4.

@paul1979uk2000 Ай бұрын

@@Slav4o911 The signs are looking promising that Llama 3 will beat GPT4 once the community starts to fine-tune them, especially looking at how big of an improvement that's been done on Llama 2, it's likely we will see some big improvements on the newer model, probably more so because these are bigger models.

@Puerco-Potter Ай бұрын

Its impressive what llama 3 8b can, I was floored with how good it can comprehend text and improvise

@basilalias9689 Ай бұрын

@@paul1979uk2000 They got there by fine-tuning the shit out of it, I have no idea how the community is supposed to put in that much power.

@metacob Ай бұрын

I did not expect that I could run an LLM that can beat an older version of GPT-4 on my own PC this year. For reference, 70B runs at ~1 token/s on a 8C CPU. Not "interactive", but I sometimes switch tabs when asking GPT-4 something bigger too. And 8B runs at 60 tokens/s on my RTX 4080, which is more than interactive!

@paul1979uk2000 Ай бұрын

Yeah, it does surprise me how quickly these open source models are developing, from a size to performance level. You get a sense that the likes of OpenAI, Microsoft and Google are using a brute force approach to A.I. which must cost them a fortune to run compared to the smart nimble way that the open source community is doing, and it makes sense, if you have limited resources, you're going to think outside the box to get better results. I really do wonder how much better a 7b, 13,b 40b and 70b can get before we get to limits that we need bigger models for better results, it looks like we are still a long way away from that because we keep finding better solutions for the given model sizes, which improves performance and like you said, it's remarkable the pace of development in just over 1 year, makes me wonder what we will see over the next 5, 10 years.

@r.k.vignesh7832 Ай бұрын

How much RAM do you need for the 70B model? And what level of quantization are you using?

@masterneme Ай бұрын

Is it possible to run 8B on a Ryzen 4700U using the iGPU paired with 32GB of RAM?

@masterneme Ай бұрын

@@r.k.vignesh7832 I got a notification but your response isn't here, anyway thanks. Is it possible to use the integrated GPU to make it a little bit faster?

@r.k.vignesh7832 Ай бұрын

@@masterneme Damn, I don't know what happened. I said that you can run but probably not very fast, as I can easily run 8B models on 16GB RAM + 6GB VRAM, and that you should try it on Ollama and see how you go

@YugKhatri-ht8kd Ай бұрын

bro you should create an LLM primer playlist, from training to inference, from a to z.

@bycloudAI Ай бұрын

I am actually planning something similar like this, it'll be sick

@chamba149 Ай бұрын

Peak thumbnail

@tannenbaumxy Ай бұрын

Hey, congrats on finishing university! Please do what you like to do the most. But in my opinion there are already a lot of AI-news youtubers that cover a lot of what is happening in the AI world on the surface but what I really like about your content is the way you try to go into one topic a bit deeper. I really like the entertaining but educational style of your videos, so keep up the great work.

@megachelick Ай бұрын

Looking forward to see more tech stuff from you. Congrats on graduating btw!

@TheMcSebi Ай бұрын

Thanks for all of your great videos! Just keep us updated with the latest and greatest AI news and tutorials :)

@matthewmckinney1352 Ай бұрын

Congratulations on graduating! 🎉That’s huge, and I have loved your videos

@hotlineoperator Ай бұрын

It is competition. Open source is way to get some users away from GPT-4 userbase. Llama is not yet ready, it make mistakes. So, not yet time to collect money from it, now is time to get postition in AI market. So, open source is clever move.

@MangaGamify Ай бұрын

Sad, I hope someone had already saved the best open source -- offline, so in the future when it become paywall people would just use the model when it was free. for DMCA I guess they should uplaod it to a torrent, so that everyone is the host.

@Slav4o911 Ай бұрын

What mistakes... have you even tested it? Llama 3 is the best open model ever released. Now open models are just a few finetunes away from flatly beating GPT4 and by a lot. Considering how much Llama 2 based models had evolved, almost nudging GPT4, I have no doubt, open source Llama 3 based models will beat GPT4, the difference is not even that big, just a little uncensoring will beat GPT4. When a model is censored it's lobotomized, so it doesn't matter how good the real GPT4 is, if people can't reach the unlobotomized model. Llama 3 will be unlobotomized by the community, there is no way a lobotomized model can ever beat a truly open and uncensored model with similar capabilities. It's funny how because of a few "bad" words, the whole AI field is lobotomized and stifled, because a few human snowflakes can't take reality and don't have the ability to think by themselves.

@elyakimlev Ай бұрын

@@Slav4o911 The problem is you can't really "unlobotomize" an LLM model without decreasing its quality. I believe the current best uncensored model is WizardLM-2-8x22b. They released it uncensored by mistake. It wasn't lobotomized in the first place. I use the IQ_4_S version and it's amazing.

@Puerco-Potter Ай бұрын

@@Slav4o911Open AI businesses model seems to be throwing more power into GPT, GPT 5 will need a small country's energy to run. Llama 3 can be run locally, that a insane difference no matter how you look at it.

@hotlineoperator Ай бұрын

@@Slav4o911 Yes it's best but ask same question twice you'll get different answers - and one answer is correct.

@Gambazin Ай бұрын

Really looking forward to your next videos man! I know you will keep doing an amazing job! Will support patreon as soon as my startup is no longer just bleeding money 😂

@AndersonPEM Ай бұрын

Thank you for your videos. You're very instructive and clear in your assessments. Keep at it :)

@AnIndieMaster 28 күн бұрын

I love when you explain research paper more than just AI News. Even this video was a little bit more in depth into the science of the machine learning than other videos out there. Hence, I continue the good work.

@AlexanderBukh Ай бұрын

Good vid bruv 🎉🎉🎉 i think you gonna have success in this.

@naptimusnapolyus1227 Ай бұрын

zuck & musk are doing some good things now.

@Puerco-Potter Ай бұрын

You are the only person that talk about AI in a way that I understand and also don't waste my time talking about random stuff for 10 minutes. I want to thank you for this. When llama 3 was announced I watched and read other channels and I was so disappointed, you have spoiled me with your quality.

@xyers9757 Ай бұрын

Congrats on graduating man!

@MaJetiGizzle Ай бұрын

Also, when it comes to LLMs, they’re spending far less money than OpenAI as well as far less compute to kneecap their competitive edge with the larger “better” models, thereby setting themselves up to capture a significant amount of the market share around AI later down the line a la Microsoft making Internet Explorer free versus Netscape who charged a bunch of money.

@beemerrox 22 күн бұрын

I think you´re right in focusing more in these in-depth researches. Makes you stand out from all the AI-influencers. You got a new subscriber!

@jsivonenVR Ай бұрын

Interesting twist of events indeed! Small yet capable models pave the way for standalone LLMs like Phi-3 🤯

@tawfikkoptan5781 Ай бұрын

Video aside (amazing video btw), the thumbnail is absolutely diabolical I cannot lie.

@H0mework Ай бұрын

Hope to see more of you. :)

@kulikalov Ай бұрын

good job! Keep it going!

@Words-. Ай бұрын

Thanks for being one of the few AI youtubers that seems very knowledgable on ML as a whole. You're doing a good job of condensing the information without leaving the juicy technicals out, imo

@yalmeme Ай бұрын

hi bro thank for video you doing a great job! just wanted to ask which software u used to create/animate you avatar at the end of the video? it's in general called png-tuber if i'm understand correctly? but which exactly do you use?

@canekpantera14 Ай бұрын

Congratulations on graduating!!

@finalfan321 Ай бұрын

you are doing great keep it up.

@syan224 Ай бұрын

Thank you for your content

@cdkw2 Ай бұрын

Full time KZfaq is a good idea but remember to keep a backup plan

@wut3v3r77 Ай бұрын

Interested in the video scripting part you mentioned during the life update section. Where can I reach out to you?

@isbestlizard Ай бұрын

Heck AI is such a vibrant and fast evolving industry this is like trying to surf a 100 ft wave and remain on top. Data curators! Ahh god that's like something from a sci-fi novel 5 years ago.... data curators... we collate and sell high quality training data ahhh

@ayushmanbt Ай бұрын

loved the video... guilty confession here: saw the thumbnail and thought it was a fireship video

@isbestlizard Ай бұрын

This llama definitely not thrown off its groove

@virtualalias Ай бұрын

Resource requirements are so high on the big models that you can effectively open and close source. Open sourcing GPT4, for instance, wouldn't halt OpenAI's revenue stream.

@KeinNiemand 28 күн бұрын

Now we just need to wait for the uncencored finetunes

@mattmantrell5708 Ай бұрын

I like your style :D

@paul1979uk2000 Ай бұрын

I suspect a big reason for them to want to release open source is because for one, the community themselves will help to improve the model a lot, which over the long run would save Mata a fortune, and two is probably to level the playing field, being that A.I. is likely going to be important in so many areas that it would be dangerous to allow so few governments and corporations control them, so open sourcing them, blows that open and puts everyone on the same playing field. If we had a situation where eventually one or two closed models dominant the market, that would give that corporation and probably the government of the country a massive advantage over everyone else, it's a given that they will use the uncensored version of the model whiles everyone else gets the restricted one, because of all this, open source is very important for A.I. models. There is also the advantage of open source models that will lower the cost for consumers and gives consumers far more control and privacy when running at a local level.

@what-un4yq Ай бұрын

Actually, it makes perfect sense to start with open sourcing. As clearly shown, AI is in its infancy And we are highly ignorant on how to properly train them. Later models can always be closed source, but this is a crucial period of information gathering and experimentation. So it's not only beyond reasonable, but actually rather smart.

@Bizlesses Ай бұрын

They guy just finished university... And here I am, having finished my Bachelor's in Software Engineering last year by cheating through all the exams, watching this video and not understanding how half the things discussed work. That is to say, you've made it, OP! Wish you luck with whatever endeavor you go for next. And to everyone else - make sure you're actually interested in the subject enough before applying! 🤣

@Lubossxd Ай бұрын

good luck with your channel, I think you can combine the mix of popular+studying. find your own mix and popularize it, not the other way around o7

@michmach74 Ай бұрын

If the whole YT thing doesn't work out, be an ML researcher lol In all sincerity, I like it when you go deeper into the papers and research. Most AI YTers either focus on AI News, test running the tools or just high level think pieces. Those are nice and all, but stuff like this is cool too. I think Yannic Kilcher does paper deep dives too? No offense to the man though, his videos are just too long. And probably too technical. While you balance the technical stuff that I'm curious about without making it well, too technical.

@bacemtayeb326 17 күн бұрын

How can I potentially test a model against gpt 4 (and other llms)? What is the name of the benchmark to be used?

@user-fr2jc8xb9g Ай бұрын

YES , continuing down the research analysis path is the more interesting option imo!

@marhensa Ай бұрын

I tried this Llama-3 on the Nvidia website and it can help my coding be very capable, maybe at the par with Sonnet level of ClaudeAI.

@Celdrith Ай бұрын

Good video!

@danial_amini Ай бұрын

The beard had me 😂

@jameshughes3014 Ай бұрын

more bycloudai videos would be awesome.

@lukacolic4193 Ай бұрын

Congratz on graduating

@DonPatro92 Ай бұрын

Tried Llama 3 Instruct on LM Studio but when I ask it something it doesn't stop generating, it just keep going. Is there any way to fix that?

@UnchartedWorlds Ай бұрын

Zuck looks more human than ever! 8:50

@c0nsumption Ай бұрын

Thanks dude. How long before we see VLMs built on this? Haha

@sjcsscjios4112 Ай бұрын

Probably llama4, and they will likely use JEPA architecture which will make it insane

@danishamin6018 Ай бұрын

Beard zuck looked more human

@SandTiger42 Ай бұрын

So. What does one need in terms of hardware to self-host a Llama 8b? or 70b?

@Lar_me Ай бұрын

If they're quantized (compressed) to the GGUF format, the numbers in their names are usually a good indicator on how much RAM you'll need to run them. For example, 8B will probably need around 6-8 GB of RAM unless you choose a heavily quantized version, which could let you get away with less RAM at the cost of a dumber AI. VRAM from an NVIDIA card will be the fastest, AMD will be a little slower (I think), and regular RAM will be the slowest. If you install LM Studio, you can view the models on Huggingface and see exactly how much RAM each version requires.

@SandTiger42 Ай бұрын

@@Lar_me Appreciate it!!!

@TheSpace81 Ай бұрын

The thumbnail is peak fiction.

@user-pm7kt8tm1s Ай бұрын

And what hardware can run that?

@jawadmansoor6064 Ай бұрын

2:21 what I am interested in (and most developers do need, though they may not realize it) is MMLU and human eval score (unbiased and uncontaminated only) because this gives the model the ability to do things that uptil now (before llama3-8B) only mixtral could do, but that is huge compared to this (don't need to mention bigger models because it is obvious they can do it too but they are just too big) so yea, I love this 8B model. I am sure next 3b or even 1b models would be as great as this (Mark Zuk promised mobile based models in 2025). So, I am really enthused and really love what meta (not facebook) is doing finally.

@Slav4o911 Ай бұрын

I think 8B models are also not very far away from running on the future mobile phones. It would be neat to have a model which can outperform GPT4 running locally on your smartphone. That reality is actually not very far away. Unless some dumb politician bans open models.

@william1106 Ай бұрын

Fire video

@aykutakguen3498 Ай бұрын

You got this broski!

@pauljones9150 Ай бұрын

Good video

@ehza Ай бұрын

thanks

@zman-1x1 Ай бұрын

Well I completed my university too. Time to experiment with llms.

@user-il1hu5xp2x 23 күн бұрын

How gpt4 is bigger then lama 3 8b with 200x like it should be 1600billion parameter at this point, or there is something i miss??

@Arewethereyet69 Ай бұрын

When in the Suck VS Susk Fight???

@ApexJnr Ай бұрын

🔥

@Gregorius421 Ай бұрын

Key takeaway: Zuck with beard looks more human.

@Matthewswanson889 Ай бұрын

Open sourcing it makes its better in the long run.

@LightPink Ай бұрын

Can you do one about ai music?

@Serizon_ Ай бұрын

isn't mistral and some other ai with name starting with p (I forgot it ) even more impressive than llama? (I think the name was phi 2 though I might be wrong)

@AaronALAI Ай бұрын

I built a 7xgpu rig that lets me run this bad boy at full fp16....frick it's amazing!

@l.halawani Ай бұрын

You make incredible youtube videos! Please more! Also if you're looking for a job related to introducing AI solutions in an enterprise, without needing to know how to strictly develop it please get in touch we're recruiting

@thebrownfrog Ай бұрын

It will sound sus coming from a guy, but your voice is nice to listen to Other "qualities"(like editing, the mamba edit, etc.) are good too

@gingeral253 Ай бұрын

Get me into the Llama club

@MarcAyouni Ай бұрын

Benchmark is one thing. But I found it gives more generic answers, even ignoring specifics in the question. So there is definitely more blur or average in it with fewer parameters.

@RichardLucas Ай бұрын

Llama3 hallucinates more than any of the other, comparable models in the Ollama index. Maybe it performs completions and follows instructions better. i haven't gotten around to that, yet. I didn't set the temperature to 0.0 and supply a seed, so your experience might be different from my own, but I casually threw it the chat prompt, "Can I get a witness?" It started off with a coherent response and around three paragraphs in, it began to respond to its own, previous paragraphs. The response was looooooong. And ridiculous. Each paragraph was a response to the previous one.

@mrrespected5948 Ай бұрын

Nice

@danielchoritz1903 Ай бұрын

10:23 is a her. The anime is called "Suzumiya Haruhi no Yūutsu" Watch it...

@EsquireR Ай бұрын

How much of this script did ai write?

@bycloudAI Ай бұрын

0 words

@nTu4Ka Ай бұрын

Besides anything else making Llama-3 open source will put pressure (remove money) on OpenAI. Lizards are cunning.

@ShaneSemler Ай бұрын

Meta open sourced it? I'm genuinely surprised.

@tsilikitrikis Ай бұрын

Mark competing OpenAI through open source

@mayatrash Ай бұрын

Meta is open sourcing it because they learned from Microsoft and vscode. They will sneak into the middle between the user and the developer and in the end the can probably monetize it somehow (think about copilot and vscode)