FREE Local LLMs on Apple Silicon

FREE Local LLMs on Apple Silicon | FAST!

Рет қаралды 136,582

Ай бұрын

Step by step setup guide for a totally local LLM with a ChatGPT-like UI, backend and frontend, and a Docker option.
Temperature/fan on your Mac: www.tunabellysoftware.com/tgp... (affiliate link)
Run Windows on a Mac: prf.hn/click/camref:1100libNI (affiliate)
Use COUPON: ZISKIND10
🛒 Gear Links 🛒
* 🍏💥 New MacBook Air M1 Deal: amzn.to/3S59ID8
* 💻🔄 Renewed MacBook Air M1 Deal: amzn.to/45K1Gmk
* 🎧⚡ Great 40Gbps T4 enclosure: amzn.to/3JNwBGW
* 🛠️🚀 My nvme ssd: amzn.to/3YLEySo
* 📦🎮 My gear: www.amazon.com/shop/alexziskind
🎥 Related Videos 🎥
* 🌗 RAM torture test on Mac - • TRUTH about RAM vs SSD...
* 🛠️ Host the PERFECT Prompt - • Hosting the PERFECT Pr...
* 🛠️ Set up Conda on Mac - • python environment set...
* 🛠️ Set up Node on Mac - • Install Node and NVM o...
* 🤖 INSANE Machine Learning on Neural Engine - • INSANE Machine Learnin...
* 💰 This is what spending more on a MacBook Pro gets you - • Spend MORE on a MacBoo...
* 🛠️ Developer productivity Playlist - • Developer Productivity
🔗 AI for Coding Playlist: 📚 - • AI
Repo
github.com/open-webui/open-webui
Docs
docs.openwebui.com/
Docker Single Command
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
- - - - - - - - -
❤️ SUBSCRIBE TO MY KZfaq CHANNEL 📺
Click here to subscribe: / @azisk
- - - - - - - - -
Join this channel to get access to perks:
/ @azisk
- - - - - - - - -
📱 ALEX ON X: / digitalix
#machinelearning #llm #softwaredevelopment

Пікірлер: 274

@camsand6109 Ай бұрын

This channel is the gift that keeps on giving.

@JosepCrespoSantacreu Ай бұрын

Another great video Alex, I really enjoy your videos. And I really appreciate your perfect diction in English, which makes it easy to follow your explanations even for those who do not have English as their first language.

@asnifuashifj91274 Ай бұрын

Great video Alex! yes please make videos on image generation!

@AC-cg6mf Ай бұрын

I really like that you showed the non-docker install first. I think too many rely on docker black-boxes. I prefer this. Thanks!

@philipo1541 28 күн бұрын

Dockers are not a black-box. You can get it in them, and change stuff!!!

@veccio 11 күн бұрын

Respectfully, Docker need not be a black box. Don’t be afraid to tinker and dig in. :) But I get how doing it manually forces you to touch different parts.

@3monsterbeast 12 күн бұрын

This channel is going to be growing so fast; you make great videos that are very helpful!

@brunosanmartin1065 Ай бұрын

These videos are so exciting for me; this channel is the number one on KZfaq. That's why I subscribe and gladly pay for KZfaq Premium. A hug, Alex!

@AZisk Ай бұрын

thanks for saying! means a lot

@RealtyWebDesigners Ай бұрын

Now we need 1TB MEMORY DRIVES (Like the Amiga used to have 'fast ram' )

@MrMrvotie Ай бұрын

@@AZisk Is their any chance you could incorporate a PC GPU Relative Performance Equivalence to each new apple silicon microchip that you review?

@7764803 Ай бұрын

Thanks Alex for videos like this 👍 I would like to see Image generation follow up video 😍

@gustavohalperin Ай бұрын

Great video!! And yes, please add a video explaining how to add the images generator.

@Ginto_O Ай бұрын

Thank you, got it to work without docker

@cstenger 12 күн бұрын

I like the manual installation process because it uses less resources than having docker running all the time on your mac. thanks for the tutorial, I really enjoyed doing it and seeing how it works.

@ChrisHaupt Ай бұрын

Very interesting, will definitely be trying this when I get a little downtime!

@davidgoncalvesalvarez Ай бұрын

My M1 Mac 16GB be real frightened on the side rn.

@blackandcold Ай бұрын

I ran 7b variants no problem on my now sold m1 air 16g

@ivomeadows Ай бұрын

got macbook with the same specs. tried to run 15b starcoder2 quantized k5m in LM studio on it, max GPU layers, getting me around 12-13 tokens per sec, not good but manageable

@RobertMcGovernTarasis Ай бұрын

Don't be, unless you are using other things that are super heavy as well. Llama3 8B(?) takes up about 4.7GB of Ram, with the Silicon's event use of the Nvme and Swap you'll be fine. (I prefer using LM Studio now to Ollama as it has CLI and Web built in, no need for Docker/OrbStack but, Ollama on its own without a WebUI works too)

@martinseal1987 Ай бұрын

😂

@DanielHarrisCodes Ай бұрын

Great video. What format are LLM models download as? Looking into how I can use those downloaded with OLLAMA with other technologies like .NET

@aldousroy Ай бұрын

Awesome thing waiting for more videos on the way

@AaronHiltonSPD Ай бұрын

Amazing tutorial. Great stuff!

@AZisk Ай бұрын

Thank you! Cheers!

@mrdave5500 Ай бұрын

Woot woot! great stuff. Nice easy tutorial and I now have a 'smarter' Mac. Thanks :)

@kaorunguyen7782 28 күн бұрын

Alex, I love this video very much. Thank you!

@ReginaldoKono 26 күн бұрын

Yes Alex, you will help us more if we could learn with you how on how to add an image generator as well. We thank you for your time and colaboraron. Your channel is a must have subscription in it now-a-days.

@ilkayayas 29 күн бұрын

Nice. Image generation and integrating new chatgpt in to this will be great.

@marcel948 7 күн бұрын

Excellent video! Keep up!

@loveenjain Ай бұрын

Excellent Video giving it a try tonight on my M3 Max 14 inch model and see what are the results will share probably...

@iv4sik Ай бұрын

if ur trying docker, make sure it is version 4.29+, as host network driver (for mac) revealed there as a beta feature

@moranmono Ай бұрын

Great video. Awesome 👏

@erenyeager655 Ай бұрын

One thing for sure... I'll be implementing this on my menu bar for easy access :D

@vadim487 Ай бұрын

Alex, you are awesome!

@sungm2n Ай бұрын

Amazing stuff. Thank you

@johnsummers7389 Ай бұрын

Great Video Alex. Thanks.

@AZisk Ай бұрын

Glad you liked it!

@gligoran Ай бұрын

Amazing video! I'd just recommend Volta over nvm.

@jorgeluengo9774 Ай бұрын

by the way, I just joined your channel, I really enjoyed these videos, very helpful, thanks!

@AZisk Ай бұрын

awesome. welcome!

@AzrealNimer Ай бұрын

I would love to see the image generation tutorial 😁

@bvlmari6989 Ай бұрын

Amazing video omg, incredible tutorial man

@AZisk Ай бұрын

Glad you liked it!

@BenjaminEggerstedt Ай бұрын

This was interesting, thanks

@guyguy467 Ай бұрын

Thanks! Very nice video

@AZisk Ай бұрын

Wow! Thank you!

@OrionM42 Ай бұрын

Thanks for the video.😊😊

@sikarinkaewjutaniti4920 Ай бұрын

Thx for sharing good stuff for us. Nice onec

@soulofangel1990 Ай бұрын

Yes, we do.

@DaveEtchells Ай бұрын

I was gonna spring for a maxed M3 Max MBP, but saw rumors that the M4 Max will have more AI-related chops, so just picked up a maxed M1 Max to tide me over 😁 Really excited about setting all this up, finding this vid was very timely, thanks!

@mendodsoregonbackroads6632 28 күн бұрын

Yes I’m interested in an image generation video. I’m running llama3 in Bash, haven’t had time to set up a front end yet. Cool video.

@dibyajit9429 Ай бұрын

I've just started my career as a Data Scientist, and I found this video to be awesome! 🤩🥳Could you please consider making a video on image generation (in LLama 3) in a private PC environment?🥺🥺

@LucaCilfoneLC Ай бұрын

Yes! Image generation, please!

@WokeSoros 24 күн бұрын

I was able to, by tracking down your Conda video, get this running. I have some web dev and Linux experience, so it wasn’t a huge chore but certainly not easy going in relatively blind. Great tutorial though. Much thanks.

@erwintan9848 Ай бұрын

Is it fast on mac m1 pro too? How many storage used for all instalation sir? Your video is awesome!

@innocent7048 Ай бұрын

Here you have a super like - and a cup of coffee 🙂

@AZisk Ай бұрын

Yay, thank you! I haven't been to Denmark in a while - beautiful country.

@XinYue-ki3uw Ай бұрын

i like this tutorial, it is computer dummy friendly~

@willmartin4715 Ай бұрын

i believe my laptop has 80 Tensor cores. for starters. This looks like a really good shift for a fri night! thanks.

@shapelessed Ай бұрын

YO! Finally hearing of a big Svelte project! Like really, it's so much quicker and easier to ship with Svelte than others, why am I only seeing this now?

@AZisk Ай бұрын

Svelte for the win!

@precisionchoker Ай бұрын

Well.. Apple, Brave, New York times, IKEA among other big names all use svelte

@shapelessed Ай бұрын

@@precisionchoker But they do not acknowledge that too much..

@youssefragab2109 Ай бұрын

This is really cool, love the channel and the videos Alex! Just curious, how is this different to an app like LM Studio? Keep up the good work!

@yuanyuanintaiwan 17 күн бұрын

My guess is that this web UI has more capabilities such as image generation which LM Studio doesn’t have. If the goal is simply to have text interaction, then I agree that this may not be necessary

@Meet7 Ай бұрын

thanks alex

@lucerocj 3 күн бұрын

Would love to see an extended video on working with local files for this option.

@Raptor235 Ай бұрын

Great video Alex, is there anyway to have an LLM execute local shell scripts to perform tasks?

@Dominickleiner 28 күн бұрын

instant sub, great content thank you!

@AZisk 28 күн бұрын

Welcome aboard!

@Daydream_Dynamo Күн бұрын

Please do a video about image generation!!

@RealtyWebDesigners Ай бұрын

BTW - One of the BEST programmer channels!

@keithdow8327 Ай бұрын

Thanks!

@AZisk Ай бұрын

Wow 🤩 thanks so much!

@akhimohamed Ай бұрын

As a game dev, this is so good to have. Btw am gonna try this on parallels for my m1 pro

@Lucas-fl8ug 29 күн бұрын

You mean in windows through parallels? why would it be useful?

@ashesofasker 28 күн бұрын

Great video! So are you saying that we can get ChatGPT like quality just faster, more private and for free by running local LLM's on our personal machines? Like, do you feel that this replaces ChatGPT?

@agnemedia624 Ай бұрын

Thanks 👍🏻

@haralc Ай бұрын

Oh you got distracted! You're a true developer!

@filipjofce 29 күн бұрын

So cool, and it's free (if we don't count the 4 grands spent for the machine). I'd love to see the images generation

@yianghan751 Ай бұрын

Alex, excellent video! Can my MacBook air m2 with 16G RAM host these AI engines smoothly?

@pixelplay1098 Ай бұрын

Amazing stuff as Usual. Now make a tutorial on Automatic 1111

@jakubpeciak429 Ай бұрын

Hi Alex, I would like to see the image generation video

@ontime8109 Ай бұрын

thanks!

@jehad4455 Ай бұрын

Mr. Alex Ziskind Could you clarify whether training deep learning models on a GPU for the Apple Silicon M3 Pro might reduce its lifespan? Thank you.

@99cya Ай бұрын

Hey Alex, would you say Apple is in a very good position when it comes to AI and the required hardware? So far Apple has been really quiet and lots of ppl dont think Apple can have an edge here. Whats your thought in general here?

@113bast Ай бұрын

Please show image generation

@gayanperera7273 Ай бұрын

Thanks @Alex, by the way is there a reason it can only use GPU, any reason not taking advantage of NPUs ?

@MohammedAraby 26 күн бұрын

Well be happ to see a tutorial for automatic 1111 ❤

@toddbristol707 Ай бұрын

Great channel! I just did a build something similar with lm studio and flask based web ui. I’m going to try this method now. Btw, what was the ‘code .’ command you ran? Are you using visual studio code? Thanks again!

@AZisk 26 күн бұрын

Thanks! and thanks for joining. I did the flask thing a few videos ago, but it's just another thing to maintain. I find this webui a lot more feature rich and better looking. And yes, the 'code .' command just opens the current folder in VSCode

@thevirtualdenis3502 28 күн бұрын

Thanks ! Is Macbook air enough for that?

@travelchoice89 Ай бұрын

🚀 Excited to dive into this video! Local LLMs on Apple Silicon for FREE? That's a game-changer! Let's see just how FAST these optimizations are! 🍏💻

@AIandVisuals 6 күн бұрын

Great video. Congrats. But how to install node? Would it not work with a Python front end?

@zorawarsingh11 Ай бұрын

Yes do images please 🙏🏻

@cookiebinary Ай бұрын

Tried llama3 on 8GB ram M1 :D ... I guess I was too optimistic

@OlegShulyakov 29 күн бұрын

When there will be a video to run LLM on an iPhone or iPad? Like using LLMFarm

@AdityaSinghEEE Ай бұрын

Can't believe, I found this video today because I just started searching for Local LLMs yesterday and today, I found the complete guide. Great video Alex :)

@scorn7931 28 күн бұрын

You live in Matrix. Wake up

@swapwarick Ай бұрын

I am running llama, code Gemma on my laptop for local files intelligence. It's slow but damm it reads all my PDFs and give perfect overview

@devinou-programmationtechn9979 Ай бұрын

Do you do it through ollama and open webui ? I m curious as to how you can send files to be processed by llms

@ShakeAndBakeGuy Ай бұрын

@@devinou-programmationtechn9979 GP4All works fairly well with attachments. But I personally use Obsidian as a RAG to process markdown files and PDFs. There are tons of plugins like Text Generator and Smart Connections that can work with Ollama, LM Studio, etc.

@TheXabl0 Ай бұрын

Can you describe this “perfect overview”? Just curious what you mean by

@swapwarick Ай бұрын

Yes running open webui for llama and code Gemma llms on windows machine. Running open webui on localhost gives textarea where you can upload the file. The upload takes time. Once it is done, you can ask questions like give me an overview of this document, tell me all the important points of this document etc

@TheChindoboi 29 күн бұрын

Gemma doesn’t seem to work well on Apple silicon

@AlexLaslau Ай бұрын

MBP M1 Pro with 16GB of RAM would be enough to run this?

@rickymassi Ай бұрын

Why not doing a deployment with Electron, so you have a desktop application. Btw I love this thing!!!

@thetabletopskirmisher Ай бұрын

What advantage does this have over using LM Studio that you can install directly as an app instead of using the Terminal? (Genuine question)

@matteobottazzi6847 Ай бұрын

A video on how you could incorporate these LLMs in your applications would be super interesting! Let's say that in your application you have a set of pdfs or html files that provide documentation on your product. If you let these LLMs analyse that documentation, then the user could get very useful information just asking and not searching through all of the documentation files!

@FelipeViaud Ай бұрын

@neoqe6lb 13 күн бұрын

Ollama has api endpoints that you can integrate in your apps. Check their documentation.

@aaronsayeb6566 28 күн бұрын

do you know if any llm would run on base model M1 MacBook Air (8GB memory)?

@uwegenosdude 14 күн бұрын

Thanks for the video. I tried to download the code companion. Do you know why when the download of this LLM is going on, happens an upload of a couple of GBytes?

@motocar1585 Күн бұрын

how to do it with docker and also can i live this host it on vercel or netlify ??

@Mikoaj-ie6gt Ай бұрын

very intresting

@reyaustineechavez9600 9 күн бұрын

This hungs up my M1 Air everytime I gave a prompt Lol

@tyron2854 Ай бұрын

What about a new M4 iPad Pro video?

@faysal1991 26 күн бұрын

lets do some image generation please it would be super helpful

@alexanderekeberg4343 Ай бұрын

should i upgrade from macbook pro 2020 (intel core i5 8th gen quad-core 1.4ghz) to macbook air m3 15 inch for coding?

@kdnz_music 12 күн бұрын

Just install LM Studio and get any model and good native ui in a couple clicks

@bekagelashvili2904 22 күн бұрын

easy question, if i am not developer, what's the benefit i get from installing LLM in my apple silicon, what's the difference, between free version, or paid version of ai models ?

@Megabeboo Ай бұрын

How do I find out about the hardware requirements like RAM, disk space, GPU?

@truenetgmx Ай бұрын

now benchmark it vs mac air :) also wonder how much these are usefull tools and not just toys

@cjchand 19 күн бұрын

Just some food for thought for future vids: Anaconda's licensing terms changed to require any org > 200 employees to license it. For this reason, many Enterprises are steering their devs away from Anaconda. Would be helpful if the tutorials used "vanilla" Python (e.g.: venv) unless Conda were truly necessary. Thanks for the vids and keep up the great work!

@AZisk 19 күн бұрын

good to know. thanks

@justintie Ай бұрын

the question is: are opensource LLMs just as good as say chatGPT or Gemini?

@jorgeluengo9774 Ай бұрын

Thank You Alex, amazing video, I followed all steps and I enjoyed the process and the results with my m3 max. I wonder if there is a GPT that we can use from the laptop and have searches online since the cutoff knowledge date of these models seem to be over a year ago or more. For example when I ask questions of what is the terraform provider version for aws or other type of platform, is old and there is a potential to have deprecated code responses. What do you recommend in this case? not sure if you have already a video for that lol.

@AZisk Ай бұрын

that’s a great question. you’ll need to use a framework like flowise or langchain to accomplish this I believe, but i don’t know much about them - it’s on my list of things to learn

@jorgeluengo9774 Ай бұрын

@@AZisk makes sense, I will do some research about it and see what I can find out to test but I will look forward when you share a video with this type of model orchestration, will be fantastic.

@WilliamShrek 28 күн бұрын

Yes yes please make a video generation video!!!

@IsaacFromHK Ай бұрын

Can someone tell me how is it different from LM studio, Anything LLM or using Lamafile? I get a bit confused with all these. Also can I make this to run with RAG?