Can I generate an installable app to deploy it in own infra? Which code language use it behind? Can I have access to code generated to custom behaviour?
@technovangelist10 сағат бұрын
For that you would need to build something yourself. This is for web based apps that you want help building
@zooloomuzik11 сағат бұрын
Hey Matt, I was really excited for this video, then only to realize theres no mention of Ollama in it!?!? I have a project on the go where I'm trying to build a multi container app using docker-compose where the containers are backend- fastAPI, frontend- Nextjs and llmServer- Ollama. I'm running into problems having the backend connect to the Ollama server ... I get the dreaded [Errno 111] Connection refused
@technovangelist10 сағат бұрын
This was 100% about using tools with ollama and docker.
@technovangelist10 сағат бұрын
Ahh. I see how you can think that. But I wouldn’t use ollama in docker anyway. This is just about the UIs.
@zooloomuzik7 сағат бұрын
@@technovangelist wow, thanks for responding Matt, much respect for what you're doing! considering your comment "... I wouldn’t use ollama in docker ..." might I be so bold as to ask... if you were me, and you needed to host this app on Azure (which I do) how would you go about hosting Ollama?
@technovangelist5 сағат бұрын
Got it. That makes sense. Docker on a host vs docker on localhost can be different. If you are running their container service rather than an instance then that makes sense. Have you had success with getting access to a real gpu? Last time I tried I could only get their generic named cards and not a real amd or nvidia card.
@mpesakapoeta12 сағат бұрын
Any tutorial on model creating from custom data,like pdf s? Like for companies?
@mpesakapoeta12 сағат бұрын
Please share the link of the video for reducing the model size for specific tasks, example, only weather, is wouldn't need the hole context for this
@technovangelist11 сағат бұрын
You would be able to fine tune for that but it wouldn"t reduce the size. Reducing the size would be a very expensive process.
@matrixview15 сағат бұрын
1973
@ISK_VAGR15 сағат бұрын
Man. I love it. I already subscribe to it. Something I really will be crazy to know is how to story my ollama local models in an external hard drive in Mac. As you know macs doesn't have much space. So, i bought a special hard drive that runs at 40G/sec to have models and other stuff and I will love to have the models in there than in my internal hard drive. Thanks for the great content and explanations.
@technovangelist15 сағат бұрын
They don't have much space?Sure they do. Mine has 4TB inside. But you can use the OLLAMA_MODELS environment variable to start storing them elsewhere.
@robwin007215 сағат бұрын
Matt, I do not see COBOL (of any flavor) especially for PCs; e.g., MicroCOBOL and or GnuCOBOL 😢😢😢 Which Ai Code Assistant can GnuCOBOL? Is LaTeX not considered a programming language? Or is LaTeX considered native to all Ai entities?
@technovangelist15 сағат бұрын
My only suggestion is to try it. Try each model to see what supports it.
@robwin007214 сағат бұрын
@@technovangelist it’s good to see ADA on that list. I had three RFC adopted in the early 1980s and it was my primary language at NASA in the early STS project.
@fabriai15 сағат бұрын
Wonderful video, Matt. Thanks so much for sharing this.
@derekf111116 сағат бұрын
I love your videos! Your explanations are amazing, thank you!
@MichaPiotrKozowski17 сағат бұрын
Thank you!
@AliAlias18 сағат бұрын
🙏🙏🙏Please how to add vision model MiniCPM-V 2.6 to ollama? openbmb/MiniCPM-V-2_6-gguf
@jimlynch939019 сағат бұрын
I'm really enjoying this series. Thanks.
@JNET_Reloaded19 сағат бұрын
what location to run that download hugging face model command? and where does it download to? same location as the others wheres that?
@JNET_Reloaded19 сағат бұрын
any1 wana swap code for tokens?
@imperativeemarketing554620 сағат бұрын
I am very interested in FABRIC, and I will probably get into a serious use of it. I think it would be very useful if you create a video taking one or two of the patterns that may not be configured to your exact use case and demonstrate how you would modify them for your use case. This type of use case from an experienced Generative AI Aficionado can be very helpful for more novice users. Thanks for all your videos. I watched a lot of them and I will watch a lot more.
@harinaren198921 сағат бұрын
is there a way i can fetch the whole response in a single response object
@technovangelist16 сағат бұрын
Sure. Set streaming to false
@spacekill21 сағат бұрын
"If, for example, I have more than one model downloaded, and one is chat, another is multimodal, and another generates images, can I make it so that Ollama chooses which model to use based on a prompt, or does it by default use the one you've chosen with the `ollama run` command?"
@technovangelist16 сағат бұрын
It doesn’t do that. But you could build an app that does that.
@spacekill14 сағат бұрын
@@technovangelist ok . 100 Thanks
@shoelessone22 сағат бұрын
Hey Matt! Off topic comment but I guess I'm feeding the ol' KZfaq algorithm anyway! I haven't watched your entire backlog so apologies if you've already covered this, but I'd love to see some content / videos on the following topics: 1. How can you use ollama in a production environment. Topics around infrastructure, reasonable techniques (e.g. handing off processing to async jobs when possible), cost, etc. I'm not sure how common this use case is but I am evaluating using something like llama 3.1 to help summarize some potentially very large text files and weighing cost differences between using something turnkey like openai's APIs vs figuring out hosting myself (well my company. There seems to be a lot less on production hardening some of these open source models (or I just haven't been paying attention!) 2. A "state of the union" high level overview of the options available to software developer new to using AI. This you have covered in a lot more detail in various forms, but an overview of what tools are actually at a persons disposal in terms of trying to use AI to solve some problem. When I first started looking at this stuff I thought the only options I had were buying a bunch of super computers to train models and learning a lot about doing matrix multiplication. But we have RAG, we have "fine tuning", we have modifying system prompts... a sort of high level overview of what a layperson can do, and perhaps where reasonable off-ramps for more advance use cases are would be super helpful (i.e. when do I need to brunch up on my linear algebra? :)) Thanks for your work!
@aaronag787622 сағат бұрын
Outstanding video. As a complete newbie this was intriguing, well explained and engaging. New subber and bell set
@artur50Күн бұрын
Excellent content Matt! Congrats! Keep on going.
@marianoarganarazКүн бұрын
I love the way you explain. Thanks
@PBrioschiКүн бұрын
Hi Matt, thank you for another amazing content. I'm working with ollama and other tools available from community to develop some solutions for my company. I need some help from a professional consultant for this job. Could you work with me, or, maybe, recommend a person who can help me to do it?
@sammcj2000Күн бұрын
I wouldn’t recommend creating models the legacy Q4_0 quant types, they’re depreciated and are worse quality than K quants (or IQ if you’re running with CUDA)
@aimademerichКүн бұрын
More cool stuff please!
@NLPprompterКүн бұрын
ah... the ollsma serve... LOL i was wasted a week until i realized it was user issue in Linux, i felt so stupid having duplicate models, and things... this is really good video any one new to ollama should watch this if i watch this before i wouldn't waste a week just to realized how stupid i am the simple user issue...
@vulcan4dКүн бұрын
Removing models is the most annoying part because you have to name it exact. Wish they made it easier to just select and delete via GUI or list and select to remove by a number
@technovangelistКүн бұрын
That’s one reason I love gollama. I have a video about it.
@shunteraКүн бұрын
Love this, I think we’ve all been doing just-in-time learning to run and keep up to date with what’s happening every couple of weeks. Great to tear it back to the foundations Matt
@k225Күн бұрын
I use the Voice Recorder on Google Pixel which does on-device live transcription with punctuation, and keeps track of different people speaking. I always assumed something like that would be built in on Apple devices
@technovangelistКүн бұрын
It is. But this is better
@federicoarg00Күн бұрын
this is amazing, super clear, thank you!
@romulopontual6254Күн бұрын
Very nice! Thank you.
@ABatorfiКүн бұрын
Thank you for this awesome course, I‘m enjoying it!
@lisagrace4114Күн бұрын
Awesome, thoughtful video, Matt! And you have an amazing voice--you should license it to ElevenLabs. 😁
@dikranhovagimian87702 күн бұрын
I was confused about function calling as I could not figure out how the model was calling a function. The model was just returning json formatted data. Thanks for the clear and concise explanation.
@rayfellers3 күн бұрын
Using this information, while interesting, with all of the caveats mentioned in getting this to work there's nothing easy about it.
@technovangelist3 күн бұрын
this is about as easy as it gets to create new apps that leverage AI.
@roccov19723 күн бұрын
I've been waiting on the CrewAI waitlist since April (it's August).
@technovangelist3 күн бұрын
They don’t have a waitlist unless you want them to host it.
@roccov19723 күн бұрын
@@technovangelist Thank you for that info.
@em22en3 күн бұрын
Loved the hints to choose the best model for the problem you want to solve
@em22en3 күн бұрын
Thanks Matt! Very helpful!
@mpesakapoeta3 күн бұрын
How can i download a model in .gguf format locally,my reason is am transferring the model to a computer being used remotely in a health facility with no phone or internet network.
@technovangelist3 күн бұрын
You want to dl the model from hf? And then add to ollama? Or you want to do with ollama then transfer to a different computer? Ollama uses gguf but I don’t understand exactly what you want
@hasaanirfan60734 күн бұрын
Great ❤
@zulteonka4 күн бұрын
thank you. if human inteligence is an ai in some sense i am definetly an insane model :D :D
@Sundaysun96334 күн бұрын
i am not good in programming ,but this video help me start my local RAG easliy
@CrazyTechy5 күн бұрын
Matt, thanks for your content. Is there an Ollama model that you can use to check for plagiarism? I am creating short articles using ChatGPT. Another question. Is there a command that can interrupt llama3.1 while it’s outputting an answer? /bye doesn’t work.
@technovangelist5 күн бұрын
Ctrl c will stop.
@technovangelist5 күн бұрын
I don’t think a model will check but that seems a good use for rag. Do a search for similar content, chunk it up and your comparison article. Then similarity search. If it has a bunch of chunks very similar to content in any one other article it would be another piece of evidence pointing to plagiarism. But it might still need some assessment to figure it out for sure.
@CrazyTechy5 күн бұрын
@@technovangelist Matt, I now understand RAG and how you can use it to extend an LLM, but I won't be able to implement your very good idea. But, I see how you think--deep tech. So, what do you think about Grammarly? It will check text, and it's just $12 a month. When I graduated in 1973, they only had mainframes. I worked for Chrysler (MI Tank). And worked with Madonna's father, Tony Ciccone.
@technovangelist5 күн бұрын
I used to use grammarly until the company I worked at banned the use of it for security issues.
@CrazyTechy5 күн бұрын
@@technovangelist OMG. I will need to do a search on that. I worry about my solar powered WiFi camera I bought from Amazon and that WiFi power adapter my wife uses to activate our coffee maker in the morning. Thanks.
@MarcioLena5 күн бұрын
Another great video! Thanks
@leendert165 күн бұрын
Thanks for the content. Can't wait to put them to use.
@JaredVBrown5 күн бұрын
Such a calm, yet punctual voice. Like ASMR + tutorials. And the content is easy to follow and understand. Great Channel!
@Aristocle5 күн бұрын
1:30 it's a similar problem with LateX(latechi). But, in linguistics you cannot use different alphabets to write the same word. So I pronounce latex as the material.
@sammcj20006 күн бұрын
Naming things is hard 😂
@jamesmcintyre6 күн бұрын
Awesome video! By the way, in general your format, pacing, conciseness and the cool but also utilitarian "code floating above your speaking" is a nice touch! I know making these videos takes a lot so just want to give kudos where it's clearly deserved! There's so many "faceless", auto-gen'd content or just click-baity low-content videos so this was a welcomed break from that!
@user-jg4ci4mf8w6 күн бұрын
Hi @Matt Williams if time permits, can you create a video showing how to deploy Dify to a cloud service like Render?
@SlykeThePhoxenix6 күн бұрын
It's not pronounced "en-gin-ex", it's pronounced "en-gin-chi".
@technovangelist6 күн бұрын
That is definitely en gin ex. Has been since it started out in Russia. I have spoken many years at nginx conf.
@srivatsajoshi40286 күн бұрын
@@technovangelist I think it was probably a joke
@SlykeThePhoxenix6 күн бұрын
@@technovangelist I know haha, I was being facetious XD.