The FASTEST Way to run Kubernetes at Home - k3s Ansible Automation

The FASTEST Way to run Kubernetes at Home - k3s Ansible Automation - Kubernetes in your HomeLab

Рет қаралды 135,848

Күн бұрын

Setting up kubernetes is hard. That's why we made it easy. Today we'll set up a High Availability K3s cluster using etcd, MetalLB, kube-vip, and Ansible. We'll automate the entire process giving you an easy, repeatable way to create a k3s cluster that you can run at home in a few minutes.
This is a great way to set up Kubernetes in your HomeLab.
Video Notes: technotim.live/posts/k3s-etcd...
Support me on Patreon: / technotim
Sponsor me on GitHub: github.com/sponsors/timothyst...
Subscribe on Twitch: / technotim
Become a KZfaq member: / @technotim
Merch Shop 🛍️: l.technotim.live/shop
Gear Recommendations: l.technotim.live/gear
Get Help in Our Discord Community: l.technotim.live/discord
2nd channel: / @technotimtalks
A HUGE THANKS to our sponsor, Micro Center!
New Customers Exclusive - Get a Free 240gb SSD at Micro Center: micro.center/1043bc
You can find the repo here
github.com/techno-tim/k3s-ans...
Videos mentioned:
Ansible Setup Video
• Automate EVERYTHING wi...
Proxmox Cloud Image Video
• Perfect Proxmox Templa...
Thank you to Jeff Geerling for the video clip and his work on Ansible + K3s!
• Raspberry Pi Cluster E...
Thank you for forking and adding kube-vip support to the k3s Ansible Playbook!
github.com/212850a/k3s-ansible
This will also be included in the documentation
l.technotim.live/docs
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 - Intro
00:58 - Micro Center Advertisement
01:49 - How Did I get here?
03:28 - What did I automate with Ansible and k3s?
04:23 - k3s Load Balancers and Architecture
05:58 - What is the architecture
06:06 - Creating machines
06:36 - Configuring Ansible Playbook
11:03 - Setting Hosts
11:34 - Running the automation
12:31 - Verifying the k3s install
13:31 - Deploying a workload to k3s
13:53 - Deploying a Service + Load Balancer
15:25 - Testing kube-vip and k3s control plane
16:31 - Tearing down k3s
17:15 - What problems did we solve?
18:09 - Stream Highlight - "If it's not DNS, it's a firewall rule"
#k3s #Ansible #OpenSource
"Mime Dance" is from Harris Heller's album "Arcade"
"Wrong Way" is from Harris Heller's album "DANSU"
"Sunday Pancakes" is from Harris Heller's album "Low-Key"
l.technotim.live/sb-music-lic...
Thank you for watching!

Пікірлер: 321

@fahadysf 2 жыл бұрын

This setup is pure gold. I can't thank you enough, within a day I've understood how MetalLB is the LB alternative for self-hosted / bare-metal kubernetes deployment and this playbook has saved me many many expensive hours which would have been needed to get my test lab up. Can't thank you enough!

@TechnoTim 2 жыл бұрын

Thank you!

@bradwilson3766 Жыл бұрын

I second this! This made me join as a member.

@chrisdelucatube 7 ай бұрын

Inconceivable!! This worked really well right out of the box as promised. I had a k3s 3-node Raspberry Pi cluster up and running in minutes - and I love the Ansible add in. I was vaguely familiar with Ansible from a introduction about a year ago, but this took my understanding to a whole new level. Thank you very much!

@DanteS-119 2 жыл бұрын

Woah... I really gotta go through this video and apply it to my setup, was totally trying to do this the other day. Ended up without it being automated. Elegant and beautiful solution man

@conorkeane 3 ай бұрын

Thanks Tim! Got a technical interview on Tuesday and you've just helped me prep for it!

@jpb2085 Жыл бұрын

So so awesome, just tried this out and works so well. Thanks for the supporting documentation as well!

@angelgonzalez2379 Жыл бұрын

Wow I set up this cluster having almost no idea what to do with it. After setting up the cluster I relied on various blogs to get services running. I'm now at a point where I've set up services using only docker documentation, docker-compose files, and Kompose. My latest project has been delving into using BGP on metallb to be able to direct traffic from certain pods to my VPN. Thank you so much Tim!!!

@ryancoble8776 Жыл бұрын

I absolutely love you. I was following all your old stuff and just beating my head into my desk trying to get it all to work right for my situation. Thank you so much for this. This needs to be the first result when you search KZfaq for k3s setup, for real.

@TechnoTim Жыл бұрын

Thank you!

@syedmwma 2 жыл бұрын

Thanks! This will help spin my raspberry pi clusters. Not having to use the external load balancer for kubernetes is awesome! Thanks again.

@johnjbateman 2 жыл бұрын

I love that you OSS guys are using each other’s work and shouting each other out. Stream on!

@unijabnx2000 2 жыл бұрын

As someone whom has been working on deploying an OVA (including application setup after the vm deploys) with ansible... i can appreciate how much work you've put into this.

@TechnoTim 2 жыл бұрын

Thank you! I am standing on the shoulders of giants who have built most of this out before me!

@kgottsman 2 жыл бұрын

Really killing it content wise... Your videos have been so helpful lately.

@TechnoTim 2 жыл бұрын

Thank you!

@Nighteater333 2 жыл бұрын

Hello from czech republic, thank you for your work, I am so glad that i have found your channel.It's very helpful.

@Alex-xb9qc 2 жыл бұрын

Dobry den :^)

@SpadeQc123 Жыл бұрын

Amazing vid as always Tim! I did the same setup but with a debian cloud image instead and it works great

@annusingh4694 Жыл бұрын

Can you please share more? Did you set it up remotely?

@suikast420 2 жыл бұрын

Great talk dude. I am exactly on this. I want provide a full secured k3s cluster for airgapped environments ( for industiral production for example ) . The final setup should like this: 1. Private registry with SSL setup 2. Provide docker on a builder node for remote builds 3. Sec Stack 3.1 Cert manager 3.2 Keycloak as ODIC provider 4. Monitoring Stack 4.1 Grafana 4.2 Loki Currently my repo is private beacaue it is in dev. After my first relase will share with you. Maybe can do more together ;-)

@willdrumforfood7371 Жыл бұрын

This is a super helpful video, thank you for putting this together! It would be great to have a followup where you add in some applications or perhaps even a container of one of your own apps. Thank you for all these great and helpful videos!

@TechnoTim Жыл бұрын

Great suggestion!

@thetruth3107 Жыл бұрын

I agree with OP, full prod WordPress / email server would be great...

@spikeukspikeuk 5 ай бұрын

Hey Tim. I did see this some time ago and tried to run it but had some issue. Don't remember what it was now so went on the back burner. Just updated the repo and run and works perfect. Lots of thanks for this.

@tcasex 5 ай бұрын

I just got my environment setup with ansible and docker compose files and then I run across this....lol. I need a break before I embark on this journey and just enjoy the homelab. Sometimes I think I enjoy tourting myself.

@PCMagikHomeLab 2 жыл бұрын

Great job like always TIM :) thx for all you hard work!

@IcyTone1 7 ай бұрын

Thank you so much for your assistance in setting up k3s using Ansible. Could you possibly create an updated video on how to install Rancher along with Traefik + cert-manager? Additionally, could you demonstrate how to use this k3s cluster with a GitLab CI/CD pipeline? It would be of great help.

@jacksmart2643 2 жыл бұрын

This is an awesome video. I have understood how HA and K3S works, but never understood how you could access the webserver from a single IP. Keep up the awesome work!

@TechnoTim 2 жыл бұрын

Thank you!

@kaelwang1251 Жыл бұрын

WOW, you make really good content, detail and well explained, thanks.

@prabhujeeva2228 2 жыл бұрын

Awesome Tim!, Thanks for automated the entire process

@TechnoTim 2 жыл бұрын

Glad it was helpful!

@thiagobarrichelo 2 жыл бұрын

Hi Tim thanks for sharing definitely very helpful and great work there! Just forked your repo as I need my CNI to be Calico instead of Flannel. Thanks a lot!

@MrToup 2 жыл бұрын

Thanks for the video. I started the journey with kubernetes for my Homelab thanks to your videos. I ended to the same results. Having it automated. I use the pretty good template from k8s-at-home. They have All setup including sops and flux.

@rubenkhachaturov3309 19 күн бұрын

This is fantastic! Thank you so much!!!👍

@iga3725 Жыл бұрын

A masterpiece!! Thx for sharing

@christiandassy8128 2 жыл бұрын

Keep up man your channel is just excellent

@lechaldon Жыл бұрын

Mate, thanks for this, now I need to go figure out how you wrote this playbook so I can understand how it all operates and how k3s works. I plan to migrate my entire docker-compose stack to HA k3s and this is perfect. Thanks again!

@TechnoTim Жыл бұрын

No problem 👍

@RicardoWagner 4 ай бұрын

Great job Tim... would be great the following follow ups: Rancher install on this cluster and how about some longhorn? Cheers

@troybrocato 8 ай бұрын

This truly is pure gold. The only thing i would add to this is to also have FORKs for different hypervisors. Ansible is very friendly with all hypervisors and can create the K3s VMs automagically.

@TechnoTim 8 ай бұрын

Thank you! This is hypervisor agnostic and even works with bare metal!

@myblindedsoul 10 ай бұрын

Hardware Haven sent me to a real expert 👍

@TradersTradingEdge 2 жыл бұрын

Super awesom, thanks Tim. 🖖

@zoejs7042 2 жыл бұрын

Tim, great work here. I'd really like to show you a similar way I did this with custom k3os images, proxmox and terraform.

@chrisa.1740 2 жыл бұрын

I find this interesting. I have been working on a combination of Terraform and Ansible to spin up a k3s cluster on the Oracle Cloud Free Tier. Would love to see your ideas on how to get this working.

@TVfen 2 жыл бұрын

Me too. I'm working with a weird cluster of raspberries, x32 laptops, and an x64 mini-pc, with proxmox, K3s, terraform, cloud-init, and ansible. And I'm interested on your project too, could you leave us a link to your project? (github, blog, even a google doc could be good) Thanks!

@TechnoTim 2 жыл бұрын

Thank you! Sounds awesome. I’ve made this to be a building block that can fit into any infra automation ☺️

@zoejs7042 2 жыл бұрын

@@chrisa.1740 okie, i'll make a video outlining how i did it :)

@saltandsham 2 жыл бұрын

@@zoejs7042 Sounds good

@islameldemery 2 жыл бұрын

It just worked from the first time! complete awesomeness!!

@TechnoTim 2 жыл бұрын

That's awesome news!

@dnldnl4880 2 жыл бұрын

Amazing as always

@dominick253 8 ай бұрын

My issue was the wrong lan network! Mine is a 10. Network not a 192. The three masters would work and join but it'd hang on joining the agents. Changed everything including vip and flannel ip ranges to my lan and it worked like a charm! Also I was using -K for become password but I tried it without that and it worked. Hope this helps anyone out who may have the same problem. Thanks for the work to get this going!!!

@gibransvarga8487 8 ай бұрын

did you make k3s api accessible from the internet?

@mitchross2852 2 жыл бұрын

There are a few assumptions in this video that a beginner will bang their head against the wall for days. I finally got this all working. This is awesome.

@TechnoTim 2 жыл бұрын

Nice work!

@TechnoTim 2 жыл бұрын

Out of curiosity, what were they?

@mitchross2852 2 жыл бұрын

@@TechnoTim If open for discord DM's ill send you a friend request. I have some general feedback I think could be used to help your channel/others troubleshoot. Else I can comment here, let me know!

@jpconstantineau 2 жыл бұрын

Great work! Have you looked into gitops with flux or argocd? I find it quite useful to simply push to git and have the cluster pick up the manifests and deploy them automatically. The first thing I do after installing a vanilla K3S install is to connect the cluster to a git repo (using flux) and send all my manifests by pushing to Git. The cluster automatically configured itself, including MetalLB. That makes it really easy to tear down a cluster and build it up again.

@TechnoTim Жыл бұрын

Yes, I have! I've done a video on flux! kzfaq.info/get/bejne/hqx8nNCBzZuRp6M.html

@webwarriorc4683 2 жыл бұрын

Yay, This video helped me learning ansible, it feel really good to make everything automated XD

@TechnoTim 2 жыл бұрын

Thank you!

@coletraintechgames2932 2 жыл бұрын

DUDE! I have not watched this video... But this seems exactly like a video I have needed for ever.... 4-ev-er. Your my boy blue!

@testdasi 10 ай бұрын

Just want to provide a testament to how good and useful Tim's work is. I messed my K3s cluster pretty badly so decided to reset from scratch. 15 minutes and 2 commands ("ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini" and then "ansible-playbook site.yml -i inventory/my-cluster/hosts.ini") and I have a fresh start. Another 15 minutes of "kubectl apply -f" to reinstate my deployment yamls and everything is back to its original working state. Thanks a lot mate! 👍

@TechnoTim 10 ай бұрын

Glad you liked it! Thank you!

@mikkel3135 2 жыл бұрын

Automated K0s and RKE2 ansible deployments the other day with some pretty barebones playbooks. It's kinda fun trying to automate and architecture everything. Want to get to a point that I can just setup a new installation of Proxmox using ansible, and have it create VMs and a cluster (or join existing).

@Subbeh2 Жыл бұрын

Love your work. Just managed to set all this up, but I'm still clueless about how to use it. Would be amazing if you could do a video on how you're using and deploying your stuff on this cluster. TA

@TechnoTim Жыл бұрын

Thank you! I have tons of videos on kubernetes apps

@peterkleingunnewiek5068 3 ай бұрын

The k3s cluster runs very smooth, thanks for your effort Tim. ik only can't install rancher on it. Everthing else works great. But Reancher does the hole cluster crache. the expose of metalLB ip to rancher Works. but the installation never ends before the cluster crashes. I looked different video how to instal rancher on a k3s cluster but it never works

@NerdzNZ 6 ай бұрын

O.M.G this was amazing, in a single night. I setup a Ubuntu Server cloud init template in ProxMox, built 9 VMs (3 masters, 6 workers) and ran though this video to get a fully HA k3s Kubernetes cluster installed. The best part, I am a freaking n00b at all of this. Such a great teacher, love your work and I am looking forward to consuming more of your content.

@TechnoTim 5 ай бұрын

Nice work!! Thank you!

@hopecomingsoon Жыл бұрын

Welp, you've done it now, Tim. Great job!

@hotrodhunk7389 8 ай бұрын

So quick! Only spent a week to get it to work in a few minutes 😂😅😂

@HootanHM 6 ай бұрын

👏👏👏 it'd have been amazing if you could make a video on how we can run Hadoop and pyspark on top this kube cluster to have some data transformation at home 🤩

@docteur3805 2 жыл бұрын

thanks you for this video ! 😊

@rickhernandez2114 Жыл бұрын

dude!! I'm using Rocky Linux and this installed like an absolute dream. I'm ready to stop having all these pets in my homelab. Time for the cattle. Thank you

@canishelix6740 Жыл бұрын

Really appreciate this video. I definitely need to research your blogs and understand them, I know what I want but the order of execution eludes me. I've got a HA SQL cluster already (so want to use that instead of etcd), I do want Longhorn, and Rancher and Traefik 2.... if I'm right I can just add the datastore param to the global_vars and it should use that SQL db, but how do I stop it installing etcd? And I'd assuming the best order of events would be the ansible playbook, then longhorn, rancher and traefik 2 (in that order)... as for cert-manager.... I guess between longhorn/rancher?

@Crimson_Tinted Жыл бұрын

I use that helm installer CRD that k3s offers and just have Ansible drop a yaml file in the respective directory to install kube-vip, personally. This approach of yours is equally valid but one lets me use the stock upstream module which is nice. It also lets me install my CD of choice (Argo in my case, saw you had a Flux guide too), and I just drop everything else to install including MetalLB into an set of app-of-apps Argo CD Applications. I find I prefer k3s only handling absolute minimum to make the control plane HA to be the easiest strategy for me and then let my CD system take it the rest of the way.

@emilhuseynli Жыл бұрын

Hi, first of all thanks a lot for such a great tutorial! Can you please elaborate why the netaddr dependency is needed? where exactly is it used?

@jamesajohnson82 2 жыл бұрын

I have 8 old Mac minis that I have been working on to make into a K3s cluster using rancher, Ubuntu 20.04, and a ton of trial and error. I am just about to spin this up, but should I scratch that and go with Ansible? Dang, as soon as you think you have a grip on something, someone awesome like TechnoTim comes along and throws a new solution right at you. Thanks for all the great videos!

@TechnoTim 2 жыл бұрын

Thanks! Haha! This is automating what you would otherwise copy and paste from docs and adds load balancers so you don't have to. :)

@diamantin55 2 жыл бұрын

Great video as usual Tim. "I love open source". It would be great if you add LongHorn support at that script. Additionally, it would be great if you can do a video on how to migrate a docker install that already have some data on a local volume to kubernetes... Thanks man for your videos!

@TechnoTim 2 жыл бұрын

Thank you! I’ve considered adding longhorn and rancher to the script but many may not need it. My other videos shows how to install these with a few commands! Will consider it in the future!

@junejuan8561 2 жыл бұрын

Wow! Another solid content! Thank you very much. Is it possible to use rke2 instead of k3s?

@TechnoTim 2 жыл бұрын

Not with this playbook, but one might exist

@roboto_ 2 жыл бұрын

thansk so much for doing this, i started working on this exact problem like a year ago but had to shelve it because i didn't have time anymore :( thanks so much!!!!!!!

@TechnoTim 2 жыл бұрын

Thank you! Happy to help!

@enkaskal Жыл бұрын

Thanks for the vid and appreciate you publishing your repo! :) Very helpful and I was able to use them along with k3s-ansible upstream, Lempa's vid, and the k3s docs to pull it all apart, figure it out, and get my own k3s setup codified. However, I skipped all the Metal LB as I found it trivial to get kube-vip to work as the load balancer for both the control plane and services. Curious as to what you got caught up on?

@chrisa.1740 Жыл бұрын

Would you mind sharing your setup that uses only KubeVIP? I would like to compare that with this, on my way to modifying the Ansible playbook to use Traefik for these tasks if possible.

@roeidalm 2 жыл бұрын

Gear video! Thanks! About deploying Traefik. Can you explain or point to an article that show how to set the traefik to work with k3s?

@JohnWeland 2 жыл бұрын

Great video sir!

@raymondvanderwerf 2 жыл бұрын

Can't wait to watch this one!! Saved...will watch and probably have a go at this coming Monday Thx Tim!! ✌️

@ryanarnold2293 2 жыл бұрын

Thanks Tim! I was able to get my cluster up and running easily with this. Question, I installed Rancher and now need to access the UI. How can I config the nginx ingress to route to the Rancher UI?

@MartinHiggs84 Ай бұрын

Watching again 😊 any chance of one with rancher/longhorn added?

@jeffreyurlwin6212 Жыл бұрын

Awesome Tim -- just setup on 4 raspberry pi CM4s using Deskpi (need 2 more pis to fill!!!). I also have some NVME storage (overkill, I know), so I put k3s & containers on the storage instead of the emmc for speed/safety. I plan on watching your video on Longhorn to do that next! Huge Thanks! at the end, you had a reboot script -- I couldn't find that - and checked, i don't see an obvious project in your github repos where it may be. I wanted to use that as a basis for a reboot script. Thanks!

@jeffreyurlwin6212 Жыл бұрын

sad. replying to myself. :( I found it in launchpad! Somewhat anti-climactic in that it was / looks sooo simple. I think setting serial to 1 should work. :) testing time :)

@eldaria 8 ай бұрын

Although this setup was great and worked really fast, I don't think I want to use it right now. The reason is that I won't learn anything from it. So if something goes wrong or I would like to tweak something, I would not have any clue how to do it. For example I noticed in the settings that there was an entry for setting up bgp and ASN, since I use pfSense I figure that would be a great way of getting the routing for the K3s cluster working. But no matter what I tried I could not get it working. But it has inspired me to actually start learning Kubernetes and start building my first k3s cluster from scratch. It would actually be great if you would break down in a more in depth video or series the different parts that you used to get this running and options one could use to tweak things. Because it is really cool how fast it goes to get the cluster up and running.

@mitchross2852 2 жыл бұрын

Is there a follow-up video for rancher and how to install k3 apps? being that this is HA/Vip Etc it would be good to have a video on how to utilize all of this to deploy trafik, maybe pihole/adgaurd HA, etc. I know you have some topics on this already, but I don't know if they still apply with this setup. If they do can you link to the next proper video for rancher, HA adgaurd/pi-hole, etc...

@xandercode 2 жыл бұрын

12:50 "Super, super awesome" Tim's smile says it all. 😁👍Well done just downloaded to try out now. Thanks Tim

@TechnoTim 2 жыл бұрын

Thank you so much! I am glad you liked it. Let me know how it works out!

@tommsla123 Жыл бұрын

Helpful video Tim. Thank you so much. I have a question : Is this prod ready or for testing only ? Should I adjust some params for prod deployment ?

@TechnoTim Жыл бұрын

it's prod ready!

@tommsla123 Жыл бұрын

@@TechnoTim thanks

@northcode_ 2 жыл бұрын

Hey Tim. I've been looking for a while now. Are there any load balancers for k3s other than klipper(svclb) that work over a layer3 VPN like wireguard? I've tried getting both metallb and kube-router working, but they won't route between nodes that are only accessible across the VPN. Probably because there's no layer2 connection between the nodes, only layer3. I'd love if there was some way to get metallb or some other lb working that can assign VPN-internal addresses to services. I'm working around it now with klipperlb and just using different ports and network policies for my internal services but it's not optimal.

@MonkeyD.Dragon 2 жыл бұрын

Please add Rancher to this

@coletraintechgames2932 2 жыл бұрын

Great idea

@JohnWeland Жыл бұрын

This! I wonder how this works in Rancher. Your preconfigured nodes, were they created in rancher or by hand?

@SuperMati9999 Жыл бұрын

First of all thank you for the video and the code. I have one question, i was trying to set ha kubernetes until i saw this video. I was doing this setup with some VPS. My question is if i have to create a net between them to run this ansible playbook. That was what i do configurating ha kubernetes. In the other hand, ther is a way to do it without a VPN? What VPN would you recommend to me, i was using wireguard with 2 servers (one master) and other the nodes as clients. This because i need to create a virtual ip and that i think that is only posible with a /24 net. Thanks for reading, sory if you dont understand my problem at all.

@ToGoMania19 10 ай бұрын

Thanks!

@TechnoTim 8 ай бұрын

You bet!

@MoviesAndTraillersTZ 9 ай бұрын

Hardware heaven sent me

@peace2941 2 жыл бұрын

Thank you so much, As a beginner, I was expecting to see how you add Traefik and configure it to proxy requests to the example service you had.

@TechnoTim 2 жыл бұрын

Thank you! I have docs on Traefik, I might break it out soon into its own playbook because not everyone will use Traefik!

@MrPatrickberry 2 жыл бұрын

@@TechnoTim Second this. Looking forward for Traefik install with Helm guide

@phk-r 2 жыл бұрын

Truly awesome, as always Tim! Im wondering, 5 nodes seems to be a bit much for my server, CPU spikes. If I want to try this with only 3 servers, is it enough to remove nodes from the hosts.ini? Or is there taints on the servers so they cant act as workers? Had a look, and I think it should work, but confirmation wouldn't hurt :) Trying to deploy rancher right now, but it seems to be running into some issues. Takes forever to deploy, thought it might be because I removed the nodes..

@TechnoTim 2 жыл бұрын

Thanks! All you need to do is update the host.ini and put them under the right header!

@phk-r 2 жыл бұрын

@@TechnoTim yeah got it to work by removing all hosts from the node header. But k3s is really killing my CPU man. Im running a Xeon E3-1226 v3, only 4 cores, but still. If I add 3 server nodes, with 2 cores each, it really struggles. Sure thats over provisioning, but with only Rancher as load, I thought I would work. I dont see how IoT devices would survive running k3s and a workload. What am I missing? :)

@willrnsantana Жыл бұрын

I gotta take some time to debug this to my use case, as the kibe-vip LB is not working o my odroid-c2 (armbian arm64) cluster. But thanks for the hard work to put it all together

@LUISPLAPINO 11 күн бұрын

Hi!! Thank you so much!! This is just a work of art. Dude: how could I add a new node without removing and restarting the existing service?

@EugeneBerger 2 жыл бұрын

Amazing work, Tim! I was looking so long for a fully automated solution. Have you found a way to overcome the k3s certificate expiration after 1 year? I know that restarting the k3s service, 3 months before the expiration, will automatically renew it, but doing it on an HA cluster...

@TechnoTim 2 жыл бұрын

Thanks! RE the expiring certs, don't you ever patch and reboot any of these nodes? It should auto renew if the certs are within 90 days the next time k3s starts (I thought)

@jeffreyurlwin6212 Жыл бұрын

@@TechnoTim LOL. I was thinking the same thing. My next plan is to make a safe patching script that walks through the nodes 1 by 1 and makes sure k3s is back alive before moving to the next one...

@ameeno 2 жыл бұрын

can you add zerotier or eireguard clustering? also how about metallb external address detection internal address detection for nodes? finally would it be possible to have pods run in control plane? I want halb but don't want to lose workernodes

@ElliotWeishaar Жыл бұрын

Love the video Tim. I'm struggling to get this up and running on 5 ubuntu 22.04 machines. I've noticed that the args in your video don't match what's in your repo. Any reason for the change or are the original args listed anywhere? Wasn't sure if I should open an issue on GH or not.

@TechnoTim Жыл бұрын

The default args in the repo should work! If you want to see what was used in the video just look at the first few commits, however the latest repo is what you want!

@ElliotWeishaar Жыл бұрын

@@TechnoTim Thanks! I think I found my issue. I'm running ansible from an RPI 4, which was running Python 3.7 and ansible core 2.11.5. I think that 2.11.5 is not new enough for split to work. I'm upgrading and hopefully that will fix it!

@yo_mono 2 жыл бұрын

Awesome, JUST what I was looking for, perfect timing!. Since the playbook works with CentOS, it should also work with Rocky Linux right?

@TechnoTim 2 жыл бұрын

Perfect! It should but please let me know!

@wva5089 2 жыл бұрын

@christian braun I used almalinux. Half the tasks it didn't run(blue/didn't match the centos or redhat facts) but the only one that mattered that I needed to force was the secure pathstuff for sudo. Worked great on alma.. I imagine your experience with rocky would be identical. But honestly I think I'll be making my own playbook that's half the size and use defacto kubeadm instead.

@dustinloring8989 2 жыл бұрын

Thanks for the Videos. I have a 5 node clustered server rack and will be trying harvester out on it. But in order to utilize it do I need to set up K3S on a RPI then deploy workload loads to the harvester cluster?

@TechnoTim 2 жыл бұрын

Set up your clusters first then create vms, then use this ansible playbook to target all machines. Or create the k3s cluster in Harvester and join your Pis to the cluster manually

@dustinloring8989 2 жыл бұрын

@@TechnoTim Thank you for the help cant wait to try it out

@jrdwiz Жыл бұрын

Hi Tim, is there an easy way to add more master nodes with etcd later on? Thank so much.

@subzizo091 8 ай бұрын

thanks for the great video content, but please this setup work with VMware workstation and if it does, what parameters should be changed

@motu1337 2 жыл бұрын

Hey Tim, awesome guide and repo. This helped expand my Ansible knowledge and produce a useful outcome of a k3s cluster with which I can mess around. Question for you, when I deploy Rancher using Helm (following another guide of yours, thanks) it doesn't seem to be accessible externally. Is this a MetalLB and Rancher conflict? Any guides I can look to that would help me resolve this? Thanks!

@TechnoTim 2 жыл бұрын

Hi. Thanks! It shouldn’t be as long as you disabled the service load balancer using the arg

@TwoThreeXray 2 жыл бұрын

Did you happen to find a way to expose rancher? I am currently trying to figure this out as well

@motu1337 2 жыл бұрын

@Tristen I was able to get it exposed using a MetalLB address by running "kubectl expose deployment rancher -n cattle-system --type=LoadBalancer --name=rancher-lb --port=443" that should work for you too assuming you've already got rancher installed and it's running in the cattle-system namespace. I found this on another guide of Tim's on installing Rancher.

@TwoThreeXray 2 жыл бұрын

@@motu1337 I think I just cam across the guide you used! lol Thank you sir for the help! That worked!

@therus000 2 жыл бұрын

@@motu1337 is it work ? without traefik

@andrewjohnston359 2 жыл бұрын

It would be great ifyou could begin tesing these setups in a real world environment. For instance, putting each VM on separate physical proxmox nodes. Then testing both performance and data integrity of the HA mysql/mariadb, and maybe some kind of php web application - and start powering off VM's mid use (while writing and reading lots of data). Any load balancer can show a static nginx page - but when you have fast changing application data tied to user sessions (you'll want redis for this) all of a sudden you realise these magic clusters aren't as magic as they make out. Also sql databses have clustering capabilities baked in (sharding) - and the sql agent itself is aware and in control of ensuring the integrity of the cluster. How do containers in a cluster ensure the database integrity? I personally love the idea of a self scaling, self healing auto magical high performance container cluster - but all you ever see are demo examples that developers show off and then destroy. I guess what I'm getting at is often the applications themselves need to be written to be cluster aware/compatible, and the architecture needs to be manually configured more often than not to make this stuff work - you can't generally just spin up 10 containers with a stock OSS container application image and have it 'just work'

@TechnoTim 2 жыл бұрын

100% agree and this topic comes up a lot, RE scaling. I discussed it on stream today too. Applications need to be written and architected with scaling in mind, most of them too stateful to run more than one. I'd love to dive deeper into more topics like this in the future if there's appetite for it!

@ekekw930 2 жыл бұрын

@@TechnoTim I would love to see you cover something like this

@crazyglue1337 Жыл бұрын

The repo works with raspberry pis (I just test my cluster up!), so you could in theory just like 5 pis and just unplug them from the network stack whenever while the performance tests are running

@TVfen 2 жыл бұрын

By the way, Tim, could you show us some provisioning from scratch. Some bare-metal way to install it all with ansible (and maybe terraform, cloud-init, or something like that).? Your videos have inspired me to start my own cluster of servers (I have low end hardware, but a bunch of it, so I'm trying to make it all work together). Thanks!

@Irish2086 2 жыл бұрын

There is a nice Proxmox Terraform plug in available to provision LXC or VM. Similar to cloud init but using TF. It is not perfect but a nice way to learn TF without having to spend money on cloud provider. I should reach out to him.

@TVfen 2 жыл бұрын

@@Irish2086 Yes, thanks, I saw those before, but... so far, for what I understood (and what I need): - Cloud-init plugin is only a "read" plugin. Meaning, I can read the file, store values read from that file into vars, then use them with other plugins. - The proxmox one, is great for the mini-PC that has proxmox. ☺️ - But for the raspberries, I think is best, not to use any type of VMs (proxmox or ESXI). I know there is a TF plugin for K8s and for docker, but at that point, I think I rather set it up with Ansible.

@TVfen 2 жыл бұрын

@@Irish2086 This is my "set up" path right now. I want to provision all the machines from scratch. But PXE is an awful, unsecure, and obsolete technology. So: 1) Copy Linux image to the SD card. 2) Copy, manually, Cloud-init to the card. 3) Let the Raspberry, boot, install, and apply all the settings on the Cloud-init file (usually some basic accounts, IPs, Hostname, SSH, Firewall, and install Terraform). 4) Download from a URL Terraform tf files, and apply. (These would finish setting the device, with secrets, better security (key-pairs, etc.), Install Ansible. 5) Setup everything else with Ansible.(K8s, images, Jenkins). 6) any future changes on the system, would be applied by Ansible (if it is setting the "apps"), Jenkins (if it is setting the contents of the apps (a website, microservice, webapps, Dbs, etc), TF (if it is the system itself). And that's as far as I have gotten so far! 🤷🏻‍♂️ (With the concepts, but I haven't applied them yet, and still 'designing' the whole system in Terraform/Ansible files.

@TechnoTim 2 жыл бұрын

You can definitely combine them! This is just one building block, a LEGO if you will!

@kjc420 2 жыл бұрын

Now wouldn't it be nice to have hypervisor support, like Proxmox? Spin up VMs on one or more hosts, retrieve the IPs for ansible, then deploy k3s on them automagically... Down the rabbit hole we go!

@ekekw930 2 жыл бұрын

Thanks a lot for sharing, Tim! One question, Is it considered HA when all of this is done one just one Proxmox server? Or is it only HA when also Proxmox has different nodes?

@LampJustin 2 жыл бұрын

It depends on the point of view, the cluster itself is ha but obviously it's not fault tolerant to a Proxmox node failure. He should've at least run the control plane distributed between his 3 nodes. :P

@LampJustin 2 жыл бұрын

And it is soo much nicer to do k8s node updates without downtime. So I think it definitely is worth it

@ekekw930 2 жыл бұрын

@@LampJustin what is worth it exactly? K8s or running this k3s HA install?

@TechnoTim 2 жыл бұрын

Agreed! With Modzilla. This is an HA k3s cluster. If you want HA through and through your server nodes would have to be spread across hosts.

@TheMrDrMs 2 жыл бұрын

For work I use Rancher + ansible for a very similar approach as yourself. As I've replicated my work environment (modified) in my homelab, I'm having trouble understanding the benefits to this over Rancher?

@TechnoTim 2 жыл бұрын

This is just a plain k3s cluster without any management. It's not an alternative to Rancher, it's a Kubernetes cluster than anything can run on, even Rancher itself since it needs a kubernetes cluster to run on. It's more foundational.

@jp_baril Жыл бұрын

Hi Tim, I've discovered and been following your channel since a year and basically watched all your videos. So well explained every time! If I were to try your Ansible script to test things out at a small scale in a first time, would the script work if I were to put the same few IP addresses both as Masters and as Workers? (I know it's not best). Also, one thing I always notice in your video is how many IP addresses you have, more precisely all the different subnets you use. It would be very useful to get a video on the segmentation logic you use. Because in the case of deploying this script, I really don't have a clue on which IP (and ranges) to use so that it does not interfere with other devices, VMs, services, etc. and so that I don't have to redo the whole deployment in the future. Thank you.

@TechnoTim Жыл бұрын

Thank you! Yes, you can have all nodes use all roles! About networks, yes! Coming soon!

@jp_baril Жыл бұрын

@@TechnoTim Thank you for your answer! Can't wait to watch it! Continue the good work!

@lightyagami0ben Жыл бұрын

awesome video. Can this setup also upgrade k3s cluster versions without downtime to apps? (I currently use system upgrade controller for that)

@lightyagami0ben Жыл бұрын

not sure if the automated setup can also create a specific version (based on user input) of the k3s cluster?

@grocerylist Жыл бұрын

I keep getting an error for my masters when attempting to provision the cluster. Something about 'no usable temporary directory found in /tmp, /var/tmp, /usr/tmp or my home/user directory'. The directories exist, not sure what it means they're not usable. I tried pasting the full error here but my post keeps getting deleted. Any idea what might be causing this and how to resolve it?

@JhonnyXpress 5 ай бұрын

I’m thinking on using this model but for 2 mini servers

@rodrimora Жыл бұрын

It may be a silly question, but deplying with ansible is an alternative to rancher? I don't fully grasp the concept of both

@TechnoTim Жыл бұрын

This is a way to bootstrap a k3s cluster. You then can install rancher on it if you like

@hb-cloud Жыл бұрын

This is great content but could you tell me how could i create a k3s cluster with Cilium cni using this setup instead of flannel ?

@djonkoful 3 ай бұрын

What do you think about talos kubernetes ? thanks

@TheJimNicholson 2 жыл бұрын

Why did you go with metalLB over klipper, which comes with k3s? Was it just preference, or were there specific reasons you chose to deploy your own load balancer?

@TechnoTim 2 жыл бұрын

I prefer MetalLB over klipper. I've used it many times outside of k3s so it's familiar and battle tested.

@gustavoganso Жыл бұрын

Will you do a follow up video on how to set up rancher on this fresh deployed k3s cluster without integrated traefik? Your "High Availability Rancher on kubernetes" misses some details as far as I get it 🙂

@TechnoTim Жыл бұрын

I don't unfortunately, that video should cover it! be sure to use the documentation too when following that video!