I automated EVERYTHING in my HomeLab with Kestra

Рет қаралды 27,078

Күн бұрын

In this video I will show you my ultimate solution for automating everything in my HomeLab, where I integrate all my favorite automation tools and workflows, including Ansible and Terraform, into a unified workflow using Kestra as the platform that enables me to orchestrate all these tools together seamlessly, creating a powerful automation ecosystem. If you're a fan of automation, you won't want to miss this!
References
- Check out Kestra's Github: github.com/kestra-io/kestra?u...
- Join Kestra's slack: kestra.io/slack?Yo...
- Kestra Tutorial: • My NEW HomeLab automat...
________________
💜 Support me and become a Fan!
→ christianlempa.de/patreon
💬 Join our Community!
→ christianlempa.de/discord
👉 Follow me everywhere
→ christianlempa.de
________________
Read my Tech Documentation
christianlempa.de/docs
My Gear and Equipment
christianlempa.de/kit
________________
Timestamps:
00:00 How I’m automating EVERYTHING
00:58 What automation tools I’m using
02:14 The problem with Ansible and Terraform
04:09 My automation pipelines in Kestra
05:26 Build an Ansible Flow in Kestra
09:22 Problems with the Flow execution
10:55 Adding the private SSH key
14:18 Expanding the Ansible Flow
16:07 Some Ansible examples of my HomeLab
21:41 Terraform Flows in Kestra
23:21 Updating my DNS using Terraform and Kestra
26:21 GitHub Actions + Kestra + Terraform + DNS
________________
Links can include affiliate links.

Пікірлер: 56

@jameskirk5914 25 күн бұрын

this channel is criminally underrated

@christianlempa 25 күн бұрын

Thank you :D

@jasonperry6046 25 күн бұрын

I completely agree.

@kylerhella6251 20 күн бұрын

I’ve basically learned how to be a sysadmin from watching this channel 😅

@weholmes5315 24 күн бұрын

Just watched your 2023 Homelab Tour where you shared your 100k Creator Award. And here you are on the brink of 200k! Next Award will be well deserved. Keep up the great work/fun. You are much appreciated 😊

@christianlempa 23 күн бұрын

Thank you buddy ❤️ let's celebrate this on the next livestream :D

@dkryptonut 11 күн бұрын

Something I'd be keen to see is a video about storing credentials securely, whether it be a key or an actual password - and retrieving it within Kestra.

@DreamerCze 25 күн бұрын

There is Event-drive Ansible now, that might be worth checking out.

@berndeckenfels 21 күн бұрын

If you automate it anyway, also manage the known hosts file with the server keys and not turn security in ssh off

@guanche011 25 күн бұрын

Kestra looks cool, and it could combine a few tools I use professionally and at home. But, having no secrets manager and needing to taking everything down every time I need to add something is not an option. Paying for features is not a problem, but the only other tier next to 'free' is "talk to us". This is a really high barrier, while this is provided and managed in a free AWX setup. Also concurrency requires separate tasks in Kestra and writing extra domain specific YAML, which makes migrating more expensive.. Overall I'm not sold yet. The product looks cool, flashy (slick UI i must say), but these are show stoppers. Would have loved to play with shiny things though ;)

@BilalInamdar 10 күн бұрын

Do u have any alternative in mind ? kindly suggest as i am also trying to evaluate the same.

@RealNeuronaut 23 күн бұрын

you should probably not circumvent permission problems by running the docker container as root... adding the following task after the local file creation does the trick just as well: - id: set_perm type: io.kestra.plugin.scripts.shell.Commands commands: - chmod 600 id_rsa

@dimitriclark9559 25 күн бұрын

Just what i needed, thank you!

@christianlempa 23 күн бұрын

You are so welcome!

@sandeepreddy3994 2 күн бұрын

Hello Christian, Loved your work…!!! i really appreciate it, It would be a great help, if you can show us how can i control the flow execution based on the exit code of previous tasks in kestra

@mathesonstep 23 күн бұрын

I am now trying to decide between using this or Ansible Semaphore

@AlphaX66 25 күн бұрын

Does Kestra have a vault features to store secrets ? Inject SSH key in the container itself looks ok, but what about some other secret that you want to you use in the ansible playbook ? Do you really need to use the container environment variables each time you need to had a secret ?

@lvmeijer 24 күн бұрын

Enterprise only

@antonkozyk 20 күн бұрын

How do you use your /app/scripts? What kestra type can be used?

@mariotubelecce 25 күн бұрын

it's basically jenkins with a nice interface and a paid option?

@cheebadigga4092 25 күн бұрын

kinda yea

@Danielo515 22 күн бұрын

What I want is a nice way to execute playbooks against custom servers in a nice UI

@hcjkruse 25 күн бұрын

Cool, I Didn't know Kestra going to look into it. Would be nice if Kestra also could run helm charts and docker-compose files. Going to look into that. My first thought was, but I have a homelab git repo, but that was anwered quickly. But.... Do you have a separate set of disaster recovery playbooks in a remote backup etc that you can use to setup the machines to run all this just from the shell? All this automation looks cool but bootstrapping it again by hand would not be fun.

@laurentiusjudhianto6631 25 күн бұрын

Yes you can run helm and docker compose by utilizing the bash - shell. This is the way to use Kestra if there is no plugin for it.

@nagavenkataphanikumarkanch5827 3 күн бұрын

Hi can we connect windows servers to run some shell scripts from Kestra? Through ssh or ftp something like that

@clarkkentgwapo1 17 күн бұрын

Deserve subs

@cheebadigga4092 25 күн бұрын

very nice! The error about the key files being too open worries me though. The correct permissions of id_rsa is 0600, so you could try to chmod it right before the ansible-playbook command. I'd be interested if that works.

@christianlempa 23 күн бұрын

I tried that, but didn't work for me. Maybe I just made a mistake 🙈

@Glatze603 25 күн бұрын

I prefer ansible and cronicle task scheduler for automating tasks in my homelab.

@johngleeson7919 25 күн бұрын

What, in your opinion, is the inflection point between time spent doing a task vs time spent automating a task? This video is cool, just wondering what your thoughts are, given in an enterprise environment, you'd typically spend a lot of time automating something that'll be reused many times. In a homelab setting, that's not necessarily the case, but you do get the opportunity to learn without the ramifications of breaking a "real" production environment.

@christianlempa 25 күн бұрын

That's a great question, I've recently covered that in my livestream, how useful is automation really in a HomeLab. Because you can argue it's wasting time :) again maybe that recording might be interesting for you

@Reprint001 25 күн бұрын

Looks powerful, but a bit OTT for my little homelab. I'm also already heavily invested in NodeRed for automating lots of things (like my Proxmox node backups to PBS). Interesting though. I can only learn so many things and this might be one too far. Also.. YAML makes me shudder.

@corincorvus6679 19 күн бұрын

So Kestra replaced your Ansible Semaphore Installation in a past Video?

@christianlempa 15 күн бұрын

I'm still running Ansible Semaphore for testing, but most things I'm doing in Kestra now. Still some Ansible Semaphore content is planned for the future :)

@Mavo1989 25 күн бұрын

Not sure why your not just using jenkins..... free open source docker image and have full devops CI/CD pipelines and also utlise github/git actions and your branches and have proper devops pipelines.... honestly jenkins is the best answer

@gautambhatnagar4765 24 күн бұрын

Top right....

@BenReese 25 күн бұрын

Yeah.... I'm gonna have to watch this one about 6 more times.

@christianlempa 23 күн бұрын

Hope you have fun :D

@isaacaymerich2741 25 күн бұрын

I'm tired to see more tools that are not solving anything, this is just another nice UI, you guys need to start learning nix, a real killer solution.

@HZGF 25 күн бұрын

AWX and terraform cloud with a self hosted runner make much more sense to me then trying to „rape“ kestra to do it in a less nice way 🤷‍♂️

@kimcosmos 24 күн бұрын

sure if you want to pay for Tower to get the terraform capability in AWX

@mickolesmana5899 25 күн бұрын

huh so it is like airflow but more for infra

@avluis86 25 күн бұрын

I heard you like to automate so I automated your automation 🤖

@christianlempa 25 күн бұрын

Haha :D

@dus10dnd 25 күн бұрын

Ansible and Terraform are both Infrastructure as Code tools. Ansible is more focused on configuration management and Terraform is more focused on deployment. Terraform + Ansible = Terrible :)

@RuanBekker 16 күн бұрын

The only place I really use ansible lately is using Packer with the ansible provisioner to build AWS EC2 AMIs and then using the data resource to retrieve those amis when deoloying ec2. Works not too bad

@itslemonandrew 25 күн бұрын

I want to disagree with the first part of the video that you need to run the commands in your project. With Semaphore that you talked about a while ago is made for this scenario.

@christianlempa 23 күн бұрын

Semaphore could be an alternative to Kestra, but without anything like it, you have the same problem.

@keywal 25 күн бұрын

The more of these videos you make the less respect I have for you. You’ve turned yourself again into an advertiser for your sponsors and in a few weeks it’ll be something else you have been using in your lab. I’m over it - cheerio

@avluis86 25 күн бұрын

Personally, I found it very helpful as I've been eyeing this product since recent discovery -- sure we may move away from it in the future but that is the point of a homelab in the first place -- trial all the things!!

@bubi352 25 күн бұрын

bye, bye!

@bluesquadron593 25 күн бұрын

There is a solution for your problem, skip the content you don't like.

@maplenerd22 25 күн бұрын

I don't see this as a problem as long as he clearly states that this video is sponsored by the product.

@sloraris 25 күн бұрын

Advertising is why you get sponsored... Kinda the whole point. Good software and content costs time, and time is money. Sorry not everything is free.