Building My Own AI Server for Just $1195.36: A Homelab Journey

Рет қаралды 20,932

Күн бұрын

The reason for this build is that one of my side projects requires a GPU node (in an existing Kubernetes cluster) and hosting on cloud was way too expensive.
Support the channel get your own headbot:
www.headbot.ai/?s=ai_server_yt
If you want to see other forms of content I create check out my blog www.thegalah.com/
Timecodes:
0:00 Introduction
0:22 Used parts
1:27 The pickup
1:58 Unexpected problem
2:34 The build
4:13 Joining the cluster

Пікірлер: 38

@ChrisDaveGG 8 ай бұрын

This is really interesting thanks for sharing!, and at the end when you displayed Headbot I literally LOL'd, a very novel idea. Cool project.

@thegalah 8 ай бұрын

Glad you found it interesting!

@Avose243 10 ай бұрын

Nice video dude, keep it up 👍 Hope that vacuum cleaner was ESD safe, all that air moving around makes a significant amount of static in normal vacuums, kills components all the time. Same with air compressors. Just a tip! Keep the videos coming 👌

@thegalah 10 ай бұрын

Oops haha had a few people tell me about this! I will be more careful in the future!

@xorcody 7 ай бұрын

Nice build. Curious what models headbot is actually running behind the scenes.

@thegalah 7 ай бұрын

Sdxl

@hwole7895 Жыл бұрын

Nice Video

@thegalah Жыл бұрын

Thanks!

@Kingfinite 10 ай бұрын

Why did they make a 3090 with a single fan? that thing sounds like a leaf blower

@thegalah 10 ай бұрын

I often question whether some of Nvidia's product decisions are in the best interests of the customers haha

@Kingfinite 10 ай бұрын

@thegalah for me I am going to go with A 7900xtx from XFX

@thegalah 10 ай бұрын

Interested to see how the build turns out

@nuck477 3 ай бұрын

so you can fit more in a case

@Kingfinite 3 ай бұрын

@@nuck477 the answer is NO.

@ProjectPhysX 6 ай бұрын

It's a shame Nvidia entirely killed off 2-slot coolers for the 3090 and 4090. That was only to prevent people from using these relatively cheap gaming cards in workstations and servers, rather than their overpriced Quadro RTX 5000 which is the same card but more expensive. These hilariously oversized 4-slot coolers don't fit in normal PC cases anymore. 2-slot 3090/4090s only rarely appear on ebay nowadays, and at super expensive prices.

@thegalah 5 ай бұрын

Nvidia really milking every last cent!

@BrunodeSouzaLino 3 ай бұрын

You can always go for the RTX A2000 12GB, which costs around 600 bucks. If you're desperate enough, you can get the A6000, which is better designed than the 4090, dissipates less heat and only needs a single EPS connector to power it instead of the badly designed pipe dream that is the 12VHWPR connector.

@anthonyrubio1194 11 ай бұрын

Nice I’m gonna code the hell out my lab

@thegalah 11 ай бұрын

What are you building?

@paultalanoa9513 Жыл бұрын

Why not just get an adapter for the second gpu power connector

@thegalah Жыл бұрын

Good question, I didn't want to risk frying the 700 dollar card for a 120 dollar component; since the PCIe pinout cables are non-standard.

@talhagaming7204 5 ай бұрын

What is cluster and , can you provide me a details? I'm using local llms want to create a server on my machine And also wana use my pc at same time . My specs are : i5 10 Ram 48gb ddr4 Rtx 3060 12 gb

@thegalah 5 ай бұрын

I'm running all my workloads in a kubernetes cluster which orchestrates all my service across my physical machines. I suggest you look into getting cuda running on your machine

@shepardcoronel1980 10 күн бұрын

what about 2080 ti sli 2x ? cool solution man...

@Comander555666 5 ай бұрын

thought you left the vacuum running or smtn lol ridiculous

@thegalah 5 ай бұрын

Haha that's the sound of the fans spinning under Max gpu load

@jj-icejoe6642 Ай бұрын

Just $1195.36 ?

@yousefabdulrhman Жыл бұрын

loud music

@thegalah Жыл бұрын

Thanks for the feedback

@genericgoon3748 6 ай бұрын

couldn't you just buy one of those nvidia tesla cards? they're designed for AI stuff and have more vram and are cheaper

@thegalah 5 ай бұрын

Great suggestion. I did some simple comparisons and for my workloads 3090s fit the bill. There is usually a premium on the tesla cards. They are more powerful for certain tasks but for my specific workloads these were the best dollar to power ratio.