The EASIEST way to Expand Your ZFS Pool in TrueNAS (But is it the Best?)

Рет қаралды 66,736

Күн бұрын

ZFS is a great file system that comes with TrueNAS and can meet all of your storage needs. But with it comes some complexity on how to manage and expand your ZFS storage pools. Over the last week I learned all about storage pools and how to move them, expand them, and even what not to do when trying to grow your storage pool. Join me as I figure out how to move a 20 TB pool to my new storage server with 100 TB of raw data.
Video Notes: technotim.live/posts/truenas-...
Support me on Patreon: / technotim
Sponsor me on GitHub: github.com/sponsors/timothyst...
Subscribe on Twitch: / technotim
Become a KZfaq member: / @technotim
Merch Shop 🛍️: l.technotim.live/shop
Gear Recommendations: l.technotim.live/gear
Get Help in Our Discord Community: l.technotim.live/discord
2nd channel: / @technotimtalks
📦 Products in this video 📦
Seagate Exos 14TB Drives amzn.to/3kaQnkN
Seagate IronWolf 8TB Drives amzn.to/3iGq3yH
See all of the storage I recommend in this kit!
kit.co/TechnoTim/best-ssd-har...
A clip used in this video was from Tom Lawrence's channel. Thanks Tom!
/ @lawrencesystems
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
#truenas #nas #zfs
Thank you for watching!
00:00 - What are my options for expanding ZFS?
00:25 - Use ZFS Snapshot Replication (My First Attempt)
02:20 - Just Copy the Data to the New Pool (My Second Attempt)
03:01 - Expand the Pool by Replacing All Disks (My Third Attempt)
04:27 - Replacing All of the Drives & Resilvering
07:16 - Pool has Expanded!
07:43 - My Beef with ZFS and Recommendations
09:20 - Stream Highlight - This is how I got into this mess with ZFS...

Пікірлер: 211

@TechnoTim Жыл бұрын

Have you experienced any issues (or disasters) with ZFS or RAID?

@maddads6492 Жыл бұрын

until now i have avoided expanding until i could afford at least 4 extra disks for a raidz2, i dont fancy the 1 by 1 resilvering process

@JGoodwin Жыл бұрын

Once, I had a home lab w/ 2 disk shelves which were off lease. I set it up as a RAID 10. Mirrored between the shelves, striped between the drives. Because it was a home lab, I didn't see the point in redundant power supplies. I have to admit, it was sketchy :) As far as actual disasters, it's been a long time. Failed drives and such, but most annoying was hardware RAID. It was great when it worked, but when it's 5+ years later, you don't generally feel buying the identical card is a good deal, but your configuration may be too tied into the old card. That's why I appreciate how well software raid works today.

@AlbusRegis Жыл бұрын

The first server I had had a hardware RAID controller, and I set up RAID5 with the 4 drives I had. It worked well for a long time, until lightning killed the server. Then I discovered that I would need the same RAID controller with the same firmware and hardware revision as the old one to acess the data, because it used propietary sauce to do its thing. This was a noob mistake to make, but I learnt a lot from it, specially not to trust hardware defined RAID. I now have a generic LSI HBA and ZFS, which is platform agnostic and a lot easier to work with when implementing things like monitoring ;)

@ajhieb Жыл бұрын

So far (vigorously knocking on wood) no issues with ZFS. Lost a 24TB software RAID array when I had multiple 3TB Seagates die at once. Lost a 100TB hardware RAID6 array when my controller lost its mind during a power failure. Since I switched to truenas I have two main pools (each has 3 vdevs with 8x6TB drives in RAIDZ2) and my backup server has 2 pools that replicate the primary server. (each one is 1vdev that is 8x16TB in RAIDZ1) I keep 3x6TB offline as spares and 1x16TB. When the 16TB drives drop a little more, I'll probably replace the 6TB drives to save on power. But so far this setup has proven to be pretty much bulletproof if not energy efficient.

@coletraintechgames2932 Жыл бұрын

Used four old drives as a zfs experiment, I had no important data on them. One started going bad, replaced and a 2nd started failing during resilver. Lost a little data. (Not important, no big deal, but yeah) then replaced that drive, went fine. Low and behold one of the replacements started going bad... Repeat! Goodnoos how much work do I have to put into a old ZFS pool. Lol

@xxcr4ckzzxx840 Жыл бұрын

Hey Tim, thanks for sharing this but i have some criticism that hopefully helps the next time you do stuff like that. Usually the way to go would be to LEAVE the old disk in place and just replace it with a new one. That way you never loose redundancy at any point and the old disk can actually be used as a reference, like the others afaik. If you can do it at all, just leave the drive where it is, put a new one in and start a replacement. Much safer. EDIT: Also please let the drives slow down for ~30 seconds after pulling. Because.. You know..They spin with 7200rpm with a head barely a few atoms away from the platter. They do not like to be moved while running.

@TechnoTim Жыл бұрын

Great feedback! Thank you for the tips!!!

@xxcr4ckzzxx840 Жыл бұрын

@@TechnoTim Thank you for being good sports about some criticism and happy hoarding!

@CaseyHancocki3luefire Жыл бұрын

@@TechnoTimI was going to write a similar comment but found this one so I will boost it instead.

@ryanmalone2681 Ай бұрын

This is really the Achilles heel of TrueNAS. I hate almost everything about Unraid, but for easy growth it’s the one and only advantage over TrueNAS. I’ve been waiting until my rapid growth slows down and becomes more linear so I can model out how much storage I will need, double it, and then build my TrueNAS pools and migrate to TruNAS off Unraid. I have 250TBs so it’s gonna take a LONG time ass time. Also, when you have this much data, spare drives are crazy. I have a total of 88TB in 4 spare drives for my 2 NAS’ (primary and backup). Kills me to see them just sitting there when I’m buying more storage.

@Seandotcom Жыл бұрын

I'm a big fan of using mirror vdevs for this reason. Gives me a lot of flexibility. I've added several mirrors over the years and even replaced old mirrors with bigger drives by replacing/resilvering one drive at a time.

@charlescc1000 Жыл бұрын

I agree. Mirrors are awesome. Yes the per TB cost is higher, but in exchange you get more flexibility and higher IOPS performance. Good trade off in my view. I think the per TB cost of mirrors is too high for Plex storage/Linux ISOs so I just keep those on an Unraid storage since the per TB cost is similar to raidZ1 but has much more flexibility. The cost of that flexibility is the lack of ZFS. I don’t need ZFS for Linux ISOs

@tobytran833 11 ай бұрын

Thanks for this! Currently have 8x 8tb's. was wondering what I needed to do if I wanted to upgrade to larger drives. This video answered all my questions.

@David_Quinn_Photography Жыл бұрын

its great that its this easy, I am a home lab use for TrueNAS core and it is a mix and match of drives that family has given me over the years and I have a 2 disk parity set up with the intent of getting actual NAS drives over time of various capacities depending on price when I buy them till the storage pool is larger as I replace the old drives.

@JGoodwin Жыл бұрын

My preferred way to upgrade is to leave the old equipment in place, stand up the new equipment, then copy the data over.

@haydenc2742 Жыл бұрын

Ultimately that is probably the safest way

@helpmeget100subs Жыл бұрын

yet not possible

@raskulous Жыл бұрын

Sure it is, you just have to create a new pool with the new drives instead of adding them as a new vdev in the same pool.

@helpmeget100subs Жыл бұрын

@@raskulous he said it wasn't possible with ISCSI block shares

@Pythonzzz Жыл бұрын

That’s definitely the safest way, but that assumes you’re buying all new equipment and that’s expensive. A lot of us want to just be able to add a drive or 2 to expand our existing zfs pool.

@alexrambolton Жыл бұрын

I just expanded by adding an other zdev to the pool. Also you can add drives as hot spares to the pool. This will auto resilver the pool upon failure (or if your pulling one at a time). This could of save you some time. At the end you could of powered down and re arranged your disks how you wanted because trunas doesn't care where you plug them in.

@rudypieplenbosch6752 Жыл бұрын

Thanks for showing us, I heard about this method but felt a bit weird actually doing it, after this it seems ok, I probably will use it as well.

@D2Dev Жыл бұрын

Jim Salter has a post on why you couldn't snapshot the ZVOLs titled, "PSA: Snapshots are better than ZVOLs". Basically, "If you don’t have at least as much free space in a pool as the REFER of a ZVOL on that pool, you can’t snapshot the ZVOL, period."

@charlescc1000 Жыл бұрын

Ending a ZFS related sentence with “period” reminds me of the 2.5 admins podcast. Jim is always sharing his ZFS tips

@TechnoTim Жыл бұрын

Thank you! I’ll check it out!

@chromerims Жыл бұрын

Thank you for this. I just checked that out. In the comments to that Jim Salter post, someone linked to a 2013 post from Nex7 who did an amazing job detailing this issue in a concise, four-minute read.

@ryanb509 Жыл бұрын

This is why I am seriously considering using unraid for when I build my NAS server. I would love to see a comparison of it to TruNAS.

@norseironworks Жыл бұрын

I started my pool with 4 x 18 TB sas drives. After a while I bought a new set of 4 x 18 TB drives and expanded my pool that way and it went perfectly

@MRPtech Жыл бұрын

My current setup is just mirrored 18TB inside TrueNAS VM. i went for 18TB just to future proof a bit with storage. At this moment i have used 25% out of 18TB. This video will be a great came-back-to once i am at the limit with free space

@scottezinn8555 Жыл бұрын

Sounds like a challenging time - an option I would have considered was setting up your new drives as mirrored VDEVs and then just doing a copy from the old pool to the new pool. Once the copy was complete and verified, you could destroy the old pool, and then add those drives as mirrored VDEVs to the pool as well (if you wanted to). You could also add additional drives as spare (as long as they are at least as big as your largest drive) Mirrored VDEVs do only give you 50% of the space, but resilvering is very quick if you ever have to replace a drive since it is just a mirror operation

@exodus454 Жыл бұрын

FYI there's a bunch of sysctl parameters that can be played with for the resilvering priority. The ZFS/truenas defaults are pretty conservative if you if you're not trying to keep the pool IO down.

@TechnoTim Жыл бұрын

This probably could have saved me a day! Thank you!

@Mr_Meowingtons Жыл бұрын

Im kind of excited for this right now i have 8 - 2TB "Used" SAS drives from Ebay right now there working good but ill want to expand them some day with 4 or 6TB drives.

@LPTechDev Жыл бұрын

Hello, congratulations for the video and for sharing the issues you've faced. I saw it, and since you mention "iSCSI", don't know if makes sense on your setup, but probably you're using this block devices as LUNs on some hypervisor. If yes, could I suggest migrate the virtual disk to new iSCSI LUNs that are hosted in the new ZFS pool? Proxmox allows you to move the virtual disks from one storage to another. Thank you once again.

@Demodude123 Жыл бұрын

I did this same strategy on my ZFS nas because of a similar reason. The complexity of my pool was too great to confidently copy off. For my next homelab nas I'm considering single node ceph. Then I can expand to clustering in the future. Start with drive failure domain, then move to host.

@davidwilliss5555 5 ай бұрын

Since you had extra drive bays in your Storinator, you could have done this in a safer way. Put the new drives in where you want them to live and also the old drives - like you had originally. Make sure the new drives aren't part of a pool or anything so that they just show up as spare drives. Then one at a time, tell TrueNas to replace a drive selecting one of the new unused drives. You still have to wait for it to resilver and it'll take just as long. Once a drive is done resilvering, the old drive will now show up as a spare and can be removed. Rinse and repeat. The only problem I had doing this is that the pool didn't auto expand when it was done and I don't know why or how to expand it. I was hoping this video would tell me, but it just worked automatically for you. [ Edit: ] Oh, that was easy. There's a button labeled Expand that I hadn't seen before.

@unijabnx2000 Жыл бұрын

Should have done an online replace (IMO) that way you could have all/some the drives replacing at the same time (if you have enough cpu horsepower). I dont know if the gui supports that, but the CLI does.

@Tekgnome. Жыл бұрын

It does! You can navigate to storage, pool, select the drive and hit replace, then select the desired disk that you want. I just did this when I lost a drive in my Z2 vdev

@CaseyHancocki3luefire Жыл бұрын

Yep

@UntouchedWagons Жыл бұрын

Why didn't you perform an online replacement?

@fluffyfloof9267 Жыл бұрын

I thought there was a way to online-replace drives, without degrading the pool. Maybe the TrueNAS GUI doesn't expose this functionality, i couldn't tell, because i'm using the command line in Linux.

@IronRex Жыл бұрын

ZFS folks are working on the ability to add drives to raidz. It is working but not yet part of current ZFS version. And I am sure it will come with some caveats but it will be possible to do. Separately you may now have enough storage space in the new pool to make that snapshot. And then you can use 6 of your old drives plus one new one to make the 7 drive pool. Then zfs send the 6 drive pool to the 7 drive pool. Then replace older drives in the 7 drive pool. TADA! You now have 7 drive pool of all new drives. Of course if you can afford to backup your 20TB somewhere else and then restore to the new pool that is the safest option. But is that even fun?

@b14ckyy 11 ай бұрын

I am building my first DIY NAS currently based on my old PC after I upgraded a few months ago. I tried TrueNAS and Openmediavault in a VM with a bunch of virtual drives to play around with. The lack of options to extend an existing pool or raid array with ZFS was a complete NoGo for me as I want to start with 3 drives first and then over time expand to 5 or 6 due to cost factors. I am not willing to pay 1000 bucks on disks all at once. after that was no option with TrueNAS I decided that Openmediavault with a Classic RAID5 and EXT4 is the much better option as extending is just a few cliicks and I can extend 1, 2, 3 or whatever amount of drives I want.

@jimholloway1785 Жыл бұрын

Thanks for the video on your ZFS pool. I see by your server the hard drives are hot swapable. What would you do if the drives were not hot swapable?? Do you need to shutdown the server and remove the failed or small drive you want to replace and put in the new larger drive and then power up the server and through the UI say to replace the drive? I am making a TrueNas server out of an older Dell I7 computer which does not support hot swapable drives, not sure how to replace a failed drive in the new system I want to use for a TrueNas home server? Any info would be appreciated. thanks Jim

@ErikHejl Жыл бұрын

Did you consider a one-time cloud backup and restore (S3 or similar) to migrate your data from the old pool to the new one?

@tekidiots6863 7 ай бұрын

could you do a video on how to setup email notifications, i tried setting these up but having no luck

@JustForUs Жыл бұрын

Snapshots of zvols should be possible. Have you set quota on your zvol? Quota limits the overall size of a dataset/zvol and all of it's children and snapshots, while refquota applies to only to data directly referred to from within that dataset/zvol.

@TheAnoniemo Жыл бұрын

Wouldn't it be easier to leave the drive you are replacing in while you replace it with the new one in a different slot? Perhaps it speeds up resilvering because ZFS can just copy the data from the old drive to the new one without the need to recalculate all the blocks and parity.

@charlescc1000 Жыл бұрын

Tim- one question here- why didn’t you just add the new drives as a second raidz2 vdev to the new server? I think the answer is because initially you have too much data in total to consolidate into either your new or old set of disks? However at the end you only have 6 disks in your server, where are all the others? Are they empty? If so, why not just add a second raidz2 vdev to your pool?

@davidstievenard6313 Жыл бұрын

you don't need to take out the drive and degrade the pool status to replace it...just add a new drive click replace

@familyrl1648 7 ай бұрын

Exact same as synology. While you have slots available

@urmastertech Жыл бұрын

Are you planning on selling the disk shelf? I potentially would be interested. Currently I'm limited to 15 total drives with the rosewill case I have and they are not hot swapable

@GaryStango Жыл бұрын

What chassy are you using for the drives?

@TanKianW79 Жыл бұрын

Been using ZFS for the longest time. This migration and upgrade never bothers me. Reason being I always plan ahead. Having 1-2 upgrade cycles already in mind after a current upgrade. Never had a data recovery disaster since I jumped on ZFS. I normally add-on vdev for incremental upgrade for existing pool, or double the drive cap for a new pool to back up the old pool before migration. I moved from 1TB, 2TB, 4TB, 8TB, 16TB. I am currently managing 3 pools in my homelab. 8x 4TB, 8x 8TB (2 vdevs), 4x16TB. Down the road will be retiring the 8x 4TB which had clocked 6 years for "less important" backing up function and the 4x 2TB (>9years) convert to a Steam storage server, followed by add in another 4x 16TB to expand exisiting main pool.

@TazzSmk Жыл бұрын

don't throw that 8x 4TB pool away, it's still "best" to sustain 10Gbe throughput if you'd make use of it

@xeghia Жыл бұрын

It looks like you had enough space to have both sets of drives online? You can keep full redundancy doing it that way which others have mentioned, but you can also replace 2 drives at the same time as well, so half the time spent waiting to expand with full redundancy during the process. Sometimes it triggers a scrub after replace, usually you can just ignore it/cancel it and scrub at the very end to confirm all is good. Not sure how that goes in the TrueNAS webui but possible on the command line anyway.

@KenJones1961 Жыл бұрын

Just converting over from a Synology to a TrueNAS system so I'm just learning. I take it you couldn't go from a 6 disk Z-1 to a 7 disk Z-2. But couldn't that extra drive be a hot-spare for that vdev? I haven't explored that yet. Does the hot spare just sit there waiting for a failure and steps in re-silvers when needed? Is it spun down in the meantime? EDIT: I'm learning A LOT in these comments! So much good stuff here.

@jdrakehoffman Жыл бұрын

This is why I use mirrors. It makes it much easier to expand a single vdev, or add a new vdev (assuming you have the open drive slots for it). Plus resilvers are MUCH faster. IMO, worth the efficiency loss.

@severgun Жыл бұрын

for resilver times there is DRAID now.

@tannerjim Жыл бұрын

Using TrueNAS, if I start with say 4x20tb drives and then say a year later I want to add another 20tb drive to my ZFS1 system, can TrueNAS integrate that into my pool and increase my overall storage by ~18tb. I know I won't get the full 20tb as some is earmark for recover of other drives.

@peterkambasis Жыл бұрын

Interesting video Tim! Thanks for sharing that info. Scared to try TrueNAS, but this makes more sense now. I guess If I ever got my hands on a storage server like that, would it be better to fill up all the slots with lower sized drives first (and I mean all the slots - from 1 to 15 - with like say 1GB drives since they are so cheap). Create the ZFS pool, then slowly introduce the 14GB replacements (as I can afford them) into the pool. But If I added even just one new 14GB drive to the pool, does TrueNAS expand the space from 15GB to (14 x 1GB)+14GB(single drive)? Or do all your drives need to be replaced before it expands it? Not sure if I'm making sense.

@zandr0 Жыл бұрын

No. You need to replace all drives, with a common larger capacity, to expand the space. Your pool will be as large as (n * smallest drive (minus parity)) until then. eg 8*1TB Raid-Z2 = ~6TB usable; which is 6 data and 2 parity. You'll need to replace all 8 drives with something larger before you can expand in to the new capacity. Hope that makes sense.

@peterkambasis Жыл бұрын

@@zandr0 Yup totally. thank you so much!

@PolSenserrich Жыл бұрын

@TechnoTim what is this hdd enclousure? Thanks

@yogugl Жыл бұрын

I personally use ceph in my setup at home. I like that you just can add a single drive one by one and it will automatically rebalance.

@aarcaneorg Жыл бұрын

How do you handle needing to keep at least 3 hosts, and preferably more, all running constantly to maintain pool health?

@yogugl Жыл бұрын

@@aarcaneorg The standard setup of ceph is that it handles one host failing. Since I don't have 3 hosts (only have 2), and I don't need HA, but only protection from drive failure/bit rot, I changed the setup the handle up to 2 drive failures. ZFS would be probably more performant in this scenario, but I like that I can add just single disks. I probably would have initially also used ZFS, since you find a lot of youtube videos about it, but I wanted to evaluate cephfs shared file system for office, because we use some vendor software that doesn't play well with nfs.

@yogugl Жыл бұрын

@@aarcaneorg I changed the default failure domain from host to osd, so it handles disk failure, but not host failure. I only have 2 osd hosts plus a raspi as a 3rd monitor.

@AceBoy2099 Жыл бұрын

Is that core or scale? Should I consider updating/upgrading my core VM to scale, or would that introduce issues to a working "file server" system? I too would love the idea of a dynamically upgradeable/changeable/configurable pool, hence why I have it as a VM on my unraid box.

@rrodriguesv2 Жыл бұрын

Hello Tim, do you know a way to activate the turbo boost of the cpu inside truenas scale v22.12.2 and passing thru to a windows 10 vm machine ? I am running a epyc 7d12 on a supermicro h11 ssl-i,,, the cpu is running on the base clock of 1.1ghz full time, seems a little slow... even when i run cinebench or cpu-z

@area51xi 8 ай бұрын

How long does it take to resilver?

@sanderdelft Жыл бұрын

So now that you've learned zfs pools the hard way, are you considering to redesign your storage solution (I.e. plan the use of your current drives as well as plan for future expansion)?

@BerserkeR_031 4 ай бұрын

Maybe some ZFS videos via CLI would be good, not necessarily on TrueNAS.

@tekidiots6863 7 ай бұрын

can you replace a drive with the system off ? i dont have hotswap option

@LtdJorge Жыл бұрын

I think it would have been a bit faster if you used all the newer drives as spares on that pool. So you don’t have to manually replace them :)

@seraphin01 9 ай бұрын

it's 2023, I really hope the truenas team will figure out a way to simply expand pools like synology does with their SHR.. synology makes questionnable hardware (mostly just very overpriced) but their software part is pretty decent albeit limited

@CrankyCoder Жыл бұрын

I do this regularly. But my main pool is configured with 12 drives. 2 vdevs with raidz2 and 6 disks each. if i upgrade all the disks in either vdev, that vdev expands, increasing my pool size. so instead of upgrading all 12 of my disks. I do 6 at a time. Any time a disk fails I replace it with the next step up in size.

@ZeroOmar00 Жыл бұрын

a small note, you could have replaced all the disks one by one in software instead of going to the rack and physically replacing one by one.

@coletraintechgames2932 Жыл бұрын

I'm not expert, I promise. But I think you can "replace" a drive while leaving the old drive in the system. This speeds up the resilver and takes load off the system. So you would put the new drive to the side of the existing. Hit the replace button, it would resilver at a faster rate utilizing the old drive as well, and then offline the old drive when done. I tried to link you something and I think KZfaq zapped my post from that. Forgot about that, and sorry if it's a dupe. I am trying to remember where I got this knowledge, I was trying to link a ixsystems article, while dated, and not explicitly stated, it was referenced. I had a test ZFS pool that I battle with this a bit . I only have 4 drives and 4 slots and a USB addition with drive was not an option, there were some adapters I could have bought but didn't. The old saying about "when one drive falls, another one might fall during resilver" happened to me. And that's why I was researching this.

@Mikesco3 Жыл бұрын

It is possible and true... 1. Plug in the new drive on the side. 2. issue the command: ``` zpool replace poolname old_drive new_drive ```

@coletraintechgames2932 Жыл бұрын

@@Mikesco3 right! And I think you can do it in Truenas as well, not sure, but I think

@CaseyHancocki3luefire Жыл бұрын

Yep

@area51xi 8 ай бұрын

Is there a link or video to how you built that TrueNAS server with the colored fans on the front face?

@TechnoTim 8 ай бұрын

There is! Here it is! kzfaq.info/get/bejne/sMdxiNFhtsuvhqM.html

@aarcaneorg Жыл бұрын

...you know you can replace the existing disk without having to remove the old disk first, right? Just zpool replace pool olddev newdev; and then remove the old disk when you're done?

@enkrypt3d 10 ай бұрын

why not assign that spare disk as a hot spare? so it should be auto-magic (if truenas supports this? I'm not sure)

@shanent5793 Жыл бұрын

You can add any number of drives of any size to expand a zpool. Don't take my word for it, it's easy enough to try it in a VM or cloud server and see for yourself.

@Kwijibob 4 ай бұрын

Surely for this scenario you could just do a bit for bit clone of the old disks to the new ones all with dd while they are offline and then swap them old for new.

@nakinney1 5 күн бұрын

Should have mounted the old pool. Created a new pool with the new drives. Created a new dataset on the new pool that matches the old. Then use rsync to copy the data over. I've updated and expanded my zfs systems multiple times this way. It's a bit slower that replication with snapshots, but a lot faster that reslivering and you get to add in more drives..

@jeffhex Жыл бұрын

TrueNAS ship of Theseus upgrade 🙂

@jedring3756 Жыл бұрын

Just curious, Couldn't Tim have added a second vdev to his zfs volume as a mirror to the existing and waited for that to complete and then remove the small vdev?

@juliansbrickcity5083 Жыл бұрын

You cannot mirror a raidz. But he could add the disks as a raidz6 vdev to increase the capacity

@computersales 7 ай бұрын

I wish it wasn't so difficult to resize and change the configuration of a ZFS pool.

@bobbrown8661 Жыл бұрын

Hopefully OpenZFS release ZFS Expansion and TrueNAS roll it out into the main release so we can expand single VDevs...

@etienneb4403 Жыл бұрын

I think you can add a new disk by doing a replace disk inside Truenas GUI and only afterwards remove the old drive. Isn’t that saver? Perhaps faster?

@Tekgnome. Жыл бұрын

Correct, its safer and faster this way

@puretrack06 Жыл бұрын

With seven drives, in my opinion you should be using the 7th job as a hot spare anyway

@MrItsChris Жыл бұрын

What case was that

@jaysonsim1462 Жыл бұрын

is that the sysrack?

@ti6i Жыл бұрын

Were you not able to just make a second Iscsi pool and then just copy them from one to the other within your OS of choice?

@chromerims Жыл бұрын

Exactly. Cloning of his TrueNAS VM and have it talk to a new separate zpool with zvol inside. Start a replication. Dunno. The advice elsewhere in these comments to tune down the refreservation size is on point for unblocking snapshots. Kindest regards, neighbours.

@sesshoumarusama7397 Жыл бұрын

I heard they are working on the expand, but they do so since yeeeeears. Its almost laughable how such an basic thing is still not supported properly. I plan to expand my TrueNAS by adding VDEVs, each one being RaidZ3. So once my 10 drives run full, another 10 will be added as new VDEV to the pool. Sounds safe enough for me. There is a true backup of cause.

@byolock6244 Жыл бұрын

I've read the title and wondered what this video will be about. I literally laughed when I found out simply adding a drive is not supported. It never occurred to me to check this because I thought that was basic function. I checked the status on that and 5 days ago IX-Systems posted on Github they are looking into integrating an outstanding Github Merge Request adding this functionality into TrueNAS Scale.I hope they can finally speed up this feature implementation.

@AcidiFy574 Жыл бұрын

Any ideas on homelabing as cheap as possible for us folks in 3rd world countries ??

@MiroslavIvanovimbmf Жыл бұрын

using a "zpool replace" command you don't need to go and replace every hard drive physically. :)

@cts006 Жыл бұрын

Can be done in the truenas gui as well in the pool status.

@MiroslavIvanovimbmf Жыл бұрын

@@cts006 I am not so familiar with truenas gui

@familyrl1648 7 ай бұрын

Synology does same process over btrfs. Tha exact thing

@ivanmaglica264 Жыл бұрын

I think you could add the new drives as a new raidz vdev into existing pool and then remove the old vdev. Never tried it myself, but as far as I know, the data should be transferred to new vdev.

@severgun Жыл бұрын

what? I think it will stripe(round-robin) data something like RAID60. So removing will destroy all data.

@ivanmaglica264 Жыл бұрын

@@severgun not exactly, it will create a second raid6, but end layout will not be raid60, but rather raid6+raid6 JBOD. Once its created, you can remove the first one via CLI and it move all data from old one to new one. It will take long time. Once its done, you can then remove the old vdev. Check zpool status before doing that. The second vdev does not even need to be same layout as first one. It can be RAID0,1,5,6...

@GaryBarclay Жыл бұрын

I've been using this process to increase size, since day dot. Just takes a long time and is a bit nerve wracking. Like you, was a bit disapointed to find out ZFS doesn't just allow 'disk add' to increase pools but is what it is. I'm going to put this out there but I don't think ZFS is actually all that well suited for HomeLabs, which need much more adaptability than ZFS can provide. Seems much more suited for it's design purpose, in money-bags enterprise operations...go figure. I often wonder if the 50bucks or so for Unraid is worth it for convenience. I also find the disk management interface in TrueNAS a bit lacking although Bluefin update improved that a lot. Could also be my poor DevOps skills. 😆

@Tekgnome. Жыл бұрын

You don't need to pull drives and swap them, he did it the hard way. All he needed to do was slot in the new drives, navigate to storage, pool, select the old drive(s) hit replace and select the new drives. It's way faster since it's using a non-degraded pool.

@nuvemequilibrio1897 Жыл бұрын

What is the board name?

@frederichardy1990 7 ай бұрын

Anyone tried to clone (with dd) all the disks of a vdev to higher capacity drives and just start TrueNAS with the new drives? If it work it should be must faster! You don't have access to your data during the copy but you don't loose any redundancy...

@z400racer37 Жыл бұрын

lol yeah I went down the TrueNAS rabbit hole for a couple years................... Then I remembered Synology was a thing 😂

@MarceloAlcantaraX 10 ай бұрын

Tim or someone here - What is the hard drive drawer shown in the video? Cheers

@TechnoTim 10 ай бұрын

It's a Storinator!

@cinemaipswich4636 8 ай бұрын

Same number of disk with same number of disks. The 7th can be a spare.

@David_Quinn_Photography 10 ай бұрын

9 hours a disk god damn well good thing my pools only 320GiB disk as of right now.

@JohnWeland Жыл бұрын

So I am fully talking out of my ass here (I don't KNOW what I'm saying other than conceptually). But after watching Christian Lempa doing some TrueNAS stuff. I thought you could just make a new vdev and add it to your existing pool. Apparently you get better performance too with multiple vdevs. For 12 drives something like 4, 3 disk raid-z1 so you would lose 4 of 12 drives for parity. Then if you have a few cheap SSDs take 2 and make a mirror for a dedupe dev, another pair for a cache dev etc. Again all this comes from watching a few videos but not actually getting in and doing. So I maybe way off base

@AidenPryde3025 Жыл бұрын

This is why in 2018 I decided to use Unraid instead of TrueNAS. The amount of stress you're putting on the drives doing this is dangerous as hell. I really want to see how Limetech deals with this issue when they introduce ZFS support later this year, as traditional ZFS expansion violates the Unraid ethos of "expand as you go."

@Tekgnome. Жыл бұрын

You don't need to pull live drives, he did it the hard way. All he needed to do was slot in the new drives, navigate to storage, pool, select the old drive(s) hit replace and select the new drives. It's way faster and safer since its using a non-degraded pool.

@RealLordy 6 ай бұрын

You know, I do understand that YT streamers use ZFS etc... for their storage due to the huge amount of data. What I do not understand however is why people in their homelab revert to solutions such as TrueNAS or other ZFS based systems for the storage system since it comes with a lot of hassles when you want to change the configuration. Every YT channel is basically killing MS Storage spaces upfront without even looking at it. However: when using MS Storage spaces (available even on Win 10) using a mirror configuration with multiple columns, performance comes close to True NAS solutions at maybe a fraction of the HW cost. If the HW fails, you can simply move the disks to any other MS system (even no need to document the order you need to attach, just attach the disks in any order) and you are up and running again in no time. The HW you attach the disks to may be of much lower grade than the HW that went down, it is only the speed that will go down. For the rest it will be fully functional Dont misunderstand me: I do LOVE your videos and I do understand the use case of TrueNAS and the ZFS file system. I just dont see the advantage for smaller home labs to use this kind of hardware and solution when there is much simpler solutions (which are much faster to recover in case of disaster). I may be wrong though...

@TechnoTim 6 ай бұрын

Thanks! Storage spaces is awesome! I just don’t have a windows license to run it and I really love the simplicity of truenas. Being able to easily configure nfs, smb, iscsi, and object storage from a web browser is awesome, the fact that it’s zfs for quick and easy snapshots is a bonus. It’s not just ZFS, it’s everything that truenas provides!

@guidefortheblindguy4319 Жыл бұрын

Stupit question. Why not creating a new pool with the new hard drives. Then migrate the data only?

@CGGC0202 Жыл бұрын

unraid ;)

@guy_autordie Жыл бұрын

Resilvering: the long method. X) Ok, I will not do that way :D

@xxgg Жыл бұрын

Just wondering what do you do with all those storage?

@Dipsomaniac Жыл бұрын

(⁠ ͡⁠°⁠ ͜⁠ʖ⁠ ͡⁠°⁠)

@JordanKetterer 11 ай бұрын

I think you wuould have been happeir just adding the new drives in as a new vdev in the pool, usualy the reason against doing this is that people cant afford multiple drives at once, but as you had all them you could have added a new vdev with the disks and your pool would have grown in total

@JordanKetterer 11 ай бұрын

In fact you would have been way better off because you would ahve used all 7 disks of the new disks if you wanted. can you make a follow up video where you go back and add a new vdev of the old drives so you atleast still use them

@JordanKetterer 11 ай бұрын

I've been reaching my brain to figure out of I was being dumb or missed another reason you did what you did. What was the thought around doing the drive swap in the same vdev?

@brahyamalmonteruiz9984 Жыл бұрын

i thought you can just add another vdev to the pool and call it a day

@chromerims Жыл бұрын

True. Mayhaps Tim was also trying to retire those 6 drives of his initial 20 tb pool. Dunno. If that was the case, then adding the new vdev --such that the pool had two vdevs in it-- would have (permanently, I think) captured his first 6 drives. Dunno. Kindest regards, friends.

@brahyamalmonteruiz9984 Жыл бұрын

RIP finger, saw the blood

@notsmirkis 10 ай бұрын

i love all your videos but this was a little tough to watch especially after just going through it myself. i just added to the new drives, created a new pool, replicated all data over to new pool, updated system config over to new pool, then gradually started disconnecting/offlining old drives from the old pool 1 by 1 til they were all finally disconnected. kab auto reconfigs and moves apps to the new pool when you go to apps and select the new pool. i can't imagine what i would have done if i came to the conclusion that i needed to wait 8hrs per drive to swap over lol. why did you not just create a new pool and replicate everything over??

@TechnoTim 10 ай бұрын

I totally should have, looking back that would have been a better and safer option!

@newstandardaccount 12 күн бұрын

Step 1: build a NAS Step 2: suddenly realize you need a 2nd NAS to back up your NAS

@hpsfresh 9 ай бұрын

Can’t you just insert new drive and replace old one to new one without ejecting the old one? It should be as fast as copy.

@franciscooteiza 5 ай бұрын

NO!!!

@FranklySean Жыл бұрын

Must have been a crazy feeling, just pulling a working disk from a running system.

@Tekgnome. Жыл бұрын

It wasn't even necessary to do it that way. All he needed to do was slot in the new drives, navigate to storage, pool, select the old drive(s) hit replace and select the new drives. It's way faster since it's using a non-degraded pool.

@chromerims Жыл бұрын

@@Tekgnome. Greetings. In the comments herein, user 'DS' mentions an useful and instructive Jim Salter post, "PSA: Snapshots are better than ZVOLs." Further in JS's 2016 post in the comments section thereto, someone linked to a 2013 post from Nex7 who did an amazing job detailing the snapshots issue in a concise read. Basically, downsizing "refreservation" unblocks snapshots for ZVOLs under certain conditions. That said, I like the advice from user 'GoRK' in the comments section herein to use 'zfs send' for defrag and new disc topology. Kindest regards, friends.

@ryanmalone2681 15 күн бұрын

Roadmap is saying we might get expandable vdevs by the end of 2024 or beginning of 2025!

@bradvanluven4857 Жыл бұрын

The easiest and quickest way to expand a zfs pool is actually to just add a second vdev to your existing pool.

@camcarlton 6 ай бұрын

This is probably the worst solution but I just run rsync between my zpools when I upgrade. Same process I run when I want to make a backup of the zpool. 🤣

@Quettesh Жыл бұрын

What? You should be able to expand vdev with a single drive.

@joshua_lee732 Жыл бұрын

Honestly btrfs makes this silly easy compared to zfs. I can just slot in all the drives and easily add th to the existing array, call a balance then tell btrfs I'm removing the old drives one at a time. You rook 3 days, I'd take 3 hours.

@Tekgnome. Жыл бұрын

You can do this in scale as well in the GUI, he did it the hard way. All he needed to do was slot in the new drives, navigate to storage, pool, select the drive(s) hit replace and select the new drives. It's way faster since its using a non-degraded pool.

@hescominsoon Жыл бұрын

looks like you are using scale..which isn't truly complete. I only use core and have not had an issue with snapshots on truenas core..but have run intow eird issues on scale.

@hescominsoon Жыл бұрын

@Noah D one of my reasons as well is the gpl.

@maikschultz8750 6 ай бұрын

you must have got a warning when using so much space for a zvol on oyur zfs pool in the first place. even so using more than 80% reduces the speed of the pool dramatically.