Add a DAS to your NAS! USB Direct Attach Storage with ZFS

No video

Add a DAS to your NAS! USB Direct Attach Storage with ZFS

Рет қаралды 42,785

apalrd's adventures

Күн бұрын

Пікірлер: 90

@HakAtIt 11 ай бұрын

I love that you have a 6 disk stripe with 3 different size and types ... At least you know the risk involved 😊

@johnredmond7085 9 ай бұрын

This is one of the greatest tech vids I’ve ever seen. I have no Freaking idea what just happened but I enjoyed watching. I was going to ask a question but I am way too embarrassed to ask here as Im a total noob . 🎉

@hotrodhunk7389 11 ай бұрын

You are a great teacher. Not too fast not too slow. Just enough details and background information. Thanks 👍 5:22 I just finished the comp TIA exam and man the USB standards are so messed up. I don't know what their marketing department was doing but really they could have simplified it so much.

@saurabhabhishek5203 9 ай бұрын

Thanks for making this no BS , info dense video. Thoroughly enjoyed it.

@apalrdsadventures 9 ай бұрын

Glad you enjoyed it!

@mistakek 11 ай бұрын

12:08, a video on using PBS to backup PVE would be awesome. I've read about this being possible, and it would be great if you could do a tutorial on it, along with a disaster recovery process.

@johndoughto Ай бұрын

becasue of your video on this, and a few others, I have been testing this with the minisforum ms01 - and a bunch of old 6tb. works GREAT to reuse those spinning disk - for now. THANKS

@FTLN 11 ай бұрын

Thanks for the great video once again Apalrd :) Also, thunderbolt 3/4 DAS with SATA controllers are pretty good too, and you don't have to worry about UAS. But they are a bit on the more expensive side.

@apalrdsadventures 11 ай бұрын

I don't have much that supports Thunderbolt, so it would be hard for me to use, but yea they are totally a thing. Some with RAID controllers as well.

@blacktrout 11 ай бұрын

See this is a cool storage set up. I think this might be purchased soon.

@itssoaztek4592 11 ай бұрын

You make great content. You are good at explaining things. Really like your versatility and focus on open source. One complaint though: when you talk into the wireless lave mic it's often difficult to clearly understand what you are saying (that is, at least for a foreigner like me). Many times I have to go back and listen again once or twice to finally get it. It's frustrating but because you make such great content it's still worth the effort. Sure, when I crank up the volume it's easier to understand you. But that makes the 'louder' parts in your speech quite unpleasant to listen to. Use of a sound/voice compressor in your audio pipeline might be an idea to improve sound quality. That's only my opinion and not meant to bash your channel. Keep up the good work. Can't wait for new uploads from you. Good stuff.

@apalrdsadventures 11 ай бұрын

I'll try to get the audio levels better

@itssoaztek4592 11 ай бұрын

@@apalrdsadventures Thank you, Apalrd! Much appreciated! There's probably several different ways to accomplish that goal, but with varying efficiency, complexity and cost. In the unlikely case you have a pile of money in your basement waiting to be spent on crazy expensive audio gear you may have a look at the new Rode Wireless Pro mics. They take away much guesswork and headaches regarding audio levels. Plus real time recording with time stamps.

@eDoc2020 11 ай бұрын

It doesn't matter for the usecase you showed but the block size of the pool you built on the command line was 8k. If you're storing VM images you probably want to use 4k blocks (ashift==12) which is the default in the PVE GUI. If the ZFS block size is greater than the guest filesystem's block size your deduplication won't work consistently.

@wyattarich 9 ай бұрын

That's a great note. Appreciate your comment

@Felix-ve9hs 11 ай бұрын

I think one use case of these DAS enclosures would be if you wanted to upgrade 6x HDDs in your NAS, but didn't have enough space to fit in the new ones. Would be interesting to know how well TrueNAS CORE / Scale handles the DAS enclosures.

@mdd1963 10 ай бұрын

He mentioned near the end of the video that TrueNAS did not 'play well' with passthru of USB drives all on the same path...

@adamm9557 9 ай бұрын

This is a real shame .. my fear with any NAS has always been if the NAS itself dies and I need to plug the drives into 'something' ASAP to use the data (like when the CPU on my old Synology died without warning) .. the DAS coupled with a micro PC running proxmox TrueNAS or something seems like the solution @@mdd1963

@mikepatton7577 3 ай бұрын

Thank you, I was looking for this exact solution!

@VierPuntNul 9 ай бұрын

Explains exactly what I was looking for :) great video... thanks!

@cyclemoto8744 10 ай бұрын

Cheers from OZ

@MRPtech 11 ай бұрын

Was using this kind of setup for about a year. Switch to stand alone Synology 4 bay NAS. DAS started to play up, sometimes after proxmox restart half of drives wasn't showup up. And one day ZFS crap it self with these drives. Moved to Synology NAS about 4 months ago and not one single issue.

@bufordmaddogtannen 4 ай бұрын

Most of the time the PSU is underspecced and the manufacturers rely on capacitors on the PCB where you connect the drives to compensate and allow the drives to spin up. When either the capacitors or the PSU start to fail you get all kind of issues, including data loss.

@dktol56 11 ай бұрын

I don't recall hearing what chipsets are used in this DAS for the SATA-USB bridge and USB hub. Can't find anything online either. ASMEDIA, maybe VIA? Hopefully, not JMICRON.

@ronsafranic5177 11 ай бұрын

I hope you read comments from old videos! it looks like there is considerable room inside that box. I would love to see you take out the USB stuff! and install a Zima Blade (Box) with a HBA card. I use Unraid but you could do a second video just on the NAS Software you could install and the performance of each. Hope you see this and appreciate the idea for a cool video!

@apalrdsadventures 11 ай бұрын

I do read comments on old videos, they all come to my comment feed. It looks like they are using a PCIe board edge connector between their backplane and the USB card, which is similar to their other units (so they probably have both a USB and NAS version of the host card for a single backplane). So I'd also need to replace the backplane with cables, or just use the USB board and connect it to the Zimaboard (or ZimaBlade...)

@ronsafranic5177 11 ай бұрын

@@apalrdsadventures doing it both ways and showing the difference in speed would be cool as well?!

@Mr.Leeroy 11 ай бұрын

HBA would suffocate in there, especially SAS3

@EduardoSantanaSeverino 10 ай бұрын

I was thinking the same thing. There is enough space inside, I know the ZimaBoard could fit in there, but what about going for a cheaper mini PC? something like the lenovo m720q. Lenovo Tiny is really cheap and powerful and quiet. it has USB3 and PCI E port as well.

@valentinedpg 6 ай бұрын

@@EduardoSantanaSeverino That's exactly what I'm planning..get a mini PC and stuf fit in there..maybe take the board out fo the mini PC case for more cooling..

@maxdiamond55 11 ай бұрын

Thanks for the video, it s inspired me to get the 4 drive version and some WD reds for my single board storage needs.

@HappyfoxBiz 3 ай бұрын

Windows: "you cannot use that command" user sweats: "please?" windows: "no!" Linux: "you made a mistake, you didn't use proper punctuation and capitalisation..." user: *checks homework and correctly retypes* Linux: "you cannot do that..." user: "I am root!" *casts a -f command* Linux: "OK!"

@llaumegui 9 ай бұрын

Using UUIDs for disk identification in ZFS is a smart choice! It ensures consistent and reliable disk recognition, making our systems more stable and resilient to changes. A true game-changer for system robustness!

@Knirin 6 ай бұрын

ZFS uses its own internal metadata after you build the pool. It can handle drives being moved around just fine.

@aussieexpat Ай бұрын

That power supply gives me the shivers... I'm very concerned about power quality with any USB based solution, but maybe i'm wrong?

@marshalleq 6 ай бұрын

I wouldn't be surprised to find out this breaks eventually. Every time /I've put ZFS on a device connected by USB ZFS kicks drives out of the pool and a scrub / rebuild etc is required. Takes a few days for that to happen though. I recall some ZFS USB bug about it but not much effort given it's not that common. Would be great if reliable though.

@brettknoss486 10 ай бұрын

I was thinking of using Quebec's OS, with a 6 cart dock, so that I could use VM and hot swap between tasks. If I did something storage heavy, I'd add a tape drive for backup and archiving.

@dirtybrokkoli 9 ай бұрын

Seems interesting, how is the noise level of this device? I live in a small flat and my partner would kill me if i my homelab stuff becomes the noiselevel of a busy street

@pixeldyne_mac 6 ай бұрын

Sure it's risky but sometimes needed. I use single stripe for game installs. If something goes wrong, I can reinstall them.

@zyghom 3 ай бұрын

Hi, I am thinking DAS for TrueNAS but you said there were problems. How about zfs on it as in this video, then TrueNAS as VM, and second disk to TrueNAS from this zfs? So not passing through usb devices but assigning space from this big pool on PVE as hdd in TrueNAS ?

@KinoapparaTom 6 ай бұрын

hey! thanks for the video! does this DAS passthrough S.M.A.R.T. data? Are you able to see temps and other details of each drive inside Proxmox?

@sliiiimjim 5 ай бұрын

I purchased one of these after watching this video as it met my requirements, however I am getting a lot of logs in dmesg about resetting the USB handler, and the speeds are inconsistent. When doing a large transfer from it I will hit about 450MB/s for a few minutes then it drops to about 150MB/s and stays there. Wondered if you had the same experience ?

@Yandarval 11 ай бұрын

Such a shame its priced around the 300 mark. Half that, and it would fly off the shelves.

@mattiaippolito1625 11 ай бұрын

- Do i have any problem if the Truenas server is installed on bare metal hardware and not in a VM? Can I use this box as I would with a SAS enclosure? - In proxmox could I use this with chefs?

@DavidAlsh 2 ай бұрын

I have a bunch of drives of different sizes, speeds and ages that I want to use. Is there HDD pool configuration that I can use with some form of parity that I can use in Proxmox directly? I don't want to install a VM with unraid as I want to share the storage between containers but I am not too familiar with RIAD, ZFS and HDD pool configurations. .... Striped it is :P

@apalrdsadventures 2 ай бұрын

in zfs, you can mix and match redundancy in the same pool. Effectively, zfs is a stripe of all of the vdevs, and the vdevs are one of {single disk, mirror, raidz1, raidz2, ...}. So ideally you can pair up drives in either mirrors or raidz1, and add that up. Remember though that you cannot remove vdevs from a pool, so if you get larger drives later and don't like the same arrangement then you have to create a new pool and copy the data over.

@DavidAlsh 2 ай бұрын

@@apalrdsadventures thanks! I'm watching your other videos which are massively helpful too 😂 This is for a non critical home NAS so redundancy is nice to have but not essential. More important is flexibility to add/remove drives. I was wondering if a simple LVM setup is good for my use case.

@juicycs 9 ай бұрын

How can i learn more about nas im confused by all the info Like trunas unraid Different raids for the hard drives Docker Proxmox Is there a good channel or video to learn this stuff thank you!

@karloa7194 11 ай бұрын

Does this DAS auto power-on after the power outage?

@apalrdsadventures 11 ай бұрын

If it was on before the power outage, in my experience

@sergiop4856 11 ай бұрын

Whats the power consumption with the drives spinned-down? Thanks!

@NetBandit70 11 ай бұрын

For the price of a $300 disk shelf, you could buy a proper case to expand and move your existing NAS into, plus all the necessary accessories. And you wouldn't be relying on rinky-dink USB connectors, or cheap brick power supplies. And if you didn't have a NAS, you could easily build one for that price (minus the hard drives).

@HTOP1982 11 ай бұрын

I think you are missing the target audience for DAS. Some companies don't have the resources to deploy another NAS, or to build another NAS. A lot of use cases don't require a NAS, but just extra storage. Also sometimes you don't want to overload the network, so DAS works in all those instances, and takes very little time to deploy and understand. For instance per day I usually generate 1.2 TB of data that needs to be accessible for 6 months or so to me, but only 1Gb for everyone else. At the end I just push it to tape. Our network is only 1Gbps. So if I transfer it to the NAS I make everyone's life miserable, while via DAS I don't even think about it as an issue.

@HerrFreese 11 ай бұрын

I think @NetBandit70 meant buying a case in which to fit the components of an existing NAS plus the additional hard drives. In this case of expanding PBS Storage that may also be a valid solution. In other cases (i.e. expanding a compact NAS-Box) it may be not. I think it's interesting to see that there are such big USB cases for hard drives.

@juliansbrickcity5083 11 ай бұрын

I just transplanted a Server from a 1U in a 100 buck 4U case that can hold 15 3.5" drives instead of 4. But expanding that further is a bit pricey where I live. The 24 bay solution would be 500 and the 36 bay case is around 700 bucks. Now I am looking into some sas JBOD shelves :}

@NetBandit70 10 ай бұрын

@@juliansbrickcity5083 You have a link on that $100 15 drive case? Sounds really good!!!

@tech-with-nd 11 ай бұрын

true nas might not recognize such stuff. I've experienced in like this one, some times truenas sees only one of two disks, and even restarts and re-connections are not working. And as I understand true NAS does not even support USB-connected drives. That is a small part of knowledge after a short time of usage. Luckily I have not faced issues with corruption or lossing data as described plenty of times on each corner where mentioned disks via USB

@notrhj 10 ай бұрын

TrueNas and USB are not an enterprise solution but home lab is fine. Im running TrueNas scale bare metal not thru Proxmox and the USB support is OK. I’ve be running the Terramaster 6 drive DAS connected to an HP microserver gen 10 plus v2 that only has room for four disks. The DAS using USB 3 gives me another pool with spares (Raid v2) for replication without saturating the network to another server. The DAS usually stays connected at all times but if you want to disconnect it hot I would export the DAS ZFS and let the drive sync scan finish before pulling the plug.

@JJJJJ405 11 ай бұрын

Love your contents. Question, if you didn’t raid 0 these drives, and you wanted to use them as individual drives, is there a way to put the individual drives that are not being used into sleep for better longevity? I was wondering if this’d be possible in a usb multiple bay enclosure.

@apalrdsadventures 11 ай бұрын

The normal Linux utilities *should* work for that (since it should just be a SATA command over UAS), but there's no explicit hardware spin down feature like the large units with staggered spin up.

@PierricDescamps 7 ай бұрын

I'm looking for this either as a 4x or 6x 2.5inch only version, or as a 4x or 6x nvme version. I've seen a video about a PCI express 4x nvme adapter showing drives independently but I do like the idea of usb for easy connection onto smaller hardware like raspberries etc.

@apalrdsadventures 7 ай бұрын

USB3 is not all that fast, compared to NVME or even SATA. So a few 2.5" SSDs will easily outperform the USB3 uplink.

@JosephFerrante 11 ай бұрын

Would this work for bare metal Truenas zfs? Looking to provide as backend vm storage for proxmox cluster either via iscsi or nfs over 10gbe

@apalrdsadventures 11 ай бұрын

It will, unless they have blocked usb disks entirely on their end.

@ayporos 10 ай бұрын

all that empty space in the side.. and they couldn't just internalize the PSU. Hell, you could even just shove that AC/DC adapter in the side, it looks like it'd fit. What a shame.

@apalrdsadventures 10 ай бұрын

They use the same case for a NAS, which has a bigger CPU board inside

@shephusted2714 11 ай бұрын

the only thing better is 2 das - the price of used spinning rust is getting low enough now that everybody van have 10-20 tb pretty easily and affordably but also keep in mind nvme is down in price - many people will just want to go with a bunch of nvme - much much faster and no moving parts - i still contend people want/need dual nas with 40g dual port cards - when you have a lot of big data plus images then the faster networking is a huge plus

@vedranart 11 ай бұрын

Is there any DAS that can be connected to two PCs at the same time? :D

@apalrdsadventures 11 ай бұрын

Not USB, but SAS supports redundant operation. It's purely failover redundant though, not for you to mount the filesystem in two places at once.

@lucasew 10 ай бұрын

12:31 "so I did some testing off camera"

@oscarcharliezulu 8 ай бұрын

Such a pity it uses a power brick. Could have been so much neater and I’m on a mission to get rid of anything that doesn’t power off mains or USB.

@marc3793 11 ай бұрын

When you say that usb passthrough to TrueNAS was not a good experience, what do you mean? Unreliable? Do you think in general drive passthrough doesn't work well or do you think it's related to this device, i.e. the 2 usb hubs used internally in the device?

@apalrdsadventures 11 ай бұрын

Passing through the individual SATA devices at the SATA layer works fine. Passing through all of the USB devices at the USB layer caused the TrueNAS (SCALE) VM to crash qemu on the host. It seems like qemu is struggling to keep up with passing USB frames through the xhci emulation layer with the emulated chipset.

@marc3793 11 ай бұрын

@apalrdsadventures ah okay thanks. This could be a problem for me then because all I have are Intel NUCs, so all of my drives were going to be over USB 🤨 (as they have lots of USB 10Gb ports and no PCIe)

@apalrdsadventures 11 ай бұрын

Without the extra pass-through layer I was able to saturate the 10G link, and with zfs compression I was able to get quite a bit over 1000MB/s write speed, without any USB issues. It's also possible that it's a bug in either qemu, my hardware selection in qemu, or the old kernel that TrueNAS is still using, I can't be sure.

@GeoffSeeley 11 ай бұрын

How is the cooling on it?

@apalrdsadventures 11 ай бұрын

There's a fan behind the drives and it's barely audible

@kymhaniford 9 ай бұрын

Just buy a 20TB HDD and be done with it. Way cheaper in the long run being only 1 HDD to power up instead of 6 and the built in USB hub in the DAS. Get the full bandwidth of the HDD when connected directly to the computer. People work out the pricing, running costs and extra complexity and use the KISS theory.

@rpm10k. 9 ай бұрын

Have you not heard of the word redundancy? There's a reason we don't go with one big drive.

@ThePsho 6 ай бұрын

@@rpm10k.Exactly, and especially true for uses like Unraid. Your huge 20tb drive would essentially be your parity drive, so now you're buying another 20tb drive for your array. But to the above point, and in the Unraid scenario, putting all your eggs in one big basket is setting yourself up for one long ass rebuild. The sweet spot these days seems to be the 4-10tb range. Something like an 8tb parity and a bunch of 4 and 6tbs in the array. Grab a pair of 1tb SSDs for the cache and you're good to go. Then, if (when) a drive goes down, you're looking at hours of a rebuild instead of potentially days.

@thewhitefalcon8539 11 ай бұрын

5:45 don't rotate drives like that while they are spinning!

@ValhalQcoholic 7 ай бұрын

sik RCA audio gen, brah

@jayt-ph1xd 11 ай бұрын

Increase your voice volume.

@zxcvb_bvcxz 11 ай бұрын

Turn your volume up

@jayt-ph1xd 11 ай бұрын

@@zxcvb_bvcxz it's full. Other tech channels are fine with mobile phone speakers. Not this channel. I am always looking for earphones with his videos.

@peteradshead2383 8 ай бұрын

I'm looking to add more storage to my to proxmox home server on a asrock mini x300 , 5900g , 64gb , limited on sata ports but it does have 2 usb3.2 gen 2 or what ever they are calling it this week ? I did get a usb3 to 4 x 2.5 dock , but it passes the drives this :- usb-SABRENT_DISK01_DB98765432146-0:0 usb-SABRENT_DISK02_DB98765432146-0:1 Which you would think is brilliant , but if you add or remove a drive it reorders them , so forcing a resilver just by adding a new drive , so a 3 drive raid and remove a drive and it's dead . Sound like passing the full drive and it's disk/by-id should solve the problem .