ZFS 101: Leveraging Datasets and Zvols for Better Data Management

Рет қаралды 29,831

Күн бұрын

lawrence.video/truenas
Why The ZFS Copy On Write File System Is Better Than A Journaling One
• Why The ZFS Copy On Wr...
Explaining ZFS LOG and L2ARC Cache: Do You Need One and How Do They Work?
• Explaining ZFS LOG and...
Level1Techs ZFS METADATA: SPECIAL DEVICE AND YOU!
level1techs.com/video/zfs-met...
Connecting With Us
---------------------------------------------------
+ Hire Us For A Project: lawrencesystems.com/hire-us/
+ Tom Twitter 🐦 / tomlawrencetech
+ Our Web Site www.lawrencesystems.com/
+ Our Forums forums.lawrencesystems.com/
+ Instagram / lawrencesystems
+ Facebook / lawrencesystems
+ GitHub github.com/lawrencesystems/
+ Discord / discord
Lawrence Systems Shirts and Swag
---------------------------------------------------
►👕 lawrence.video/swag/
AFFILIATES & REFERRAL LINKS
---------------------------------------------------
Amazon Affiliate Store
🛒 www.amazon.com/shop/lawrences...
UniFi Affiliate Link
🛒 store.ui.com?a_aid=LTS
All Of Our Affiliates that help us out and can get you discounts!
🛒 lawrencesystems.com/partners-...
Gear we use on Kit
🛒 kit.co/lawrencesystems
Use OfferCode LTSERVICES to get 10% off your order at
🛒 www.techsupplydirect.com?aff=2
Digital Ocean Offer Code
🛒 m.do.co/c/85de8d181725
HostiFi UniFi Cloud Hosting Service
🛒 hostifi.net/?via=lawrencesystems
Protect you privacy with a VPN from Private Internet Access
🛒 www.privateinternetaccess.com...
Patreon
💰 / lawrencesystems
Time Stamps ⏱️
00:00 ZFS Dataset and ZVOL explained
01:04 ZFS Pool Structure & Special VDEV Types
02:20 Zvol Features
05:18Dataset Features
08:07 Creating a zvol in TrueNAS
09:18 zvol in TrueNAS virtualization
09:45 zvol use in TrueNAS iSCSI
10:59 Dataset use in TrueNAS iSCSI
#zfs #storage #truenas

Пікірлер: 76

@GeoffSeeley Жыл бұрын

Tom's status: Degraded

@LAWRENCESYSTEMS Жыл бұрын

Having two arms does offer me some level of redundancy 😜

@beardoftears Жыл бұрын

The Other Arm is not a backup. You need an off-site prosthetic just in case.

@Yandarval 10 ай бұрын

However, he looks like he is resilverd.

@mithubopensourcelab482 Жыл бұрын

Wish you a very fast recovery. Your dedication to work is very much appreciated. You are a special person indeed.

@JohnAngelmo Жыл бұрын

Thanks for a great video and take care! You will get back on your bike in no time!

@mattwhite532 Жыл бұрын

Great video! Thank you for taking time out of your recovery to do this. I don't want to overstep, but as a healthcare professional, I'd be remiss not to suggest getting more of a cushion under the strap, up by your neck. The weight of your arm on your shoulder/neck area over time can cause additional undo pain and discomfort. I'm so grateful to you for your time w/ these vids! Believe it or not, they've saved me from making many silly/stupid tech choices.

@ajhieb Жыл бұрын

I hope you have a speedy recovery and thanks for all the great content!

@LAWRENCESYSTEMS Жыл бұрын

I appreciate that!

@Felix-ve9hs Жыл бұрын

I once built a TrueNAS CORE server with all flash storage and used it for testing with four VMware ESXi (one for vCenter, three for VMs). It was actually really fast and ended up running rock solid for over 100 days because I upgraded to a never release.

@jayjarrett732 Жыл бұрын

I feel for you bro. I wish you a speedy recovery. Thanks for another great informative video. You are my go too guy!

@LAWRENCESYSTEMS Жыл бұрын

I appreciate that!

@darylnd 3 ай бұрын

Great videos, Tom! I've been binge watching, including revisiting a few I've seen before. I'm rebuilding my unRAID home server after multiple drive failures (two HDD, one SSD, and the boot USB had to be repaired--so much for the supposed improbability of losing two drives at once) and switching to TrueNAS Scale. I did have some experience with FreeNAS. The box itself is modest: it's the first-generation Ryzen 5 I built, years ago, as a desktop system, 16MB RAM (for now). I'll be using four, 4TiB Seagate Ironwolf HDDs in RAID-Z2, and a SATA SSD for boot.There's no NVME, just a m.2 slot that uses one of the six SATA lanes, so no need to spend on that. That leaves one SATA lane open. I'm thinking of pulling the now unnecessary video card, plugging a Sun F80 into that slot, and making a pool of two mirrored stripe VDEVs for virtualization. Hope you've healed up by now. Falling off isn't important: getting back on is.

@LAWRENCESYSTEMS 3 ай бұрын

Thanks, all healed up and riding again!

@olegurvingurigard8373 Жыл бұрын

Hope you get back on the bike quick Tom, I love riding bikes in the woods too. Nothing more relaxing, currently running a drz400sm but wouldn't mind a spare set of knobbies for better traction!

@zacharysandberg 2 ай бұрын

DRZs are great bikes! I also had a 400SM

@ryanmalone2681 2 ай бұрын

Adventure riding has broken a fair few bones of mine too. Hope you’re back on by now. Appreciate all these videos about the TrueNAS concepts as I plan for a migration from Unraid.

@m3ellis Жыл бұрын

Oof, been there before as far as the clavicle is concerned (cycling accident). Hope it heals fast. Thanks for all the content 🙏.

@LAWRENCESYSTEMS Жыл бұрын

Thank you

@justisstudding2101 Жыл бұрын

I was just recently involved in a Hit n Run on my bike coming home from work. Broad side. Smashed my left foot. Broken Heel, Ankle, and now the Ankle is having serious healing issues. I have to return to get antibiotics, pain meds. Driver is still unknown. Good news though. I got my equipment in the post office. New drivws to set up my TrueNAS Scale. I'm excited...

@bikerchrisukk Жыл бұрын

Sorry to hear that and hope you recover eventually (it can be a long road, but stick with it, do physio, etc.)

@justisstudding2101 Жыл бұрын

@@bikerchrisukk thanks.

@Tsuroerusu 8 ай бұрын

Watching in early November, 2023 and I hope your recovery is coming along well, thanks for the video. Best wishes from Denmark.

@MactelecomNetworks Жыл бұрын

Get well soon Tom!

@LAWRENCESYSTEMS Жыл бұрын

Thank you

@itssoaztek4592 Жыл бұрын

Tom, thank you for yet another great video. Most of what I know about Truenas and ZFS I learned on your YT channel. Suggestion for another video: What are recommended steps to test a backup of a system drive? I do not want to overwrite a functioning system drive with a new backup just to test if the backup works fine. Is the only solution to test the backup by writing it to a new physical drive and swapping that drive with the existing system drive? Sorry to hear about your unfortunate bike accident. I wish for a speedy recovery!

@bikerchrisukk Жыл бұрын

Are you talking about a boot drive or zpool?

@itssoaztek4592 Жыл бұрын

@@bikerchrisukk Sorry, I realize my post was not well written and confusing. I meant it to be a suggestion for Tom to consider making a general video on a proper testing procedure for backups of system drives. Don't know if such a tutorial can be made general enough to cover a variety of systems (TrueNAS systems, system drives of regular PC) or if it would be more advisable to limit the scope and make it for a specific system only. In my particular case, I am least worried about the TrueNAS boot drive, slightly more worried about the zpools and most worried about the backup of a Windows PC boot drive. If my thinking is correct, the TrueNAS boot drive can be easily recovered without fear of data loss (and without the need for an additional disk) as long as I make sure to save a configuration file. I have two NAS (one for backups). So I guess I can use the second NAS to store a replicate of a zpool of the first NAS and restore it for testing purposes to a new zpool on the second NAS (provided that the second NAS has sufficient storage capacity). At least in theory. My NAs is fairly new, so I have not actually tried it yet to restore a backup. The system drive on my PC is a much worse case and a totally different story for various reasons. Would be quite a revelation for me to see how people test their backups on a Windows PC.

@nid274 Жыл бұрын

great video...and hoping a speedy recovery!

@LAWRENCESYSTEMS Жыл бұрын

Thank you!

@markpreece1565 Жыл бұрын

Get well soon !

@prahe86 9 ай бұрын

Great video, get well soon

@chinchye Жыл бұрын

crazy you are still doing videos in sling! Get well soon!

@LAWRENCESYSTEMS Жыл бұрын

Thanks, I guess you could say I'm doing these videos.. single-handedly 😜

@mattvisaggio Жыл бұрын

Would love a video for setting up wg-easy for remote access using TrueNAS app but then general next steps with router and phone app to access it. No one has done one yet on YT that I can find...

@DarrolKHarris Жыл бұрын

get better soon

@austinwilson930 Жыл бұрын

Tom, how is your ix_applications dataset encrypted? I had a pool set to be encrypted, and truenas would never encrypt the auto created ix_applications dataset.

@joshua_lee732 Жыл бұрын

Hope your arm is ok and gets better than it was. Have no context on what happened and won't inquire further. I am curious though on how this would be configured via command line rather than TruNAS. I always feel limited when I look at the TruNAS interface.

@RaulCastillo Жыл бұрын

I guess you didn't watch the video

@joshua_lee732 Жыл бұрын

@@RaulCastillo I made the comment in the first 15 seconds.

@SnizzleTille Жыл бұрын

Does compression do something for you if you have a thick provisioned zvol with compression enabled? I tried that a few days ago and I see any advantage in doing so.

@entelin Жыл бұрын

Yes, it works, depending on the kind of data you should see some performance improvement on spinning disks. Keep in mind that the filesystem on the zvol will not itself see the saved space, but rather the zfs pool under it.

@salvanation Жыл бұрын

Hi I am using TrueNas Scale and can you tell something about datasets and the Case Sensitivity option in a next video? My client is Windows and because I chose SMB when creating all datasets, all datasets are on case insesitive. Now I would additionally work with Linux and I don't know if I can leave everything as it is or should I better redo all datasets with case sensitive and choose generic as share? Or what happens if I switch completely to Linux and then want to use NFS or should I stay with SMB and make additional settings for SMB so that I can work with windows and linux at the same time? The manual also mentions the option "mixed" but in the GUI I can only choose between sensitive and insensitive.

@LAWRENCESYSTEMS Жыл бұрын

Whenever I use a Dataset for an SMB share I use Case Sensitivity:ON

@salvanation Жыл бұрын

@@LAWRENCESYSTEMS Thx! Strange if I create a new dataset and choose Share Typ SMB, the Case Sensitivity is changing to Insesitive and the box is now greyed out? Is this a new behavior? Looks like I need to leave the share typ at generic at first to create the dataset.

@GXShade 11 ай бұрын

I hope recovery goes well :D

@gjkrisa Жыл бұрын

Never really noticed you had long hair. Kinda look like the Gibson from hackers or his tag the plaque

@3k3k3 Жыл бұрын

Ouch Tom.. be careful out there, you have a family of 294.000 that is worried about you :) What is you take on Scale vs Core, it seems as if Scale is supporting more hardware? But can and should i run it instead of core, and expect it to be just as reliable?

@LAWRENCESYSTEMS Жыл бұрын

For NAS functions Scale is fine

@3k3k3 Жыл бұрын

@@LAWRENCESYSTEMS so where does Core fit in?

@LAWRENCESYSTEMS Жыл бұрын

@@3k3k3 As a very stable NAS that has been around for a long time.

@d3wy Жыл бұрын

Ouch on the arm :(

@d3wy Жыл бұрын

Hope you have a speedy recovery Tom, take your physio seriously.

@wpoole10 11 ай бұрын

Tom or ANYONE, How can I setup a permissions in a dataset to where I (dad builtin_admin) has full control and want my other users (wife and kid builtin_users) to be able to read and write files if they want, but not be able to delete anyone's files ONLY the ones they created/own? I am able to have each user write files to the dataset but the user that creates the file cannot delete their own file. If I give them modify permissions they can delete anything in the dataset. How do I fix this???

@wyattarich 8 ай бұрын

look up setuid and setgid bits

@Nostang3 Жыл бұрын

I broke my collarbone once so I feel your pain.

@Mikesco3 Жыл бұрын

I wonder if any of your clients freak out whenever you have an accident 😂

@glmchn 11 ай бұрын

8:37 "we don't need sync on this" please why ? doesn't seems that obvious to me

@LAWRENCESYSTEMS 11 ай бұрын

Sync is a more complex topic and is covered here kzfaq.info/get/bejne/nNVjeKqmqNneoHk.html

@jGRite Жыл бұрын

Well, at least you messed you up, and no one else messed you up.

@randleqgod Жыл бұрын

Get better. Saw your LinkedIn post. Definitely a freak accident.

@rpWeb3 Жыл бұрын

I also broke my collar bone on my kx125 .. good luck sleeping.. ugh

@danielchristie6546 Жыл бұрын

Why are you in a sling tom?

@LAWRENCESYSTEMS Жыл бұрын

It's explained in the video

@NickyNiclas Жыл бұрын

13:27

@danielchristie6546 Жыл бұрын

@@LAWRENCESYSTEMS i know, so funny the way you said that at the start

@dooetboog Жыл бұрын

get well quick... resilver quick.

@LAWRENCESYSTEMS Жыл бұрын

Thanks

@eidodoos 11 ай бұрын

he unmount his hand. hope he can mount it back sooner.

@richardbennett4365 Жыл бұрын

ZFS is a ROW System not a 🐮 cow.

@LAWRENCESYSTEMS Жыл бұрын

If you're looking for a more specific answer it's a COW for general writing and a ROW for snapshots

@richardbennett4365 Жыл бұрын

@@LAWRENCESYSTEMS, copy-on-write requires three I/Os per write; however, and it is why btrfs and ZFS are redirect on write systems, only performs one. These systems are trying to save writing, so that's why they redirect on write instead of copying on write.

@ewenchan1239 Жыл бұрын

Sorry to hear about your arm. I hope that you will feel better soon and I wish you a speedy recovery! Three questions: 1) If the dataset is used either "by application" or because you want to set different permissions "by dataset" level -- but if you have only one application (dumb file server), for one user, and therefore; one set of permissions -- would I be correct in that use case/usage scenario - datasets wouldn't really offer much in the way of benefits? 2) If snapshots only record the incremental data, will the total volume of disk space used by the data and its snapshot be equal to each other (i.e. you have initial set of data, take a snapshot, add more data, take another snapshot, therefore; the total volume of disk space used: data + differential = data@snapshot + incremental@snapshot?) I'm still a little fuzzy in regards to the comment that you made in the video: "you can create as many of these as you have space for". And what I am still unsure of is whether the total volume of disk space consumed would group regardless of whether you're doing snapshotting or not, or if there is a little bit more of an "overhead" in terms of the total volume of disk space consumed would be a little bit more than the data itself. Your help in clarifying that is greatly appreciated. 3) If I have a ZFS dataset - how would I back that up to LTO-8 tape? Thank you.

@LAWRENCESYSTEMS Жыл бұрын

1) if you don't need them then don't use them they're not required if you don't have a use case. 2) correct that snapshots are only the difference from the previous snapshot therefore only take up that much data. Creating data sets does take up a few kilobytes which means you could create several thousand of them without a problem on a reasonably sized system 3) I'm not aware of anything in ZFS that automatically backs up to lto

@ewenchan1239 Жыл бұрын

@@LAWRENCESYSTEMS Thank you. 1) Some people were trying to educate me about the benefits that datasets brings beyond those that you've mentioned in your video, so I was looking to get your insights/perspectives in regards to this kind of a recommendation. 2) Thank you. This is good information to have and know. 3) I haven't really figured out a particularly great way to be able to backup block devices (e.g. zvols) to tape because the files aren't exposed to the TrueNAS host (unlike datasets, because I can see the files/folders in a dataset, on the host system). Right now, I am just using rsync to send the files/folders over to tape (over NFS), but that's causing a high CPU load average, so I didn't know if there might be a better way to do this vs. my current process.