What do you guys think of ZFS root on LUKS2, as opposed to LVM+EXT4 on LUKS2? Does it make sense on an old, slow Thinkpad?
>>108570420No, it's a NAS filesystem
>>108570420>LUKS on an old, slow ThinkpadAll boils down to what cpu and it's support for AES, even slightly older cpu's have no problem with this..
>>108570636It's more about ZFS than LUKS itself. I've heard that ZFS eats CPU and RAM like there's no tommorow.
>>108570420you don't need LUKS zfs has native encryption, in fact using luks 2 on top would be a bad idea as zfs works best when it has direct control of a drive / partition.i have my root on zfs, it's the only fs i use except exfat for usb drives.but even external hdd i sometime use are zfs.literaly the best fs there is now, it's sometime a bit annoying to update because it's an out of tree module but it's imo worth it.
>>108570673I've heard that ZFS native encryption uses the same outdated key derivation algo as LUKS1, LUKS2 is better in this regard.
>>108570646>>108570673same guy, if you don't got a shitty cpu it's fine.it takes about 1 to 2GB of ram (which you can make smaller with settings, it's just caching).my cpu is almost always between 0 and 1% and i run a shit ton of stuff on my machine.>>108570684doesn't realy matter, the drive is encrypted using AES which is pm the best thing there is.the real encryption key is encrypted with another key that can be derived from various sources, ie password file etc.that's also why it's cheap to change the password or way you decrypt your thing.and honestly it doesn't realy matter how the key is derived, the drive is encrypted with top notch crytpo, and you can roll your own key derivation if you want.ie you can just give it a file or the key itself for the decryption of the master key, how that file is generated is your choice and can be done programaticaly with whatever meme cypher / derivation technique you want.ie if you realy wanted you could have your computer fetch the decryption key from your nas, ask for a password to decrypt it with whatever scheme you want then use it to decrypt the pool's master key.if you did that you could for ex have a laptop that can be blacklisted from booting at all if you lost it or whatever.
>>108570673Is it worth using it on a single drive? What benefits do you get over standard Linux filesystems?
>>108570721I want it for snapshots, compression, bitrot protection, having native LVM, and being less gay&retarded than btrfs.
>>108570721>Is it worth using it on a single drive?i think so.>What benefits do you get over standard Linux filesystems?you can make snapshots instantly and rollback as well.ie i just type zsnap (which is a script i made), literaly takes like 100ms.the snapshot doesn't take any space (except if you delete data it's obviously still gonna be on disk until the snapshot is deleted).besides snapshots you have datasets which are handy, ie you don't make partition, you make datasets, that can be moved, renamed etc.so my home is a dataset, the root is another, i can dynamicaly change my root if i want, they can each take all the remaining space if i want, but you can also enforce quotas.then you have native disk encryption, which is self explaining.you also have native whole disk or per dataset compression, which is also handy, saves a lot of space on non media stuff, i have some dataset that 2x less space thanks to it, and that's just lz4 (which is fast), gzip would be even better ratio.also the compression means you get an effectively higher read and write speed to your drive as the bottleneck is the i/o not compressing / decompressing especialy with lz4 (which is near instant).also, you can send datasets over the network / through ssh on other machines, which is nice for backup because it's a byte for byte identical copy, including all timestamps, ctime, btime, mtime and even inode number etc, which you couldn't save with just rsync.oh, and lastly, even on a single disk you can still scrub because it has per block checksum, so even if it can't repair the block (unless you used copy=2 or something on a dataset) you could still know which file got corrupted.
>>108570721>>108570751also that's why here >>108570673my stock pool is a mirror of 2 4TB ssd, this way i have corruption protection and redundancy, altough it's not as necessary as on HDD, zfs did save me from some bitrot in my previous computer which had 2 hdd in mirrors.
>>108570755>>108570751>>108570721also, zfs send | zfs receive.is realy handy, when got my new computer last year, i just copied my whole home dataset over ssh through zfs send.the command was something like zfs send snapshot | ssh newpc zfs receive.and it's a byte for byte copy, even inode numbers are preserved, and you know there was no corruption in transit because zfs checks for it.i can also do incremental snapshots backup, ie if i make a snapshot, i can send it over the network to my nas, and if i now make a second snapshot i can send just the diff to the nas.because of that it's pretty cheap to do and i can run it daily as a cron job.
>>108570751interestingdo you happen to know if zfs is resilient against abrupt power loss?and how do you handle workloads like for example libvirt virtual disk images, where on something like btrfs you must disable COW (although I think that is done automatically these days), or databases, torrents, etc?
>>108570819had tons in the last decade, never had any issues, even unplugged my computer whilst on a few times in the past.zfs is literaly built to not lose your data, afaik it has multiple copies of the metadata table thingy.also all transactions are atomic, so if it lose power mid write no damages should be there, data is written to new blocks and then a pointer is atomicaly updated, it's not an fs that has "partial" or invalid states.you can even see videos of people literaly unplugging the hdd from the computer whilst copying data to it and they are fine.zfs may complain that the pool was not exported properly the last time but it won't have an issue importing it again.i did try btrfs in the past and it caused me a lot of trouble, i had a random kernel panic and then the data was unrecoverable.zfs has just been the most reliable fs i've ever used.as i said before, it is indeed a pain in the ass to have an out of tree module and i wish it was built into linux and didn't have those licensing issue bullshit.but even that annoyance is worth what i get out of it imo.
>>108570819>>108570910>and how do you handle workloads like for example libvirt virtual disk imagesso for libvirt, if you are running linux you can literaly set a dataset as your root so you don't even have to bother with a qcow taking more space than it need.if you do for windows for ex, you can just give it a qcow, also because it's zfs and lz4 is enabled by default, zeros don't take any space so a 200GB qcow that's barely filled won't take much space.>where on something like btrfs you must disable COWyou don't have to, it's not shitty like btrfs.also with zfs you can also make zvols which are basicaly block devices but part of your pool, so if you wanted to have something that behaves like a disk partition, i never realy used it though.>or databases, torrents, etc?never had issues either.i run postgres, works fine.also i use lxd a lot, what i like is that i don't have to make a lxd volume that's some bullshit partition, lxd is zfs aware and can use dataset, see picrel:
>>108570819>>108570919oh also the CoW is actualy realy nice.i can instantly do a zfs clone on a dataset.and it doesn't take any extra space.i can then delete data from it etc.that's how i sent a bunch of my stuff over to the new computer.i cloned some big ass dataset, i removed the file i didn't need in the new computer from the clone, then made a snapshot of the clone and just sent the clone.so instead of sending 5TB i just sent like 300GB.
>>108570734>i want it for <things btrfs has>>>108570751>what does it have over standard linux filesystems>describes things btrfs has>>108570777btrfs has send/receive support as welldo you guys even know btrfs exists? did you switch to zfs in 2009 and think we're still using ext4?
>>108570972brtfs is the only fs that I've ever heard people complain about data corruption on
>>108570972>>i want it for <things btrfs has>btrf is utter trash though.also tons of things zfs has btrfs doesn't, it's just worse in every meaningful ways.the only pro it has is that it's not an out of tree module, but i don't give a shit because it's trash.>>describes things btrfs hasagain, no.btrfs doesn't even have a working equivalent to zraid and draid.>btrfs has send/receive support as wellwhen it doesn't corrupt your whole drive maybe.>do you guys even know btrfs existsyes and if you read what i wrote you'd know i've lost a drive to it.literaly kernel panic out of nowhere (because of btrfs) and then the data was unrecoverable.the drive was physicaly fine and working.but btrfs was not capable of reading it again or recovering it.>>108570979this one gets it.anyway the ergonomics of btrfs are just shitty compared to zfs and it lacks tons of features zfs just hast, heck it doesn't even has native encryption, let alone sending encrypted datasets.
>>108570972no one that cares about their data would use btrfs.at that point you may as well just store your install on ram and periodicaly make a copy to disk rather than that meme fs, probably would be more reliable and lead to less data loss.
>>108570972>btrfs has send/receive support as wellidgaf, 99% of the send receive i do is to my nas, and i'd never use btrfs for a nas, even if it was not constantly corrupting itself, it has no equivalent to zraid1,2,3 and draid.
i'm not touching zfs until orakike cuts their licensing bullshit and lets me use it out of the box on an actually good distro
>>108571043nixos has out of the box support for it.arch doesn't have out of the box but it's pretty easy to get it, just use the archzfs repo.ubuntu has out of the box afaik, i don't think it's a good distro but idk what you think is one.
>>108571052too much work. i'd never use arch on a NAS, server etc but btrfs just werks on a machine i would use arch on>ubuntu has out of the box afaik, i don't think it's a good distro but idk what you think is one.exactly the problemit's pure retardation honestly, oracle has almost no involvement in zfs on linux dev
>>108571066Void has first class support for ZFS and I don't see a reason why it couldn't work on a NAS. I don't now about Debian or Devuan but I would be surprised if they don't support it. What do you consider "good distro"?
>>108571043I've run OpenZFS on Linux for ten years and Oracle has never said anything to me, not even when I visited their corporate headquarters in Austin with my devices running OpenZFS.
>>108571034all i'm saying is that anon is asking for what zfs has over standard linux filesystems, that includes btrfs. zfs has features btrfs doesn't, so why list off features btrfs does have?been using btrfs on multiple machines since 2013, and while i would not have recommended it back then as it was still very new and did have some corruption issues, it has been perfectly stable in more recent yearsi also want to be clear i have nothing against zfs either, i used zfs prior to btrfs in fact. i like what it has over zfs, such as more flexible raid, more flexible support for mixed-size hdd's, the fact i don't need to handle an out-of-tree module, the fact it can use linux's native IO cache, etc. it doesn't have native encryption, though that can be worked around, and it doesn't have native l2arc, though i have worked around that with bcache with some success (i don't do that anymore as ssd's got bigger and i just put everything i want high speed access to on ssd's entirely)
>>108571082If you're going to use btrfs on root, you need btrfs on the backup pool, so you can send/receive seamlessly. On the other hand, if you use ZFS on the backup pool (which is a better idea), you need ZFS on root. So ZFS wins I think, and as a bonus you're gaining free know-how on managing ZFS over time.
>>108571097>if you use ZFS on the backup poolthis is the only option as btrfs doesn't have a working zraid 1,2,3 and draid equivalent.also zfs has zraid expension now which was the one feature i was missing.
>>108571127Yes that's what I'm getting at. ZFS backup forces ZFS root, BTRFS really has no place in the end.
>>108571077>What do you consider "good distro"?not ubuntudidn't know void had good support for it tho. i'm used to systemd admin but i'm open minded.
>>108571082when he asked the question i did not consider btrfs as a standard linux filesystem, yes it is a linux filesystem and built into the kernel, but it is very much so non standard and perpetualy beta / unstable software.and filesystems are one of the rare things i don't want to use unstable software for.i'd rather lose all the feature zfs / btrfs have in common and be on ext4 than being on btrfs as i don't trust it to be able to keep my data reliably.
>>108571141If you want systemd for a server it's Debian or Fedora, I would recommend Void though.
>>108571097i would still recommend zfs for more "serious" setups, like if buying 20 identical hdd's to put into a dedicated rackmount storage server with ecc ram is not an issue for you then yea, go with zfs, and as you said, if you already have that then having zfs on auxiliary machines makes sense too for send/receive.for most home users however i think btrfs works in more places, like just a handful of mismatched hdd's is better handled by btrfs>>108571142just because you don't believe btrfs to be stable doesn't mean it's not stable. i really don't want to start a fight but it should be noted that zfs isn't perfect eitherhttps://www.phoronix.com/news/OpenZFS-Encrypt-Corrupt
>>108571144yeah no shit i'm talking about zfs support not general distro recs. zfs is usually a pain in the ass to use as a root filesystem because of license idiocy. it's annoying enough that companies like red hat bother to make silly hacks like stratis to try to replace it.proxmox is the only other debian-based system i know that makes it easy and i don't really need vmware: poorfag edition
>>108571149first anon isn't me.but for the send receive part i think zfs is generaly more suited for nas, and because of it i'm using zfs everywhere, i don't want to mix and match them.>corruption issuefair enough, still, i've never had any issue with zfs, btrfs made me lose data, not important data but still, i cannot trust it now.it shat the bed not even 2 months after i started using it.my point is that yes, zfs isn't perfect either, but it has vastly less issues imo.also the issue you linked was kind of a niche usecase, ie sending encrypted datasets in non raw format, which i've never did nor see the point in ever doing.it's not like corruption that just happens whilst you are normaly using the fs, it's corruption following an action.most btrfs corruption stories are with user literaly doing nothing, it's not a command they typed that has a small bug leading to corruption, it's just btrfs running normaly and killing itself randomly.
>>108571158>zfs is usually a pain in the ass to use as a root filesystem because of license idiocyThe only issue is that every upgrade you need to recompile the DKMS module, but that's not the issue, you can automate upgrades in the background because you're using ZFS, so if anything breaks you can just restore the snapshot. ZFSBootmenu makes this all seamless.
>>108571158>zfs is usually a pain in the ass to use as a root filesystem because of license idiocyi mean it's annoying but it's not *THAT* bad, sometime i can't update the kernel for a few days and that only happens on bleeding edge distros, there is worse.i update like once a month anyway.
>>108571173>>108571177i don't use the dkms version and i'd recommend against it as sometime dkms will compile succesfuly but it'll break at boot, at least that happened to me a few times when i was still using dkms.i just use zfs-linux non dkms, and it has worked like a charm, it may freeze updates if you are on a rolling release distro but generaly it's just a few days and you can always ignore the kernel update.
>>108571181With snapshots and zfsbootmenu you can just boot the old kernel or snapshot, nothing breaks
>>108571189fair enough, i just rather not being able to update a bit longer than having a failed boot.also i'm a bit paranoid about some dkms sheenanigans causing unexpected behavior and whatnot, not sure if that's a legitimate fear though.
>>108571173he might be talking about zfs on root which is much more complicated to setup than zfs on data drives because it's out of tree and you have to manually do all the partitions and shit like on arch
>>108571189You can do the same without DKMS, so that's a moot point.
>>108571079You should ask Oracle when they're gonna start shipping Oracle Linux with native root on ZFS.
>>108571158Ironically, RHEL is one of the better distros for ZFS, if only due to the precompiled OpenZFS kmod. Other than that, you'll pretty much have the same drawbacks on any distro.
>>108570420not sure about encryption, but zfs pool from 2 SSDs runs just fine on T440p
>>108570673>scrub repaired 0B in 00:09:57how the fuck did you get it to run so fast? is your pool completely empty? I have a 36tb (of data) pool and it takes around a day to scrub.
>>108572086skill issue
>>108570420LUKS ext4 is fine just go for that.or else you could just go for encrypted btrfs and watch you entire laptop freeze for a minute when you delete a 50gb big vm image.
I'm just not gonna use out of tree kernel modules because they're a pain.
>>108570420>old, slow Thinkpadlunix users are all poor and unemployed
>>108571935Nice.
I have a 2017 thinkpad that I put mint on with a root zfs, but I think mint removed that option from their later distro versions, the bastards. Still it hasn't given me any troubles.On my gentoo PC I used to dual boot so I have an ext2 boot partition with grub but root zfs for the rest. No issues. I have a raidz2 external storage setup that I have encryption on and is currently undergoing raidz expansion. ZFS is great, I can't imagine choosing a different filesystem again unless it's a throwaway device or something niche like a boot partition. From my friends' reports btrfs seems fine for a completely managed synology NAS, I wouldn't use it for anything else.
>>108572086it's 1TB on mirrored nvme
>>108571935do you use root on zfs, and what do you use the laptop for?
>>108570673zfs native encryption is much slower than luks unfortunatelyif you want to do a mirror on a chinkpad, two luks drives with zfs on top is much faster
>>108570420zfs on root can be a pain to setup, but zfs is super useful for all the same reasons people use it on servers.being able to boot multiple different OSes is nice, as well as being able to manage your data logically rather than being forced to stay along partition lines. sure lvm can do some of that, but its way less mature and way more esoteric, plus if you are doing raid/luks/lvm the slowdown is noticeable. zfs can in theory do all of that, however i would still use luks rather than native encryption
>>108572086if you look closely, those are nvme ssd's.the 8TB is gen 5 and the others are gen 4 also this is a pretty recent install, they are like 25% full.
>>1085771134TB but yes.
>>108572086>>108577113>>108578585also my 8TB mirror of hdd took about 6H in the past.so yea, hdd's are much slower.anyway i scrub like once a year.
>>108578292>i would still use luks rather than native encryptionwhy?also i'd not install zfs on top of lucks.if i realy wanted to use luks i'd use it on top of zfs ie through a zvol, i don't need everything to be encrypted anyway.
I dislike ZFS because it is GPL incompatibleI like ext2, reiser and (for games) XFSProbably, it's better than Btrfs, but still...
>>108578611>muh GPLfuck the GPL honestly, it's a terrible license that only benefit megacorps.
>>108578611>>108578660and it comming from a jew is all you need to know.
>>108578603because zfs native encryption leaks a suprising amount of metadata, and because zfs on top of luks is much, much faster
>>108578253This hasn't been my experience. I get my expected hdd and nvme performance based on their specs for read and write with encryption and zstd compression. What slowdowns are you seeing, in what configurations?>>108578712Besides pool, dataset, and maybe snapshot names, what other metadata is leaked?
>>108578712>leaks a suprising amount of metadatait leaks pool creation time, dataset name and creation time and pool history.that's not sensitive data imo but i can get it.>zfs on top of luks is much, much faster[x]but i'd be happy if you got some data on that.thing is, i don't want to encrypt my whole pool anyway, ie i don't need steamshit to be encrypted.
if you don't care about the specific things zfs encryption leaks, then it's not a big deal. with luks, everything but the luks header metadata is encrypted. it's still a tradeoff to consideras for speed, if you say so, perhaps it's improved since my install.
>>108570420You want your root filesystem support baked into the kernaljust juse BTRFS on LUKS>>108571913neveroracle linux is just white label RHELits like rocky/alma but better
>>108578949>You want your root filesystem support baked into the kernalThis is completely irrelevant, Linux kernel is not monolithic.
>>108571913If I did that, my honest expectation is that most people I talk to would not even know what ZFS is. Moreso than most big tech companies, people at Oracle seem to know a decent amount about their little slice of reality, but know basically nothing about what Oracle does or has done at a broader scale.That's going to be somewhat true of employees at any large company, but it seems utterly pervasive at Oracle.