Per title. From quick googling, I understand that zfs is a filesystem that provides some kind of software RAID and resiliency (eg CRC checks at the block level). Is my understanding right that it just good for detecting storage corruption, but not really self healing?

My home server just for a mini storage/ media server and I am broke to afford RAID. So at the moment, it’s just 2 consumer grade drives .

So pros with zfs I can think of, I get slightly better data resiliency.

But after reading the docs, the cons for me are that a) it’s not so straightforward/ somewhat complex to tune and use, b) it’s not default in mainline linux, and c) slightly lower performance?

What do you think and is it worth learning to use it vs just slapping a plain xfs/ ext4 ?

  • Dingaling@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    17 minutes ago

    Personally, I don’t use it, but only because I’ve never needed to. That and its legendary need of ram to be performant. But it has some nice features and if those sound good to you, go for it.

  • Shimitar@downonthestreet.eu
    link
    fedilink
    English
    arrow-up
    2
    ·
    49 minutes ago

    It’s my personal opinion but: no

    Btrfs is integrated in the kernel already and pretty solid. I have been running it for the last few years. I use Linux mdarray to create a raid5 and on top of it, format btrfs.

    Never had issues, and yes it did catch data corruption (and fixed it) so I am pretty happy .

    Zfs on the other hand, needs specific kernel patches and I don’t like that too much.

  • robber@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    30 minutes ago

    I’m happy with ZFS, on Debian it’s not that complicated to set up I’d say. Just make sure your consumer drives are not DM-SMR drives, otherwise the performance is bad, especially when “scrubbing” (data integrity verification) or “resilvering” (updating a disconnected, or replacing a failed drive).

    See here for more info.

  • neidu4@piefed.social
    link
    fedilink
    English
    arrow-up
    2
    ·
    60 minutes ago

    Depends on your requirement and hardware really. If you don’t have the RAM for zfs, but a proper RAID controller, just go for raid 6. (The ones crying about raid and dataloss never use a proper RAID controller). If you do not have one, then zfs is a pretty decent choice, but it benefits greatly from having loads of RAM. If you need something ridiculously large, beegfs is your friend.

    For me, personally, I use zfs to serve a volume of 24 physical drives at home. I find that zfs sits in the middle where it’s flexible enough to serve a lot of those intermediate situations where mdraid isn’t enough, you don’t have a (proper) raid controller, and it’s not enough disks/machines to justify beegfs.

    With 2 drives I’d just go mdraid.

    • Shimitar@downonthestreet.eu
      link
      fedilink
      English
      arrow-up
      1
      ·
      38 minutes ago

      If you don’t have ram for zfs, go with btrfs, it’s pretty much lightweight in my experience. But never tried zfs so cannot compare

      Also btrfs has zero licensing issues and it’s standard in the Linux kernel without the need for additional patches like zfs.

  • Zwuzelmaus@feddit.org
    link
    fedilink
    English
    arrow-up
    7
    ·
    2 hours ago

    2 legends about ZFS seem never to die: that it is difficult in normal linux, and that it needs much extra RAM.

    Truth is, it is availible in the normal kernel now and several distros offer it from start. There are tools for maintaining it, and after a while, you will find that it needs less routine maintenance that most traditional file systems.

    Truth is also that it can absolutely work with less RAM. That thing is only important for servers with a high load. There it is recommended strongly to have lots of RAM, but even there it is not for the sake of ZFS itself, but it is for the performance of your server. It makes good use of all that RAM in ways that other file systems cannot.

    Is my understanding right that it just good for detecting storage corruption, but not really self healing?

    If you use it on just 1 drive, and the default setting, then this is true. It still detects bit rot, but it cannot repair it automatically. But it has also an option for the self healing on only 1 drive (for the price of drive space then).

    If you use 2 mirrored drives, or more as a raidz, the self healing is always there.

  • BeigeAgenda@lemmy.ca
    link
    fedilink
    English
    arrow-up
    1
    ·
    43 minutes ago

    I chose btrfs over zfs, because you can add and remove disks from a volume (I’m not sure if zfs has this feature now) I use this to move the volume from old to new disks.

  • ulemmyagainst@lemmy.world
    link
    fedilink
    English
    arrow-up
    10
    ·
    2 hours ago

    I use and highly recommend truenas. It uses zfs by default and has a nice web ui, so you don’t have to worry about setting up zfs. It has docker built in and can also do virtual machines and containers.

    • orbituary@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 hours ago

      I’ve used it for about 4 years now. It is a good platform with a few limitations, but largely worth it. Do not use anything but TrueNAS SCALE or you’ll be disappointed by the availability of packages for it.

  • lemmylommy@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    2 hours ago

    It’s not that hard to learn. It’s a few commands to set up a pool, vdev and a dataset. The defaults are fine, though you might want to enable compression, set a large block size and optionally enable encryption via parameters.

    Contrary to popular myths it’s not ram heavy either, at least with default parameters. This might come from the ram heavy (optional) deduplication or the fact that it likes to use unused available ram for caching (as it should).

    With your two disks though you could either mirror them or stripe them. Both are not ideal, it really shines when you have enough disks to use the parity, ie in Z1 you can lose one of your disks without losing data, z2 you can lose two disks. I run it as Z2 with six disks, so any two can fail without losing data and I get the capacity of four disks.

    The main advantage of zfs over other file systems is that it will checksum absolutely everything, including your data, and restore it, if possible (at least a mirror or Zx; a simple stripe can not recover anything of course).

  • 2ndDegreeOWO@lemmy.world
    link
    fedilink
    English
    arrow-up
    1
    ·
    49 minutes ago

    Personally, I use ZFS heavily, and I’d highly recommend it … but for such a simple system with only two drives and you just want redundancy, ZFS is kind of overkill. The only benefit you might get is the ‘copy on write’ feature, which gives you the ability to mount older snapshots of the drive and potentially recover data that has been deleted or overwritten. (Without unmounting the current version of the partition, you can mount an older version that still has the files that are gone. However, the deleted/overwritten files don’t stay around forever, especially if the drive is nearly full or gets written to often. It’s kind of like a fancy ‘recycle bin’, but it’s smarter about what files to keep shadow copies of – for example, files that existed for longer are more likely to be retained than files that only existed for a brief time.)

    Where ZFS really shines is in more complex setups where you want to take advantage of the performance gains. It can do some pretty cool stuff because unlike most other systems, it’s aware of both the layout of the data on the physical drives and the files and filesystem. In most other systems, those two things are handled by different subsystems that aren’t really aware of one another and can’t optimize things as nicely. For example, in a normal RAID system, the data blocks that are mirrored/striped across multiple drives are always the same size. But ZFS can mirror/stripe data blocks of varying sizes, allowing it to better optimize for a mix of small and large files. You can also do cool stuff like designating particular partitions/drives as file directories or caches to further optimize performance. (Great use for that old fast but small SSD you have lying around.)

    I got into ZFS with very little experience, and it was actually pretty straightforward to set up. If you want to use it just to learn it, I’d recommend that. But for the very simple setup you want, there are probably simpler solutions if you want something that just works out of the box without needing to configure things in the command line.

  • Err(()).unwrap()@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 hour ago

    I absolutely recommend ZFS. I’ve been burned by MD (the other software RAID) because it doesn’t scan the array for bad data, so bit rot and silent errors can go undetected until a process wants to read it. ZFS periodically scans the array and fixes corrupted data if possible.

    ZFS can use more CPU resources compared to MD, but the difference in performance is insignificant compared to the read bandwidth of mechanical hard drives. Memory usage is comparatively high because it uses the RAM as cache, but you can set a limit. My current 8TB RAIDZ1 array uses only 8GB of RAM and it could still be lower.

    https://pve.proxmox.com/wiki/ZFS_on_Linux

    it’s not default in mainline linux

    It’s purely an issue of licenses. ZFS is a mature solution, and being out-of-tree does not reflect a qualitative issue. It’s available as a separate package on most distributions.

    • Shimitar@downonthestreet.eu
      link
      fedilink
      English
      arrow-up
      1
      ·
      40 minutes ago

      You cannot compare zfs to md, they are different like apples and oranges.

      MD it’s a raid device, not a filesystem.

      You should compare btrfs to zfs, they pretty much share the same capabilities and features.

      I use MD as my raid array, and btrfs on top of MD.

      Of course if you use ext4 on top of MD you get nothing of the good stuff, except a very solid array management due to md.

  • BCsven@lemmy.ca
    link
    fedilink
    English
    arrow-up
    10
    ·
    3 hours ago

    Self healing for bit-rot in files. So if your data is critical it is worth it.

  • IsoKiero@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    4
    ·
    2 hours ago

    ZFS requires also quite a bit of RAM, so if you’re tight on it then just go with traditional filesystem. But if you’ve got the hardware to support it, it can absolutely be worth it. But, like any RAID or other similar solutions, it’s not a backup and all hardware will fail, so plan accordingly.

    • Shimitar@downonthestreet.eu
      link
      fedilink
      English
      arrow-up
      1
      ·
      37 minutes ago

      Try btrfs if ram footprint of zfs scares you. Pretty solid , all the same features (at least the ones I need) and no licensing issues and already baked in Linux kernel.