PSA: Nix on XFS on ZFS

5 Likes

Fun!

Pretty sure you already know this, but an alternative is to tune a separate ZFS dataset to perform well with databases (at the expense of small files, which is why you make it a separate dataset) - see Tuning recordsize in OpenZFS - Klara Systems for a random example article.

1 Like

Yep, I do, I do that for PostgreSQL, where I have found 8k to work well. But I haven’t really found anyone talking about what is the correct tuning for SQLite and honestly, I couldn’t be arsed to find out :D. So I just slapped it on XFS.

1 Like

How did the thought of moving it to XFS occur to you in the first place? What’s the reasoning here?

I use ZFS and face similar issues, but I would never think about this.

Sorry if the question is stupid but I’m no filesystems expert.

FYI the ZFS sqlite performance issues should be resolved with 2.4.0

1 Like

I’ve been running ZFS on everything for nearly 5 years and i maintain a sizable homelab (more like homeprod these days). I’ve had issues with zfs performance in the past, especially when it comes to databases. A database like sqlite and postgres are actually somewhat similar to zfs. The internal data layout is different, but they both keep a write ahead log, journal, zfs intent log, whatever you want to call it. It’s purpose is to allow the program to quickly and reliably make sure that all changes make it to disk when desired. Then, the log is applied to the actual on-disk datastore.

But yeah, honestly, mostly a hunch. If i had more time on my hands, way more time, id also attempt to write a patch for Nix which allows it to store store paths in a prefix form. So instead of /nix/store/aaaaaaaa youd have /nix/store/aa/aaaaaa which should help with the ZAP (data structures zfs uses for directory lookups). Youd also need a kernel module which undoes this prefixing so it still looks like a normal store to the rest of the system.

1 Like

I’ll check, i might be on 2.4 already. Will update this comment when I do.

@norpol yes i am:

zfs-2.4.3-1
zfs-kmod-2.4.1-1
1 Like