Using ZFS on Ubuntu: A Practical Guide

Now, for the practical part. In my previous article, I introduced how to select ZFS during installation using the installer.

This time, I will show you how to add ZFS to an existing Ubuntu system running on ext4.

In this configuration, the system resides on ext4 while the data storage uses ZFS, creating a hybrid structure that offers significant benefits.

When using ZFS on Linux, as mentioned in the previous article, kernel support is not an issue, but there is one thing that could potentially be a problem. That is, GRUB's ZFS support is weak. This means that if you place the kernel on ZFS, there is a possibility of encountering issues during boot.

Ubuntu makes it possible to install with a ZFS root, but to avoid the (potential) GRUB issue, it uses a mechanism called initramfs.

This involves launching initramfs, which is a mini Linux system created on a RAM disk, and then loading the full kernel located on ZFS. While somewhat complex, it is a boot method that has been used in Linux for a long time and is proven and highly reliable.

ext4 root + zfs data configuration results in a standard and simple boot process, making it a configuration you can use with even more peace of mind.

Now, I will show you how I actually added ZFS to my Ubuntu machine.

My Ubuntu machine has two NVMe drives, with the second one used as a data disk. Given the recent surge in NVMe prices, this might be a rather luxurious configuration.

I migrated this second NVMe drive from ext4 to ZFS.

Installation

To use ZFS on Ubuntu, you first need to install zfsutils-linux.

With just this, you will be able to use management commands like zfs and zpool. The kernel modules should load automatically, but rebooting is the safest way to ensure everything is loaded.

Checking Device Names

First, confirm the device name (device file name) of the storage you intend to use.

In my case, it was an NVMe drive, so the device file was /dev/nvme1n1. For SATA HDDs/SSDs, it would likely be /dev/sda or /dev/sdb.

Creating a Pool

Create a pool on the storage you are using. A pool is a mechanism that combines multiple storage devices into a single virtualized storage unit. You create a pool even if you are using a single storage device.

While general logical volumes are realized by virtualizing file systems like ext4 using various tools, in ZFS, all of this is integrated into the zpool command.
Various RAID configurations can also be achieved with a single zpool.

If you have only one storage device, do the following.

The pool name can be anything. I usually name it 'zdata'. Here is a concrete example.

If you want to create a mirrored configuration (so-called RAID1) with two storage devices, do the following:

like this.

With just this, a pool named 'zdata' is created, automatically mounted to a directory with the same name as the pool, '/zdata', and is ready for immediate use.

Generally, when setting up RAID in Linux, steps like 1) mdadm --create, 2) mkfs.ext4, 3) editing /etc/mdadm.conf, and 4) editing /etc/fstab are required, but in ZFS, it can be done in one go with 'zpool create'. Moreover, it finishes instantly. (With mdadm + mkfs, it would take 5 to 10 minutes.)

Creating a dataset

Although you can use the pool as is, it is generally used by dividing it into units called datasets. This is because snapshots, quotas, and compression are handled at the dataset level, and it is easier to use when partitioned. It is similar to dividing a disk into partitions, but it is extremely flexible because the capacity is variable and does not need to be determined in advance.

Creating a dataset is also simple.

As an execution example,

is how it looks. In this case, it is automatically mounted to /zdata/data.

If you want to mount it to a different location,

it is also possible to do it like this. It might be a bit confusing, but zdata/data will be mounted to /data. You can verify this by using the mount or df commands, and it should make sense.

What makes it even more complicated is that datasets can be nested indefinitely like

and any level can be mounted anywhere. For example, you can do something like

.

Since you can perform file system operations such as snapshots, compression, encryption, NFS exports, backups, and quotas (capacity limits) on a per-dataset basis, you should split your datasets according to your needs for these features.

It is so flexible that it can feel like a labyrinth, but it is generally best to think about it in terms of how you want to take snapshots.

A common practice is to separate datasets for each virtual machine folder and take snapshots of them, right?

Migrating the Home Directory

Another useful tip is to keep your home directory as a dataset. In this method, where the system and data are separated, your home directory should be on the ext4 system partition. Let's move this to a ZFS dataset.

First, create the dataset. Let's name it home.new to avoid double-mounting it at /home. In this example, we will migrate /home/user from ext4 to ZFS.

Now that the created dataset should be mounted at /zdata/home.new/user, change the owner from root to user.

To copy the entire home directory, use the rsync command.

Next, rename the original home directory.

Rename the ZFS side.

Let's set the mount point to /home.

The mount point for zdata/home/user will not automatically become /home/user, so don't forget to set the mount point.

With this, the migration of the home directory is complete. Once you have used it for a while and are confident it is working correctly, it is okay to delete everything under /home.old.

If you have virtual machines such as VirtualBox or KVM, it becomes much easier to manage them by creating a dataset for each virtual machine in the same way as /home.
There is also a dedicated mechanism called ZVOL, which I would like to introduce if the opportunity arises.

How to take a snapshot

In ZFS, you can take snapshots very easily.

Here is a concrete example.

You can check it using zfs list.

If you do the following, you will be able to take snapshots even with user privileges. (It becomes executable without sudo)

Actually, you can also allow rollback and destroy, but to prevent accidents such as accidental deletion, it is better to only allow snapshots.

Comparing it with btrfs snapshots, you can see how easy ZFS snapshots are.

In btrfs,

As shown above, you need to specify the snapshot by its path name. In other words, you must specify both the storage location and the name. The example is the standard way to do it, but in this case, you need to create the .snapshots directory in advance. Compared to ZFS snapshots, where you only need to specify a name, having to manage where the snapshot is placed feels quite cumbersome.

In ZFS, when you take a snapshot, a directory like structure called .zfs/ is created directly under the dataset. It appears to contain the files and directories as they were at the time the snapshot was taken, and by doing something like

, the file is restored as an actual file, allowing you to easily recover files without performing a rollback when you accidentally delete them. It is extremely convenient.

.zfs is not a directory but a mysterious read-only entity; it does not appear when you run ls -a, but you can see its contents by running ls .zfs.

Reference Links

OpenZFS

Ubuntu Tutorial

Arch Linux ZFS

Summary

In this way, my home supercomputer, the zen4 Ryzen9 Ubuntu machine, has become a hybrid configuration of ext4 and ZFS. I will use it like this for a while, and if any problems arise, I would like to write an article about it on note. (It might turn out that "it's better to avoid ZFS on Linux" after all.)