path: root/content/blog/2024-02-06-zfs.org

#+date: <2024-02-06 Tue 00:00:00>
#+title: Step-by-Step Guide to Creating a ZFS Storage Pool on Ubuntu Linux
#+description: Learn how to create and configure high-performance ZFS storage pools on Ubuntu Linux, including setup, expansion, and best practices for reliable data management.
#+slug: zfs

This post details the process I used to create ZFS pools, datasets, and
snapshots on Ubuntu Server.

I found the following pages very helpful while going through this process:

- [[https://ubuntu.com/tutorials/setup-zfs-storage-pool][Setup a ZFS storage pool]]
- [[https://wiki.ubuntu.com/Kernel/Reference/ZFS][Kernel/Reference/ZFS]]
- [[https://blog.victormendonca.com/2020/11/03/zfs-for-dummies/][ZFS for Dummies]]

* Installation

To start, I installed the ZFS package with the following command:

#+begin_src sh
sudo apt install zfsutils-linux
#+end_src

Once installed, you can check the version to see if it installed correctly.

#+begin_src sh
> zsf --version

zfs-2.1.5-1ubuntu6~22.04.2
zfs-kmod-2.1.5-1ubuntu6~22.04.1
#+end_src

* ZFS Configuration

Now that ZFS is installed, we can create and configure the pool.

You have various options for configuring ZFS pools that all come different pros
and cons. I suggest visiting the links at the top of this post or searching
online for the best configuration for your use-case.

- Striped VDEVs (Raid0)
- Mirrored VDEVs (Raid1)
- Striped Mirrored VDEVs (Raid10)
- RAIDz (Raid5)
- RAIDz2 (Raidd6)
- RAIDz3
- Nested RAIDz (Raid50, Raid60)

I will be using Raid10 in this guide. However, the majority of the steps are the
same regardless of your chosen pool configuration.

** Creating the Pool

To start, let's list the disks available to use. You can use =fdisk= command to
see all available disks.

#+begin_src sh
sudo fdisk -l
#+end_src

Or, if you currently have them mounted, you can use the =df= command to view
your disks.

#+begin_src sh
> sudo df -h

Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sda1       7.3T   28K  6.9T   1% /mnt/red-01
/dev/sdb1       7.3T  144G  6.8T   3% /mnt/red-02
/dev/sdc1       7.3T  5.5T  1.9T  75% /mnt/white-02
/dev/sdd1       9.1T  8.7T  435G  96% /mnt/white-01
/dev/sde1       7.3T   28K  6.9T   1% /mnt/red-03
/dev/sdf1       7.3T   28K  6.9T   1% /mnt/red-04
#+end_src

If you're going to use mounted disks, make sure to umount them before creating
the pool.

#+begin_src sh
sudo umount /dev/sda1
sudo umount /dev/sdb1
#+end_src

Now that I've identified the disks I want to use and have them unmounted, let's
create the pool. For this example, I will call it =tank=.

#+begin_src sh
sudo zpool create -f -m /mnt/pool tank mirror /dev/sda /dev/sdb
#+end_src

See below for the results of the new ZFS pool named =tank=, with a vdev
automatically named =mirror-0=.

#+begin_src sh
> zfs list

NAME   USED  AVAIL     REFER  MOUNTPOINT
tank   396K  7.14T       96K  /tank
#+end_src

#+begin_src sh
> zpool status

  pool: tank
 state: ONLINE
config:

    NAME        STATE     READ WRITE CKSUM
    tank        ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        sda     ONLINE       0     0     0
        sdb     ONLINE       0     0     0

errors: No known data errors
#+end_src

We can also look at the mounted filesystem to see where the pool is mounted and
some quick stats.

#+begin_src sh
> df -h

Filesystem      Size  Used Avail Use% Mounted on
...
tank            7.2T  128K  7.2T   1% /tank
#+end_src

** Expanding the Pool

If you want to expand this pool, you will need to add a new VDEV to the pool.
Since I am using 2 disks per VDEV, I will need to add a new 2-disk VDEV to the
existing =tank= pool.

#+begin_src sh
sudo zpool add tank mirror /dev/sdX /dev/sdY
#+end_src

If you're adding disks of different sizes, you'll need to use the =-f= flag.
Keep in mind that the max size will be limited to the smallest disk added.

#+begin_src sh
sudo zpool add -f tank mirror /dev/sdX /dev/sdY
#+end_src

I added two 8TB hard drives and this process took around 10 seconds to complete.

When viewing the pool again, you can see that the pool has now doubled in size.
We have 14.3 TB useable space and the same space used for mirroring.

#+begin_src sh
> zfs list

NAME         USED  AVAIL     REFER  MOUNTPOINT
tank         145G  14.3T      104K  /tank
tank/cloud   145G  14.3T      145G  /tank/cloud
tank/media    96K  14.3T       96K  /tank/media
#+end_src

*** Converting Disks

Some disks, such as NTFS-formatted drives, will need to be partitioned and
formatted prior to being added to the pool.

Start by identifying the disks you want to format and add to the pool.

#+begin_src sh
sudo fdisk -l | grep /dev
#+end_src

I am going to format my =/dev/sdc= and =/dev/sdd= disks with the =fdisk=
command.

See below for instructions on how to use =fdisk=. Here's what I did to create
basic Linux formatted disks:

- =g= : Create GPT partition table
- =n= : Create a new partition, hit Enter for all default options
- =t= : Change partition type to =20= for =Linux filesystem=
- =w= : Write the changes to disk and exit

I repeated this process for both disks.

#+begin_src sh
> sudo fdisk /dev/sdc

Welcome to fdisk (util-linux 2.37.2). Changes will remain in memory only, until
you decide to write them. Be careful before using the write command.

This disk is currently in use - repartitioning is probably a bad idea. It's
recommended to umount all file systems, and swapoff all swap partitions on this
disk.


Command (m for help): m

Help:

  GPT
   M   enter protective/hybrid MBR

  Generic
   d   delete a partition
   F   list free unpartitioned space
   l   list known partition types
   n   add a new partition
   p   print the partition table
   t   change a partition type
   v   verify the partition table
   i   print information about a partition

  Misc
   m   print this menu
   x   extra functionality (experts only)

  Script
   I   load disk layout from sfdisk script file
   O   dump disk layout to sfdisk script file

  Save & Exit
   w   write table to disk and exit
   q   quit without saving changes

  Create a new label
   g   create a new empty GPT partition table
   G   create a new empty SGI (IRIX) partition table
   o   create a new empty DOS partition table
   s   create a new empty Sun partition table
#+end_src

Once the drives are formatted, we can add these disks to the pool.

#+begin_src sh
sudo zpool add tank mirror /dev/sdc /dev/sdd
#+end_src

When we list the pool again, we can see that our size is now updated to
approximately 22TB. This represents my hard drives totalling 45.6TB when shown
with =fdisk -l=, with a Raid10 configuration using 22TB for mirroring and 22TB
of useable space.

#+begin_src sh
> zfs list

NAME         USED  AVAIL     REFER  MOUNTPOINT
tank         145G  21.7T      104K  /tank
tank/cloud   145G  21.7T      145G  /tank/cloud
tank/media   145GT 21.7T       96K  /tank/media
#+end_src

** Creating Datasets

According to [[https://docs.oracle.com/cd/E18752_01/html/819-5461/ftyue.html][ZFS Terminology]], a =dataset= can refer to “clones, file systems,
snapshots, and volumes.

For this guide, I will use the =dataset= term to refer to file systems created
under a pool.

Within my =tank= pool, I am going to create some datasets to help organize my
files. This will give me location to store data rather than simply dumping
everything at the =/tank/= location.

#+begin_src sh
sudo zfs create tank/cloud
sudo zfs create tank/media
#+end_src

Once created, you can see these datasets in the output of your pool
list:

#+begin_src sh
> zfs list
NAME         USED  AVAIL     REFER  MOUNTPOINT
tank         752K  7.14T      104K  /tank
tank/cloud    96K  7.14T       96K  /tank/cloud
tank/media    96K  7.14T       96K  /tank/media
#+end_src

** Creating Snapshots

Next, let's create our first snapshot. We can do this by calling the =snapshot=
command and give it an output name. I will be throwing the current date and time
into my example.

#+begin_src sh
sudo zfs snapshot tank@$(date '+%Y-%m-%d_%H-%M')
#+end_src

We can list the snapshots in our pool with the following command:

#+begin_src sh
> zfs list -t snapshot
NAME                    USED  AVAIL     REFER  MOUNTPOINT
tank@2024-02-06_19-41     0B      -      104K  -
#+end_src

** Destroy Snapshots

You can always destroy snapshots that are no longer needed:

#+begin_src sh
sudo zfs destroy tank@2024-02-06_19-41
#+end_src

Once deleted, they will no longer appear in the list:

#+begin_src sh
> zfs list -t snapshot
no datasets available
#+end_src

* My Thoughts on ZFS So Far

- I sacrificed 25TB to be able to mirror my data, but I feel more comfortable
  with the potential to save my data by quickly replacing a disk if I need to.
- The set-up was surprisingly easy and fast.
- Disk I/O is fast as well. I was worried that the data transfer speeds would be
  slower due to the RAID configuration.
- Media streaming and transcoding has seen no noticeable drop in performance.
- My only limitation really is the number of HDD bays in my server HDD cage.