In ZFS you can use Special Devices to store metadata, deduplication tables, and optionally small files up to a user-defined size.
Special Allocation Class
The allocations in the special class are dedicated to specific block types. By default this includes all metadata, the indirect blocks of user data, and any deduplication tables. The class can also be provisioned to accept small file blocks.
A pool must always have at least one normal (non-dedup/special) vdev before other devices can be assigned to the special class. If the special class becomes full, then allocations intended for it will spill back into the normal class.
So this can be used to accelerate disk accesses using an SSD disk. You can choose to mirror multiple metadata devices for redundancy, or just use the SSD as the only metadata storage. AFAIK when that special device is full then metadata will be stored in the main storage so things will work just as normal
Special VDEV Class
In ZFS 0.8 and later, it is possible to configure a Special VDEV class to preferentially store filesystem metadata, and optionally the Data Deduplication Table (DDT), and small filesystem blocks. This allows, for example, to create a Special VDEV on fast solid-state storage to store the metadata, while the regular file data is stored on spinning disks. This speeds up metadata-intensive operations such as filesystem traversal, scrub, and resilver, without the expense of storing the entire filesystem on solid-state storage.
ZFS Special Device
A special device can improve the speed of a pool consisting of slow spinning hard disks with a lot of metadata changes. For example workloads that involve creating, updating or deleting a large number of files will benefit from the presence of a special device. ZFS datasets can also be configured to store whole small files on the special device which can further improve the performance. Use fast SSDs for the special device.
Example on how to set up this
Adding the special vdev to an existing zpool
#implied stripe of mirrors, for this special device
zpool add elbereth -o ashift=12 special mirror /dev/nvme0n1 /dev/nvme1n1 mirror /dev/nvme2n1 /dev/nvme3n1
Setting the Small Blocks size:
# zfs set special_small_blocks=128K elbereth
Or if you have a “videos” dataset like ours:
# zfs set special_small_blocks=128K elbereth/videos
ZFS Metadata Special Device: Z