6

I am looking at implementing btrfs in raid 10 configuration for a database server and I am an confused about the nodatacow option.

According to https://btrfs.wiki.kernel.org/index.php/Gotchas:

Files with a lot of random writes can become heavily fragmented (10000+ extents) causing trashing on HDDs and excessive multi-second spikes of CPU load on systems with an SSD or large amount a RAM. On servers and workstations this affects databases and virtual machine images. The nodatacow mount option may be of use here, with associated gotchas.

The documentation then states that nodatacow option is:

Do not copy-on-write data for newly created files, existing files are unaffected. This also turns off checksumming! IOW, nodatacow implies nodatasum. datacow is used to ensure the user either has access to the old version of a file, or to the newer version of the file. datacow makes sure we never have partially updated files written to disk. nodatacow gives slight performance boost by directly overwriting data (like ext[234]), at the expense of potentially getting partially updated files on system failures. Performance gain is usually < 5% unless the workload is random writes to large database files, where the difference can become very large. NOTE: switches off compression !

Does this mean that this option should be selected for disks in database servers AND this using this option will disable corruption checksums?

2 Answers 2

3

Does this mean that this option should be selected for disks in database servers?
Probably. The amount of change a database imposes on a filesystem will be amplified by the copy-on-write and checksum process. [1][2] Even normal file system operations can noticeably slow down an active database, which is why many high-performance DBMS support raw disks for storage. [3][4][5]

Does using this option disable corruption checksums?
Unfortunately, yes it does. [6]

[1] https://en.wikipedia.org/wiki/Copy-on-write#Copy-on-write_in_computer_storage
[2] https://btrfs.wiki.kernel.org/index.php/Gotchas#Fragmentation
[3] https://lists.fedoraproject.org/pipermail/devel/2011-July/154251.html
[4] https://blog.pgaddict.com/posts/friends-dont-let-friends-use-btrfs-for-oltp
[5] https://www.google.com/search?q=btrfs+virtual+machine
[6] https://btrfs.wiki.kernel.org/index.php/FAQ#Can_data_checksumming_be_turned_off.3F

3

you should definitely use the nodatacow option on database directories. if you have a database with a lot of writes, it will first slow-down and then destroy your btrfs filesystem in months! i have had this multiple times; btrfs filesystem becoming read-only and failing because the massive amount of fragmentation (and one and another bug that may be fixed now, maybe not).

since using the nodatacow option, the problems went away. there is no point in using COW on a database, as databases are doing their own, more advanced COW logic. yes, you will loose data checksuming, but using COW is still no valid choice for a database.

you do not need to disable cow on the whole filesystem (as per mount option), its sufficient to disable it only on the database directories. To do so, stop your database, create a new directory, disable COW on that using "chattr +C " and copy (not move!) all database files over. check filesystem permissions then move the new db dir in place and start the database. setting chattr +C on a directory disables COW on all NEWLY created child directories and files.

1
  • Note that high fragmentation can be fixed with a balance. If running a DBMS workload on btrfs with datacow, regular balances should be scheduled. It's a cost-benefit calculation whether the additional overhead of running a balance is worth the data checksumming features btrfs provides, in particular the ability to self-heal bitrot with a profile providing redundancy. "Use datacow for everything" and "use nodatacow for databases" are both oversimplifications.
    – cdhowie
    Sep 13, 2020 at 1:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .