3

I have a problem with disk space at btrfs volume. df shows there is plenty of disk space. But when I'm trying to copy 10GB test file system say no disk space on this device.

df -h | grep /mnt/ssd:

/dev/sda 448G 135G 313G 31% /mnt/ssd

Same here:

btrfs filesystem df /mnt/ssd:

Data, RAID1: total=446.12GiB, used=133.29GiB
System, RAID1: total=8.00MiB, used=80.00KiB
Metadata, RAID1: total=1.00GiB, used=609.05MiB
GlobalReserve, single: total=405.53MiB, used=0.00B

I don't know how to read output of this one:

sudo btrfs filesystem show:

Label: none  uuid: aba64e21-69d1-46c1-b3f2-dfda832b67fd
    Total devices 2 FS bytes used 133.88GiB
    devid    1 size 447.13GiB used 447.13GiB path /dev/sda
    devid    2 size 447.13GiB used 447.13GiB path /dev/sdb

So there is 133.88GiB or 447.13GiB used? Very confusing.

5

To understand what's going on here, you need to first understand that BTRFS uses a two-stage allocator. The first stage allocates large chunks of space (actually called 'chunks' in most of the documentation) which get used for exactly one type of allocation, either data (used for the data in files only), metadata (things like file names, directory structure, access times, ownership, permissions, etc), or system (used solely to store data about chunk allocations). Once a chunk has been allocated, the space in that chunk can only be freed by moving all the data out of it.

So, what exactly does this mean in terms of your filesystem?

Well, your output from btrfs filesystem df shows the following:

Data, RAID1: total=446.12GiB, used=133.29GiB
System, RAID1: total=8.00MiB, used=80.00KiB
Metadata, RAID1: total=1.00GiB, used=609.05MiB
GlobalReserve, single: total=405.53MiB, used=0.00B

The total values indicate how much space has been allocated to that type of chunk, while the used value shows how much space is in use within those chunks. In your case, you have 446.32GB of space allocated to data chunks (almost the whole disk based on regular df and btrfs filesystem show output), but only 133.29GB of that space is actually in use. Given this and the symptoms described, BTRFS is trying to allocate a metadata chunk but has no space to do so (because all the free space is inside already allocated chunks), so you're just getting an error instead.

To recover from this, you'll have to run a balance. A balance quite literally sends all the data from selected chunks (or all of them if you pass no options) back through the allocator, which has the net effect of freeing up empty or mostly empty chunks because it packs things back into partially full chunks.

I would start with:

btrfs balance start -dusage=0 /mnt/ssd

That will remove all data chunks that have no actual data in them, which may be enough to get things working again for now, but will still leave you vulnerable to the same problem in the future.

To help compact things completely, repeat the above command with increasing values for the -dusage option. I usually bump it by 5 each time up to about 50 (past 50, you're usually wasting time). The usage filter (specified above for just processing data chunks) will tell balance to select chunks that are only at most that percentage full, so by incrementally stepping it up bit by bit, you can more easily compact things without running into other issues.

You can help head off issues like this in the future by running something like the following regularly (I usually run it daily on my systems):

btrfs balance start -dusage=25 -dlimit=10 -musage=25 -mlimit=10 /mnt/ssd

That will balance the first 10 data and metadata chunks that are less than one quarter full, which should complete in a few seconds in most cases.

  • Than you for this comprehensive answer. Things got better after balancing with -dusage=25. Now I have about 50% space free :). Could you tell me why balancing with -dusage over 50 is wasting time? – Kristopher Feb 1 '18 at 7:58
  • @Kristopher It's essentially an efficiency issue. THe more data is in a chunk, the longer it takes to move the data to a new chunk, and the lower the benefit it will have when resolving this type of issue. By focusing only on chunks that are less than half full, you have a reasonable guarantee that you will free up a lot of space, while balancing chunks that are more than half full may only free up a single chunk (if it frees any at all). – Austin Hemmelgarn Feb 1 '18 at 15:42
1

I've never had a problem of this type with a Btrfs filesystem on a HDD, but I've had the same on my SSD. The SSD doesn't know which blocks are really free, you need to trim it.

But when the problem happens fstrim -v /mnt/ssd shows almost no space being trimmed! My solution:

btrfs balance start /mnt/ssd
  #  you can monitor its progress by 'btrfs balance status /mnt/ssd'
fstrim -v /mnt/ssd

The second command should trim a lot of space this time. After this my free space is really available to me.

Note however: I'm using Btrfs on a single SSD, there's no RAID in my case and I don't know if it makes any difference (I'm counting on your feedback).


About the confusing part: Btrfs as a filesystem can enlarge or shrink itself inside a device (or devices) assigned to it. As for now your filesystem is bloated. It uses the whole 447.13GiB on every device. I think this whole space is "in use" as long as fstrim is concerned. Inside the filesystem 133.29GiB is used by actual data though. Balancing the filesystem should shrink it and only then fstrim is able to do its job.

The filesystem will bloat again with time. That's why I've learnt to perform the above maintenance periodically, especially before apt-get upgrade.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.