63

Do ext4 filesystems need to be defragmented? If so, how do I defragment them?

If not, could you post a simple explanation of why they do not need to be defragmented?

4
  • 4
    If you are as interested as you sound, perhaps this paper would be of interest regarding this: Fedora Project, specifically starting at page 179 (which is really page 3)
    – nerdwaller
    Jan 15, 2013 at 21:39
  • @nerdwaller Interesting, though a little to in-depth for me to fully comprehend.
    – BenjiWiebe
    Jan 15, 2013 at 21:42
  • 2
    Related question on Ask Ubuntu: Why is defragmentation unnecessary?, where the current accepted answer is: The underlying filesystems used by Ubuntu, like ext2 and ext3, simply don't need defragmenting because they don't fragment files in the same way as NTFS. There are more details athttp://en.wikipedia.org/wiki/Ext3#Defragmentation. Jun 14, 2013 at 17:28
  • If anyone is interested in Takashi Sato paper "ext4 online defragmentation" mentioned in the comment from @nerdwaller above, it looks like Fedora Project is no longer hosting that paper -- here's a link to it in the Internet Archive Oct 31, 2021 at 21:46

3 Answers 3

42

Do ext4 filesystems need to be defragmented?

Yes (but very rarely).

If so, how do I defragment them?

Copy all the files off the partition, erase the files from the partition, then copy the files back onto the partition. The file system will intelligently allocate the files as you copy them back onto the disk.

If not, could you post a simple explanation of why they do not need to be defragmented?

ext4 acts in a more intelligent way than merely adding new files into the next available space. Instead of placing multiple files near each other on the hard disk, Linux file systems scatter different files all over the disk, leaving a large amount of free space between them. When a file is edited and needs to grow, there’s usually plenty of free space for the file to grow into. If fragmentation does occur, the file system will attempt to move the files around to reduce fragmentation in normal use, without the need for a defragmentation utility.

Thanks to a Comment by @Green Reaper my attention has been drawn to e4defrag.

9
  • 1
    So there is no on-demand defragmenter software?
    – BenjiWiebe
    Jan 15, 2013 at 21:42
  • 17
    A link to the source you copy/pasted from never hurts... ;) howtogeek.com/115229 Nov 17, 2013 at 2:10
  • 5
    The tool known as e4defrag can be used to defragment ext4. It is provided by e2fsprogs on most distros. ext4 also tries to avoid fragmentation, but it is not always successful. Dec 15, 2016 at 23:01
  • Is there a way to view the percentage of fragmented files? Oct 25, 2017 at 18:26
  • Be aware that e4defrag only defragments files, not the free space. If your free space is fragmented, then e4defrag won't work well, because there's no contiguous space to move any files to.
    – jlh
    Mar 17, 2019 at 9:39
26

I have been using ext4 partition for over a year. I often ran out of space, moved a lots of files in and out etc etc... these things are bad for fragmentation and yesterday I checked for fragmentation for the first time and only 2 files (not %, just two files) were fragmented.

On the Windows partition that was used for about a year longer with much more free space available, I have 95% fragmentation.

So no, you really don't need to defragment ext4 and if you want to be sure, leave the default free space for ext4 (default is 5%, can be changed by ex2tunefs -m X).

2
  • 4
    How do you check for fragmentation?
    – Sparhawk
    Apr 26, 2015 at 10:06
  • 16
    If you have an ext4 file system created with the extent option (it's the default in most recent distros), you can check fragmentation on it with e4defrag -c /path/to/check, and defragment it without umounting with e4defrag /path/to/check. But if you have enough free space you won't need to do so.
    – gerlos
    May 8, 2015 at 8:46
2

Here's an update from 2022

Defragmentation occurs due to multiple reasons:

  • When an OS cannot manage file intelligently
  • There's not enough continuous space to write the file
  • When multiple applications write different files to the the disk at the same time

Reasons to defragment

Why was it needed to defragment files at all? On a HDD this is required to make access to files faster or move them to the area of the disk which provides higher speeds (the outer side of the platter). However there's another important thing many people are unaware of: when files are fragmented it becomes impossible to restore them when you intentionally or accidentally delete them since with so many fragments it's impossible to understand in which order they need to be reassembled. The normal sector size for ext4 is 4K so a 1GB file can theoretically have up to 262144 fragments. And before people say this can never happen I've had servers where our programmers wrote to two MySQL files simultaneously at approximately the same speed, so the resulting files were heavily fragmented and their read speed was well below 1MB/sec vs around 200MB/sec for "normal" files.

As for SSD there are almost no reasons to defragment but there are reasons not to do that since SSDs have a limited number of erase/write cycles. But that still requires an enormous amount of data to be written since modern SSDs feature quite large spare areas to alleviate the issue.

In terms of data recovery there's an issue. If you enabled the discard mount option deleted files are gone for good, so whether or not they were defragmented earlier it doesn't matter, you cannot physically recover them.

If you choose not to use this option and instead rely on a weekly job which does that (fstrim), then defragmenting makes sense.

Finding out the amount of fragmentation

I really doubt the previous comments were honest about the level of fragmentation that ext4 has. It can and it will heavily fragment files under certain circumstance, some applications like systemd-journald do that intentionally.

You can run e2fsck to check how heavily your partition is fragmented. On mounted partitions this will spew a ton of errors, please disregard them:

sudo e2fsck -n -v -f /dev/partition

Actually defragmenting

For the purpose of defragmenting ext4 I've written a script which needs to be run under root/sudo - you can get it here. It will defragment everything except the binaries of running applications - you may want to exit them beforehand. It needs a single argument to run - a path, e.g.

sudo defrag /

The script only defragments files on a given partition. If you have mounted partitions, specify them directly, i.e. sudo defrag /mnt/archive. It will probably won't defragment your swap file either, so running sudo swapoff -a prior to it is a necessity.

This will not defragment directories which can become fragmented as well. For that you'll need to run e2fsck on an unmounted partition:

sudo e2fsck -D -f -v -C 0 -n /dev/partition

Sometimes unmounting partitions becomes very difficult, please use any live Linux distro for that, e.g. System Rescue.

Free space defragmentation issue

The biggest issue with ext4 is that it's only capable of defragmenting individual files, thus you cannot defragment or make continuous free space (for this you'll want to use a different filesystem, e.g. XFS). It's perfectly possible to have a ton of free space yet being unable to defrag relatively small files.

E.g. my root FS for Fedora has 20GB of space, less than 7GB are occupied yet I cannot defrag a few binaries and libraries (chrome, libxul.so) which weigh in at less than 200MB. The reason? The average Linux distro contains tons of files (over 30K for my installation which is not even Gnome/KDE) and when you spread out these many files across the partition, there are not that many "holes" with a lot of space available.

Conclusion/TLDR

  • If you have an HDD you probably want to defragment it
  • If you have an SSD there's generally no need to defragment it unless you want to increase your chances of successful recovery of deleted files
  • If you want to fully defragment your filesystem, you'll need a different filesystem, e.g. XFS.
8
  • I checked the e2fsck command you mentioned, and on my system it does not say how fragmented the partition is. It lists the total number of files and the total number of blocks. It does not say what percentage is contiguous.
    – BenjiWiebe
    Jun 16 at 22:42
  • @BenjiWiebe this doesn't look right, here's my output: pastebin.com/raw/7rwJxySk Either you're not using it right, or it's an old version, or there's a bug. Jun 16 at 22:53
  • Yes I am running a slightly older version. Apparently that was added fairly recently.
    – BenjiWiebe
    Jun 17 at 17:48
  • It showed fragmentation status as early as 10 years ago. Are you using Debian 9? RHEL 7? Jun 17 at 20:18
  • Fedora 33. e2fsck 1.45.6 (20-Mar-2020). EDIT I see what's going on. Your e2fsck command was ran against a partition that wasn't cleanly unmounted. You need to add a -f to force checking of a clean partition.
    – BenjiWiebe
    Jun 17 at 22:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.