up vote 3 down vote favorite
3
share [g+] share [fb]

Most of my user data stay on an external HDD. Currently, it is FAT32 formatted, because at first thought I might want to connect it to my Windows 7 laptop sometimes.

Now I started sorting my e-books using Calibre. Calibre makes its own copies of the books on import, using author names, series names and titles in the file and directory names, which results in the creation of lots of directories and the copying of lots of small files. Then I correct the details Calibre got wrong, and this results in the directory and file names being changed too. Exporting the books to another format means the creation of lots of new files. After several dozens of books all this started getting really slow. Defragmentation helps somewhat, but not for long.

As I noticed that I usually don't use the HDDs on the laptop (except for the defragmentation itself!) I thought that maybe using another file system will speed things up, and possibly eliminate the need for defrags. So is there a file system which you'd recommend?

I only want to change if there will be a noticeable speed gain. If other filesystems only have the advantage of less corruption chance, I'll stick with FAT32 because of the compatibility.

Edit: Forgot to say, my home PC runs Ubuntu 10.4

link|improve this question
76% accept rate
Thank you for all suggestions. I got ext3, because it seems that there isn't a clear-cut answer and that way I can still read it on Win7 if I really need it. Accept goes to slartibartfast because he reminded me of the noatime option which I would have forgotten ;) – rumtscho Jul 11 '10 at 12:12
feedback

4 Answers

up vote 3 down vote accepted

Default options for XFS might be bad, but I think it can be tuned to your benefit based on what you are describing. It depends on how much man page reading and testing you're interested in doing.

Besides the filesystem, you should consider putting this data on a separate filesystem (with whatever format) and using mount features like 'noatime' and 'nodiratime' to improve performance. From the use you describe, this could be a big win.

I think you'll get a significant benefit from using a non-FAT32 filesystem on Ubuntu.

link|improve this answer
What is XFS a decendant of, and where is it used?* – Kaitlyn Mcmordie Oct 27 '11 at 18:31
1  
Wikipedia has lots of detail, but basically, SGI created it, and it is used in SGI's Irix and a few Linux distros. Apparently no-one else uses it. FreeBSD has read support for it primarily to ease transition from Linux, but that might be it. – Slartibartfast Oct 30 '11 at 7:21
feedback
up vote 4 down vote

ReiserFS seems to be the best choice to manage an huge amount of small files.

link|improve this answer
feedback
up vote 3 down vote

For lots of small files within a directory, ext3 or ext4 with the dir_index capability gives excellent performance.

link|improve this answer
There are only a few files within a directory (under 10) but hundreds of directories, and Calibre is often copying or renaming dozens of files or directories at once. Is the result the same as with "lots of small files within a directory"? – rumtscho Jul 9 '10 at 21:49
It can help, but I don't believe that it would be the best performer. – Ignacio Vazquez-Abrams Jul 9 '10 at 21:58
feedback
up vote 2 down vote

It depends.

In general, ext3/ext4 are the most "stable" file systems. Everything is pretty fast. I'd suggest using ext4 for what you are doing.

However, a lot of other file systems will beat ext3/ext4 by huge margins in some tests, but lose badly in others.

Personally, I'm formatting all my new disks with Btrfs, now that the disk format is supposed to be stable and the performance is already better than ext4 in some cases.

Some other file systems you might consider are reiserfs, and XFS.

Phoronix often does bencharks of file systems, such as this one, in which Btrfs beat ext4 in nearly every test.

link|improve this answer
ext4 stability: last I know, the write then rename semantics weren't in fact fixed. Expect empty files, much fsync, and a bone-headed maintainer. – Tobu Jul 9 '10 at 23:05
XFS is great for massive files, but expect a lot of small files to throw it for a bit of a loop. – Ignacio Vazquez-Abrams Jul 9 '10 at 23:40
@Tobu: I meant stability not in regards to buginess, but that the performance of ex3/4 is rarely a huge surprise, whereas other file systems often have huge issues with certain benchmarks but outperform all the other file systems by miles on another. – Zifre Jul 9 '10 at 23:56
I've always heard of XFS but never knew which operating system it was usually coupled with, and could you explain how XFS is supposed to work? (benefits of using XFS over others as well...) – Kaitlyn Mcmordie Oct 27 '11 at 18:47
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.