3

I have a folder with a few hundred thousand small files, totalling about 14 GB of data. This is a folder in my ecryptfs encrypted home directory.

Doing a "du -sh folder" takes over 9 minutes. Doing a cp -ral to a non-encrypted location takes an hour and 15 minutes. CPU load during this time is mostly IO-bound (80% wa in top)

Doing a "du -sh encryptedfolder" only takes 15 seconds and a cp -ral to the same location takes but 80 seconds. 'encryptedfolder' is the folder in /home/.ecryptfs/myname/.Private that contains the encrypted files.

I am baffled how this performance hit happens. This folder is backuped nightly via rsync, which now takes more than two hours. Before I switched to ecryptfs, I used truecrypt and backup ran in 12 minutes.

Why is ecryptfs so abysmal slow in this scenario? The du -sh and cp -ral operations do not involve any decryption of file contents, just finding the right filename. Is there some way to speed this up?

P.S.: This runs on Ubuntu 11.04

Improve this question

2 Answers 2

Reset to default
3

There are a couple contributing factors here.

  1. Getting a list of all filenames in a directory requires decoding, parsing and decrypting the lower filenames.

  2. The stat() calls from du cause a lookup, which requires allocating an eCryptfs inode, reading part of the lower file metadata, checking to make sure it is an eCryptfs file and then parsing out the unencrypted file size to set the eCryptfs inode's i_size field. Keep in mind that reading the metadata from the lower filesystem involves reading a page into the lower filesystem's page cache.

Because of the design of eCryptfs, it has some unfortunate overhead when dealing with a large number of files. I'm sure there are some improvements/enhancements to be made, despite the design, but optimising this part of the code has not previously been a focus of mine.

Improve this answer
1
  • Okay, this is a bit disappointing. But I'll just move the folder out of my home directory and on to a TrueCrypt secured disk.
    – Guy
    Mar 5, 2012 at 23:11
0

The simple answer is it isn't. The performance hit isn't from encryptfs being slow, it's from the need to allocate very large numbers of inodes and perform disk maintenance to put all of the metadata associated with the files on disk one-by-one.

If the folder is backuped nightly you might find it more useful to first "tar" the entire directory, compress the resulting file and then encrypt that (don't encrypt and then compress, because compression doesn't work on encrypted files). This way you'll have a backup that will be markedly smaller, and much faster to create and move around.

Improve this answer
10
  • (1) For du -sh there is no need to put anything on disk, this is purely reading. (2) Your suggestion with tar is not faster, because it still would have to go through ecryptfs (as that's where the folder is stored in), also I'd prefer to do incremental backups and not get a new huge file every day.
    – Guy
    Mar 5, 2012 at 21:42
  • Reads still need to go through the inode structure (which is lots of indirection each requiring disk access and a reset of the encryption vectors). Your point about tar is valid, but if you want incremental backups you should probably be using SVN or GIT instead of taking a local copy
    – SecurityMatt
    Mar 5, 2012 at 22:36
  • Thanks for the clarification. However I do not accept the premise that svn/git would be a replacement for backups. Ever. In this case I am not even sure it would work at all, or make things significantly faster (it still would need to go through all files to see what has changed).
    – Guy
    Mar 5, 2012 at 23:10
  • To justify my point: blog.codekills.net/2009/12/08/… / ewout.name/2011/10/do-not-store-database-backups-in-git
    – Guy
    Mar 6, 2012 at 8:37
  • This is the first time you've mentioned databases. You're right that you shouldn't back a database up in GIT, but for normal files which change infrequently (such as source-code) it's still a good choice. For database backups you'd probably be better off taking incremental diffs of the database, compressing (and maybe encrypting) the result and storing those off site. Remember that a backup onsite isn't a backup at all when your site burns to the ground.
    – SecurityMatt
    Mar 6, 2012 at 10:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .