Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My email storage requirements are outgrowing anything reasonable with respect to local mail storage. As we speak 99% of my home partition is filled with personal mail in Thunderbirds mail dirs. Needless to say, this is just painful, badly searchable and as history has proven me that backups work, but Thunderbird is capable of loosing a lot of mail very easily.

Currently I have an remote IMAPS server (Dovecot) running for my daily mail, accessible from anywhere, which from my own practice works efficiently up to about 1000 emails. Then some archive directories should be used to move mail around.

I have been looking into DBMail, but I wonder if I make my case worse or better which such solution. None of the supported database employ string deduplication or string compression out of the box, so is this going to help me with 20GB+ mail? What about falling back to a plain old IMAP server? A filesystem like ZFS would support stuff like GZIP transparently, which could help.

Could someone share their thoughts? The 20GB mostly consists of mailinglists, and normal mail. Not things like attachments.


To add some clarifications;

  • As we speak, my mail is not server side indexed at all - only my new mail arrives at a remote IMAP server. It is all local storage from former POP3 accounts, local mirrored Gmail and IMAP accounts. In my perspective it is not Thunderbird that sucks, its fileformat that sucks.

  • Regarding the 1000 mails. On the road I am using Alpine and MobileMail, quite happy with both of them, but some management is required to actually manage the mail. Sieve helps a lot with that, but browing through 10.000 e-mails is not fun, especially not on a mobile client.

  • I am quite happy with Dovecot, never had any issues with it. I just wonder if this is the way to go. Or if there are any other better solutions.

  • What my question is: what is the best practice solution that allows 20GB+ mails and is -on demand remotely accessible, easy to backup and archive worthy. It doesn't need to be available 24x7.

The final approach I took was installing a local IMAP server (Dovecot), configured it for being my archive, using the following guide: http://en.gentoo-wiki.com/wiki/Dovecot/InstallThunderbird

share|improve this question

migrated from serverfault.com May 6 '12 at 2:56

This question came from our site for professional system and network administrators.

    
If 99% of your maildir is mail, expand your homedir and store more other stuff in there. 20GB of mail, with server-side indexing, shouldn't really pose any problems. Are you actually having specific problems (other than "Thunderbird sucks", for which my answer is "so don't use Thunderbird"), or merely posing rhetorical questions? –  womble May 6 '12 at 2:17
1  
Can you clarify what is your question? –  poige May 6 '12 at 3:15
    
I have had a dovecot server holding my email for the last 6 years or so to the tune of about 100,000+ messages from a dozen or two high traffic mailing lists and it seems to work fine. Please clarify your question. –  psusi May 6 '12 at 5:03
    

2 Answers 2

up vote 3 down vote accepted

IMAP, unlike its predecessor POP, is designed so that the "master" home of the email is on the server. Anything that may be stored by the IMAP client (whether it's Thunderbird or a webmail client or anything else) is treated as a cache for what's on the server. As such, it is completely discardable and non-valuable. If Thunderbird doesn't manage it well and corrupts it or loses it, it is of little concern. It will be rebuilt from the information on the server. At worst if it's really corrupt it may need to be deleted. Don't back it up, and don't worry too much about it.

A good well-managed Dovecot server will NOT lose or corrupt your email in any way. (But I can't guarantee that, of course... bugs and disasters are always possible.) I've built several very large email servers using Dovecot.

Your claim that a Dovecot IMAP server "works efficiently up to about 1000 emails" sounds wrong. My personal email is stored on a Docevot IMAP server. I have over 100000 individual emails there, excluding mailing lists. This is using regular Maildir storage on a regular ext3 filesystem. I could upgrade to dbox but there hasn't been a need. It works fine.

20GB is not an enormous amount of storage these days. I don't even think I'd be worrying at all about data deduplication and compression for that amount of data.

share|improve this answer

I would suggest you to make a backup of your email messages in the maildir format, (a single individual file for each message).

You could save them as either EML or PDF. For security you could compress and encrypt them using the ZIP format, one ZIP file for each message because of the following:

Why single files.

  • They are easy to move around, copy and synchronize
  • The chances that the files get corrupted are minimal as oposed to storing them in a big monolithic file (mbox, pst , db, dbx )
  • You can manage them using only windows explorer

You could save your exported files to a portable hard drive or burn them to DVDs to have an offsite copy.

The Dovecot server supports both the mbox and maildir formats and using the rsnapshoot you can backup the /var/mail folder. The problem of this approach is the time that is needed to download as oposed to only downloading new email messages.

MailExporter can export messages from IMAP and from Thunderbird and save them as single files with optional ZIP compression. If needed you can export those saved files back to an IMAP server, Thunderbird or Outlook.

(Disclaimer, I am the author of MailExporter)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.