Convert raw IMAP server data into local folders, then upload partial dataset to new IMAP server?

Question

I am transitioning a company with about 30 IMAP accounts, loaded with data (about 77GB total), to a new email host. The majority of the data will be converted into a local archive and distributed to the company computers as a static reference data set.

The server side folders the users absolutely cannot do without being on the server will be uploaded back to the new server.

I used Mac OS X Mail (Snow Leopard 10.6.6) to download the content. I notice some messages have the name [xxx].partial.emlx, which leads me to believe they have not been downloaded all the way.

I have root access to the mail server data and could download the IMAP server data via FTP. I am not sure what utility to use to convert that data to local Mail.app mailboxes.

Furthermore, I would appreciate any input on the best way to upload a portion of the data to the new server (GoDaddy), preserving the original dates of the messages.

edit

OK - forget the raw server data. I found a script that apparently does pretty good archiving IMAP folders to local mbx files. My main quest now is to batch upload a mailbox hierarchy to the new IMAP server without having to start-stop and deal with similar issues. Anyone know of a utility (hopefully for OS X, but if not, I'll fire up my XP virtual system...) that would be capable of this?

Thanks,

M

Answer 1

I am assuming the IMAP server uses the Maildir formnat for storing it's data (/path/to/mail/user/Maildir/cur/)

You can synchronise a directory structure between two Unix (or OS/X) machines using the rsync command line tool.

Say you have your mail server at mta01 (old) and mta02 (new) and the mail is stored in /var/mail...

Copying the mail from the server to the local computer would be:

$ rsync -avP root@mta01:/var/mail mailBackup

Prune the contents of mailBackup to suit your needs, then it can be copied back with:

$ rsync -avP mailBackup/* root@mta02:/var/mail/

Obviously you'd need to enable root ssh logins to the server for this to work, or use another user that has access to the full email structure.

You may then also need to change the file permissions on the email (depending on how your email is set up) and this might work (or something similar):

(on the server as root)
# cd /var/mail
# for USER in `ls`
> {
> chown -R $USER $USER
> }

Obviously it all depends on what email server software you are running, but most use Maildir for the format.

The Maildir format stores each email as an individual plain text file. A typical file structure is:

/var/mail/matt/Maildir/ <- root Maildir folder
/var/mail/matt/Maildir/new/ <- New messages in INBOX
/var/mail/matt/Maildir/cur/ <- Existing (read) messages in INBOX
/var/mail/matt/Maildir/tmp/ <- Temporary work area
/var/mail/matt/Maildir/.Friends/ <- Root of the 'Friends' folder
/var/mail/matt/Maildir/.Friends/new/ <- New messages in the 'Friends' folder
/var/mail/matt/Maildir/.Friends/cur/ <- Existing (read) messages in the 'Friends' folder
/var/mail/matt/Maildir/.Friends/tmp/ <- Temporary work area

... and so on.

Answer 2

You may find offlineimap to be useful for syncing some mail from one mail server to another. To synchronize imapserver1 to imapserver2, set up a config for synchronizing server1 with a local maildir folder, and set this configuration to exclude any folders that you don't want. Then, set up another configuration to sync the local maildir folder in full to imapserver2.

Alternately, you can do the same thing graphically using Mozilla Thunderbird (or maybe Apple Mail too. I haven't used it). Simply set up the source and destination IMAP accounts in Thunderbird, then drag and drop (and wait).

Of course, all this assumes that you have both the old and new IMAP servers running.