Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Is using mdadm's RAID-1 of 2 partitions (one on laptop's internal HDD, one on external HDD) a good idea.

I want the system to work as RAID-1 if both drives are present, work as regular volume (degradad RAID-1) if external HDD is unplugged and quickly resync when I plug external HDD again.

Questions:

  1. Is it a good idea?
  2. Will write-intent bitmap be enough for this task or I need something else?
  3. Should I consider doing it at filesystem level (3b. if yes, how?).

Basic requirements are:

  1. Quick resync when I re-add the external drive (provided I hasn't changed that partition).
  2. More or less consistent data on the removed drive if I remove it not during write/resync operation.

If I remove the drive during resync I expect the data to be somewhat inconsistent, but expect quick resync completion when I re-add it again. E.g. I want the the remaining drive to track what is changed (there can be a lot of changes) and that sync back only those parts that need it.

share|improve this question
2  
remember that RAID ≠ backup. –  squircle Mar 30 '10 at 3:19
    
@thepurplepixel It is something in middle between RAID and backup. –  Vi. Mar 30 '10 at 3:37
    
Just finished waiting for "Rebuild Status : X% complete", unplugged the drive, plugged it again, issues "mdadm /dev/md0 --add /dev/sdb5" and it's again 0% and fully rebuilding. What is "--bitmap internal" then? Why is it rebuilding from scratch? –  Vi. Mar 30 '10 at 3:46
    
@Vi Tried again, looks like smart rebuild using bitmap works... –  Vi. Mar 30 '10 at 4:07
1  
@Vi How is RAID 1 "somewhere in the middle"? –  Dave M Mar 30 '10 at 12:16

3 Answers 3

I would say that this isn't a good idea. All you are protecting against at most is physical/electrical drive problems, and you only ever have one copy of the backup which is very vulnerable while being refreshed.

If a fault occurs on the other drive, or with your machine in general, during the resync you could be left with the main drive being out of action and the "bakcup" drive having a partially updated (i.e. potentially very corrupt) filesystem. This is on of the reasons why most backup solutions include multiple backups (another key reason being that you can have at least one backup offline (and preferably offsite) while updating the another), but is made worse by the fact that the RAID resync is not going to be filesystem aware so will not be able to do anything to ensure a partial resync leaves the filesystem in anything close to a consistent state (for instance it might sync written data before related journal blocks and so on).

If you are RAIDing your whole system in this way then the write-intent write performance hit could be quite significant too (see posts like http://blog.ganneff.de/blog/2008/01/30/write-intent-bitmaps-considere.html), though if you are only RAIDing filesystems that don't see much write activity this may not be a concern for you.

If you do use this technique, make sure you run sync before forcibly ejecting the drive.

Have you considered using rsync to update the backup filesystem instead? This would be slower in your solutions best-case (as a full scan of file sizes and dates is needed even if nothing has changed, which could be significant if the filesystem contains many files) but is usually fast enough and will certainly be faster than a full resync where the RAID driver to decide this is needed.

share|improve this answer
    
"If you are RAIDIN your whole system in this way" -> No, it's just one volume, mostly with bare git repositories and data I don't want to lose. It is not a root partition. "i.e. potentially very corrupt" -> I've got used that "--rebuild-tree" saves from even moderate corruption (for example, resuming from hibernation after external mount with replaying journal). Should I expect to lost mostly the new data (that were syncing at that moment). "you can have at least one backup offline" -> May be it will be usual rare DVD archives, like I used to create before that. –  Vi. Mar 31 '10 at 19:09
up vote 0 down vote accepted

Looks like external_HDD-based RAID-1 array idea works.

The array starts degraded and tracks changes to it using write-intent bitmap.

When I connect my external HDD, my udev script detects that one of partition suits my array and adds it. It synchronizes (instantly if there were no changes) and then acts as speed-up for read access.

When I disconnect my external HDD (including sudden removal), another udev scripts registers that the drive is removed and the array becomes degraded again.

There is noticeable write performance drop however, but it is acceptable for this usage.

Other advantage is that I can temporarily format that partition for some other use and then bring it back to the array (with a full rebuild).

share|improve this answer

One problem I see in this, that the filesystem on the removed external HDD might be in a state that cannot be mounted, though modern filesystems with journaling etc do mitigate this risk.

But if you are using mdadm, I can assume you are using some flavour of Linux and would recommend using rsync instead. This would be fast as only changed files would be copied over and it also would allow you to have a script that creates a folder with current date on the external HDD and then create a full backup of your internal harddrive under it but hardlinking the files against the previous backup, thus using space only for the new data. That way you can easily recover a file that might have been deleted on the internal HDD.

share|improve this answer
    
Can I run rsync in some continuous mode to make it track changes and update the backup copy? (I was also thinking about some FUSE filesystem that will remember the changes and propogate them to the copy.) Will your rsync variant handle recovery (without complete erase and rebuild) after removing the drive in the middle of synchronisation? And I think reading from the second drive and speed up some things (working copy on internal drive, git repository is read from external drive) in some cases. –  Vi. Apr 1 '10 at 17:36
    
There is a app that keeps track of any modifications on the source drive and then immediately rsyncs them to the target drive. You can remove the drive during rsync running, this will just cause the currently copied file to be partially copied, running rsync again will just continue where it left off. Rsync wont give you any speed up with reading, but unless you are using eSata for the external drive, the read speedup will be quite marginal and write speed would be limited by the slower of the drives. –  Raynet Apr 2 '10 at 14:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.