Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have RAID 5 setup with a 3ware 9650se controller in my HTPC. I recently lost power to the system, and upon boot up, the RAID 5 reported that it was degraded.

Using the 3ware RAID interface (the one that appears after the POST screen), I added the "missing" disk back to the RAID array, told the array to rebuild, and booted into Linux. I then left the system to rebuild, and headed off to work.

The rebuild process is currently at 47%, and won't be completed for another couple of hours, but from what I've read, the system should be usable, if a little slow, during the rebuild process. However, I now have over 500 media files on my RAID 5 drive that are zero bytes.

Does anyone have any suggestions / experience with zero byte files during a RAID 5 rebuild? Am I going to have to restore from backup, or will the data in my media files magically re-appear after the rebuild has finished?

System Specs:

  • Ubuntu 11.04
  • Gigabyte GA-MA785G-UD3H Motherboard
  • 4GB RAM
  • 3ware 9650se RAID controller
  • 4x2TB SATA II hard drives

/etc/fstab contents:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
UUID=XXXX       /               xfs     defaults        0       1
/dev/sda3       /boot           ext4    defaults        0       2
UUID=XXXX       none            swap    sw              0       0

Output from tw_cli /c0 show

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    REBUILDING     47      -       256K    5587.9    ON     ON     

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     DEGRADED         u0     1.82 TB     3907029168    WD-XXXXXXXXXXXX    
p1     OK               u0     1.82 TB     3907029168    WD-XXXXXXXXXXXX     
p2     OK               u0     1.82 TB     3907029168    WD-XXXXXXXXXXXX     
p3     OK               u0     1.82 TB     3907029168    WD-XXXXXXXXXXXX     
p4     NOT-PRESENT      -      -           -             -
p5     NOT-PRESENT      -      -           -             -
p6     NOT-PRESENT      -      -           -             -
p7     NOT-PRESENT      -      -           -             -

Update

Reading through the XFS FAQ, it looks like I should have disabled the write cache on the hard drives when I installed the RAID array, as "the individual hard disk write caches need to be turned off, as they are not protected from a powerfail and will just lose all contents in that case.".

I wasn't writing to all 500+ files when the power was turned off, so this isn't likely the root cause of the issue I'm seeing, but it looks like it' something I should have done previously.

Update 2

The RAID has finished rebuilding and verifying, and I still have 470 zero byte files. Some of the files re-appeared after the rebuild, but most did not.

In order to help prevent further issues, I've disabled write caching as recommended for the 3ware RAID controller in the XFS FAQ.

As for the actual cause of the issue, I'm not sure. All I know is that a chunk of my data's gone, and I'll need to resurrect what I can from backups.

share|improve this question

2 Answers 2

up vote 1 down vote accepted

You are going to have to restore from backup. What filesystem are you using and with what options? Not having barriers enabled could explain this kind of corruption in the event of a crash. Ext3 defaults to having them off, ext4 defaults to on, but you can switch it with the mount option.

share|improve this answer
    
The partition that's having the issues is XFS with stock options set when Ubuntu was installed. I chose XFS mainly for xfs_growfs functionality. –  Taeram Aug 3 '11 at 18:53
    
@Taeram ext3/4 also can be resized. I'm not sure, but it sounds like XFS defaults to not using barriers unless you specify the barrier mount option. In the future, I suggest either switching to ext4, or enabling the barrier mount option to prevent this from happening again. –  psusi Aug 3 '11 at 19:12
    
According to the XFS FAQ (xfs.org/index.php/XFS_FAQ#Write_barrier_support.) write barrier support should be enabled for all kernels >= 2.6.17. Ubuntu 11.04 comes with 2.6.38, so write barriers should be enabled by default. –  Taeram Aug 3 '11 at 19:17
    
Marking as "accepted" as the final solution to my issue is to restore from backup. –  Taeram Aug 3 '11 at 21:11

Until the rebuild is complete, do not alter (add/remove) the data on the array. Reading it shouldn't hurt anything, but it will of course be slow to respond and slow the rebuild process. It is typically recommend to not use the array at all during the rebuild process.

share|improve this answer
    
When a rebuild is in progress, what would you recommend I do: something like boot into single user mode and wait it out, or wait at one of the post-BIOS screens, like GRUB selection? –  Taeram Aug 3 '11 at 18:55
2  
There is not true at all. The whole point of raid-5 is that the system can remain online and fully functional, including writing to the disk, throughout a disk failure, replacement, and rebuild. –  psusi Aug 3 '11 at 19:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.