Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have two servers with identical hardware, both running CentOS 5.8. They both have 40 TB external RAID arrays attached (via eSATA) to LSI MegaRAID SAS 9280 cards. The array has a GUID partition table and there is one big XFS partition. Disk performance on Machine1 is much better than Machine2, and I am trying to figure out why. It's evident from this hdparm output:

[root@Machine1]# hdparm -t /dev/sdc1
/dev/sdc1:
 Timing buffered disk reads:  1150 MB in  3.00 seconds = 383.27 MB/sec

[root@Machine2]# hdparm -t /dev/sdc1
/dev/sdc1:
 Timing buffered disk reads: 286 MB in  3.03 seconds =  94.55 MB/sec

...and it's also evident when I have big jobs running with intensive disk I/O (processes with heavy IO tend to go into uninterruptible sleep (status "D" on top) on Machine2, but not on Machine1.

I noticed that the block size usage reported by ls -ls (first colum) differs for the same files compared across machines:

[root@Machine1]$ ls -ls
      8 -rw-rw-r-- 1 user1 user1        600 Feb 18 15:41 file1.txt
     48 -rwxrwxr-x 1 user1 user1      43814 Feb 18 15:31 file2.gz
    696 -rwxrwxr-x 1 user1 user1     706287 Feb 18 15:41 file3.txt

[root@Machine2]$ ls -ls
      4 -rw-rw-r-- 1 user1 user1        600 Feb 18 11:57 file1.txt
     44 -rwxrwxr-x 1 user1 user1      43814 Feb 18 10:15 file2.gz
    692 -rwxrwxr-x 1 user1 user1     706287 Feb 18 10:15 file3.txt

The only other difference I could find was using hdparm -g to show geometry. It reveals differing number of cylinders (first number):

[root@Machine1]# hdparm -g /dev/sdc
/dev/sdc:
 geometry     = 40529/255/63, sectors = 85930803200, start = 0

[root@Machine2]# hdparm -g /dev/sdc
/dev/sdc:
 geometry      = 5348945/255/63, sectors = 85930803200, start = 0

But fdisk reports the same number of cylinders for both machines (only showing Machine1 below) -- or maybe fdisk can't be trusted due to the warning messages?

[root@Machine1]# fdisk -lu /dev/sdc

WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.

WARNING: The size of this disk is 44.0 TB (43996571238400 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID 
partition table format (GPT).

Disk /dev/sdc: 43996.5 GB, 43996571238400 bytes
255 heads, 63 sectors/track, 5348945 cylinders, total 85930803200 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1  4294967295  2147483647+  ee  EFI GPT

xfs_info output is identical for both machines (only showing Machine1 below):

[root@Machine1]# xfs_info /dev/sdc1
meta-data=/dev/sdc1              isize=256    agcount=41, agsize=268435455 blks
         =                       sectsz=512   attr=1
data     =                       bsize=4096   blocks=10741350391, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal               bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

I'm currently stumped. Can anyone come up with ideas for next steps?

Thanks!

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.