Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My VMware server behaves strangely recently. It turned out to be a file system error. I ran the following command:

voma -m vmfs -f check -d /vmfs/devices/disks/naa.600508e00000000075b722086590fd04

and the output was:

Checking if device is actively used by other hosts
Running VMFS Checker version 1.0 in check mode
Initializing LVM metadata, Basic Checks will be done
Phase 1: Checking VMFS header and resource files
   Detected VMFS file system (labeled:'OS_storage') with UUID:4ea00d03-feebd256-54a5-001e4f3a025a, Version 5:54
 ON-DISK ERROR: lockAddr 6470656 Invalid lock type 2428502018
 ON-DISK ERROR: Invalid lock : 
     [type 90c00002 offset 6470656 v 39, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 40436
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: Invalid lock address 9223372036861427712
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 9223372036861427712 v 12, hb offset 3444736
     gen 89, mode 1, owner 4eb26cf8-bc424318-b504-001e4f3a025a mtime 40506
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: Invalid lock address 9223372036861493248
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 9223372036861493248 v 12, hb offset 3444736
     gen 89, mode 1, owner 4eb26cf8-bc424318-b504-001e4f3a025a mtime 40207
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 8636416 Invalid gbl state
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 8636416 v 23, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 39642
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 8752128 Invalid lock numHolders 2147483648
 ON-DISK ERROR: lockAddr 8752128 Invalid state
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 8752128 v 7, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 39631
     num 2147483648 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: Invalid lock address 2156236800
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 2156236800 v 6, hb offset 3444736
     gen 89, mode 1, owner 4eb26cf8-bc424318-b504-001e4f3a025a mtime 39549
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 8757248 Invalid gbl state
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 8757248 v 7, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 39626
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 9940992 Invalid gbl state
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 9940992 v 7, hb offset 3780608
     gen 9, mode 0, owner 00000000-00000000-0000-000000000000 mtime 2384
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 9965568 Invalid gbl state
 ON-DISK ERROR: Invalid lock : 
     [type 10c00002 offset 9965568 v 11, hb offset 3780608
     gen 17, mode 0, owner 00000000-00000000-0000-000000000000 mtime 1275
     num 0 gblnum 0 gblgen 0 gblbrk 0]
Phase 2: Checking VMFS heartbeat region
Phase 3: Checking all file descriptors.
   Found stale lock [type 10c00001 offset 124340224 v 507, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1521
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124346368 v 56, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1755
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124350464 v 58, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1756
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124360704 v 57, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1761
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124422144 v 548, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1554
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124438528 Invalid gbl state
 ON-DISK ERROR: <FD c277 r49>Invalid lock : 
     [type 10c00001 offset 124438528 v 469, hb offset 3444736
     gen 119, mode 0, owner 00000000-00000000-0000-000000000000 mtime 983
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124440576 v 611, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 627
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: <FD c277 r50>: invalid address Invalid address 80000000
 ON-DISK ERROR: <FD c277 r50>: invalid address Invalid address 80000000
   Found stale lock [type 10c00001 offset 124446720 v 134, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 867
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124448768 Invalid gbl gen 9223372036854775808
   Found stale lock [type 10c00001 offset 124450816 v 136, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 868
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124463104 v 138, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 873
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124479488 v 682, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 521
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124491776 v 530, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 519
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124497920 v 531, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 530
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124504064 v 487, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 569
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124536832 v 532, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 542
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124538880 v 501, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 557
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124542976 v 492, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 572
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124547072 v 534, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 613
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124549120 v 535, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 625
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124551168 v 536, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 636
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124553216 v 537, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 664
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124555264 v 538, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 676
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124561408 Invalid gbl state
 ON-DISK ERROR: <FD c277 r109>Invalid lock : 
     [type 10c00001 offset 124561408 v 400, hb offset 3444736
     gen 119, mode 0, owner 00000000-00000000-0000-000000000000 mtime 2132
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124616704 Invalid gbl state
 ON-DISK ERROR: <FD c277 r136>Invalid lock : 
     [type 10c00001 offset 124616704 v 535, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 40023
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124682240 Invalid gbl state
 ON-DISK ERROR: <FD c277 r168>Invalid lock : 
     [type 10c00001 offset 124682240 v 499, hb offset 3444736
     gen 119, mode 0, owner 00000000-00000000-0000-000000000000 mtime 1872
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124710912 v 529, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 504
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124764160 Invalid gbl state
 ON-DISK ERROR: <FD c278 r8>Invalid lock : 
     [type 10c00001 offset 124764160 v 31, hb offset 3444736
     gen 89, mode 0, owner 00000000-00000000-0000-000000000000 mtime 39727
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: <FD c278 r12> : Invalid TBZ block count 2147493888 should be 10240.
 ON-DISK ERROR: lockAddr 124778496 Invalid gbl state
 ON-DISK ERROR: <FD c278 r15>Invalid lock : 
     [type 10c00001 offset 124778496 v 24, hb offset 3780608
     gen 19, mode 0, owner 00000000-00000000-0000-000000000000 mtime 4606
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 124784640 Invalid gbl state
 ON-DISK ERROR: <FD c278 r18>Invalid lock : 
     [type 10c00001 offset 124784640 v 27, hb offset 3780608
     gen 19, mode 0, owner 00000000-00000000-0000-000000000000 mtime 4647
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124790784 v 27, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1507
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124792832 v 28, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1519
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124794880 v 29, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1530
     num 0 gblnum 0 gblgen 0 gblbrk 0]
   Found stale lock [type 10c00001 offset 124796928 v 30, hb offset 3444736
     gen 119, mode 1, owner 52cff1f4-1e762926-cb53-001e4f3a025a mtime 1569
     num 0 gblnum 0 gblgen 0 gblbrk 0]
 ON-DISK ERROR: lockAddr 130932736 Invalid gbl state
 ON-DISK ERROR: <FD c293 r20>Invalid lock : 
     [type 10c00001 offset 130932736 v 28, hb offset 3780608
     gen 9, mode 0, owner 00000000-00000000-0000-000000000000 mtime 1777
     num 0 gblnum 0 gblgen 0 gblbrk 0]
Phase 4: Checking pathname and connectivity.
 ON-DISK ERROR: Invalid direntry <293, 532> index 21 
 ON-DISK ERROR: Invalid direntry <277, 589> index 16 
Phase 5: Checking resource reference counts.
 ON-DISK ERROR: [FD] inconsistency found: (277, 85) has refCount 32768 in the header, but 0 referenced.
 ON-DISK ERROR: [FD] inconsistency found: (293, 23) has refCount 32768 in the header, but 0 referenced.
 ON-DISK ERROR: [FB] inconsistency found: (111, 41) has refCount 32768 in the header, but 0 referenced.

Total Errors Found:           44

I checked the state of the HDDs and I didn't find any problem with them. This seems to be a file system problem. Is there a way to fix this without formatting the HDDs?

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.