1

I have a raidz vdev with 3 disks. Disk0 now failed and I try to recover, however I cannot run zpool import. Disk0 is still functional, however the partition table was corrupted and I tried to reconstruct.

$ zpool import
   pool: epool
     id: 15752543265619539307
  state: FAULTED
 status: One or more devices contains corrupted data.
 action: The pool cannot be imported due to damaged devices or data.
    The pool may be active on another system, but can be imported using
    the '-f' flag.
   see: http://zfsonlinux.org/msg/ZFS-8000-5E
 config:

    epool                                               FAULTED  corrupted data
      raidz1-0                                          DEGRADED
        ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1642572-part1  UNAVAIL  corrupted data
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1KPRKPX-part1  ONLINE
        ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N7ZXC1E0-part1  ONLINE

Is there any chance to repair this setup?

$ zdb -l /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1KPRKPX-part1
------------------------------------
LABEL 0
------------------------------------
    version: 5000
    name: 'epool'
    state: 0
    txg: 108340
    pool_guid: 157525$43265619539307
    errata: 0
    hostname: 'eiselekd-lnx'
    top_guid: 5366409938038884073
    guid: 3063814570901399057
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 5366409938038884073
        nparity: 1
        metaslab_array: 35
        metaslab_shift: 36
        ashift: 12
        asize: 9001706717184
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 9231358437407247535
            path: '/dev/disk/by-id/ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1642572-part1'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 3063814570901399057
            path: '/dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1KPRKPX-part1'
            whole_disk: 0
            DTL: 136
            create_txg: 4
        children[2]:/dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N7ZXC1E0-part1
            type: 'disk'
            id: 2
            guid: 10884448206527621454
            path: '/dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N7ZXC1E0-part1'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
    labels = 0 1 2 3 

In the failed partition there are no labels shown:

# zdb -l /dev/disk/by-id/ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1642572-part1
--------------------------------------------
LABEL 0
--------------------------------------------
failed to unpack label 0
--------------------------------------------
LABEL 1
--------------------------------------------
failed to unpack label 1
--------------------------------------------
LABEL 2
--------------------------------------------
failed to unpack label 2
--------------------------------------------
LABEL 3
--------------------------------------------
failed to unpack label 3

however there are Ueberblocks (magic 0cb1ba00 found at the beginning and end of the partition): I used zfs_revert-0.1.py (https://gist.github.com/jshoward/5685757 ) too look at the txg information for disk0 and something in disk0 seems there.

TXG     TIME    TIMESTAMP       BLOCK ADDRESSES
108173  08 Feb 2018 02:24:11    1518053051      [360, 872, 5860494696, 5860495208]
108193  08 Feb 2018 02:25:59    1518053159      [264, 776, 5860494600, 5860495112]
108199  08 Feb 2018 02:26:30    1518053190      [312, 824, 5860494648, 5860495160]
108219  07 Mar 2018 16:44:43    1520437483      [472, 984, 5860494808, 5860495320]
108307  08 Mar 2018 15:58:41    1520521121      [408, 920, 5860494744, 5860495256]
108313  09 Mar 2018 14:25:03    1520601903      [456, 968, 5860494792, 5860495304]
108319  12 Mar 2018 17:33:00    1520872380      [504, 1016, 5860494840, 5860495352]
108325  14 Mar 2018 19:23:31    1521051811      [296, 808, 5860494632, 5860495144]
108331  15 Mar 2018 14:36:25    1521120985      [344, 856, 5860494680, 5860495192]
108337  15 Mar 2018 20:07:58    1521140878      [392, 904, 5860494728, 5860495240]

So I guess that maybe the partitioning shiefted the start of the partition. Not shure. I did a dd if=/dev/sda | strings | grep WCC4N1KPRKPX-part1 however the disk label info is not there...

Is it possible to replace ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1642572-part1 with a new disk and resilver? The http://zfsonlinux.org/msg/ZFS-8000-5E link in the zfs import output doesnt give me hope, but still....

1

There is no way to recover zfs pool without zfs import working. However zfs import is not a error recovery tool and therefore the general strategy is too have a backup instead of relying of zfs being failsave. I extended https://github.com/hiliev/py-zfs-rescue to support linux pools: https://github.com/eiselekd/dumpbin-py-zfs-rescue. With this I was able to recover my disks.

0

Your pool consists of a single raidz1 vdev, which can survive the loss of any one constituent device, and one device in it is "corrupted", so everything needed to import the pool should be right there.

I'm not sure why ZFS wouldn't import your pool right away and just flag the one device as UNAVAIL, but did you try using -f as the output of zpool import suggests? That's the first thing I'd try.

Once the pool is imported, it should be a trivial matter to either zpool offline the problematic partition, or even better just zpool replace it to rewrite the data and regain the redundancy.

As a side note, I recommend to not use partitions with ZFS unless you have a really good reason to. Just give ZFS the whole disk (e.g. /dev/disk/by-id/ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1642572) and let it handle partitioning on its own.

  • zpool import is a requirement when running any recovery. I couldnt get zfs import to accept my disks. The root cause of failure was a faulty SATA controller. When I found it out it was already too late because I hade tried too many of the import options, including the ones that try to fix the metadata. The solution then was to extend github.com/hiliev/py-zfs-rescue to support my linux style zpools: github.com/eiselekd/dumpbin-py-zfs-rescue . With this I was able to recover my data. – Konrad Eisele Jul 28 '18 at 17:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.