I am trying to clean up a mess on an Ubuntu 12.04 LTS server system using ZFS. Here's what zpool status shows:
pool: TB2 state: UNAVAIL status: One or more devices could not be used because the label is missing or invalid. There are insufficient replicas for the pool to continue functioning. action: Destroy and re-create the pool from a backup source. see: http://zfsonlinux.org/msg/ZFS-8000-5E scan: none requested config: NAME STATE READ WRITE CKSUM TB2 UNAVAIL 0 0 0 insufficient replicas sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 sdh ONLINE 0 0 0 sdi ONLINE 0 0 0 sdj ONLINE 0 0 0 sds ONLINE 0 0 0 sdt UNAVAIL 0 0 0 pool: TB4 state: DEGRADED status: One or more devices could not be used because the label is missing or invalid. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the device using 'zpool replace'. see: http://zfsonlinux.org/msg/ZFS-8000-4J scan: resilvered 2.52T in 16h41m with 0 errors on Tue Feb 6 09:27:46 2018 config: NAME STATE READ WRITE CKSUM TB4 DEGRADED 0 0 0 raidz2-0 DEGRADED 0 0 0 ata-Hitachi_HDS724040ALE640_PK1331PAG9MBVS ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK2311PAG8G71M ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK1331PAGH0LHV ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK2331PAG8MV3T ONLINE 0 0 0 spare-4 DEGRADED 0 0 0 ata-Hitachi_HDS724040ALE640_PK2311PAG614MM UNAVAIL 0 0 0 ata-Hitachi_HDS724040ALE640_PK1331PAGH0EAV ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK2331PAGH2XRW ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK1331PAG7TGDS ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK1331PAGG3K0V ONLINE 0 0 0 ata-Hitachi_HDS724040ALE640_PK2311PAG59PYM ONLINE 0 0 0 spares ata-Hitachi_HDS724040ALE640_PK1331PAGH0EAV INUSE currently in use errors: No known data errors
I want to do two things: 1. Replace the faulty drive in pool TB4. This I know how to do. 2. Completely destroy and recreate pool TB2.
Normally, I'd just do a zpool destroy TB2 and start over. However, the previous admin used sd* names for TB2 and disk id's for TB4. In looking at /dev/disk/by-id, I discovered that two of the TB4 drives (...71M and ...EAV) are symlinked to /dev/sdj and /dev/sds respectively. But these sdj and sds are both listed as part of the TB2 pool. I'm afraid that doing a zpool destroy TB2 will corrupt the drives in the TB4 pool, since the docs say that destroy writes to the member disks. Is there any way to get ZFS to simply forget about TB2 without actually writing?
I asked the previous admin why he used two different methods (/dev/sd* and by-id). He said that the assignment of drive letters to specific hard drives didn't seem to be repeatable from boot to boot, so when he created TB4, he used by-id. I guess this entanglement of TB2 and TB4 is a result of that.
My current thought is to do this:
- shut down the machine
- pull all drives.
- zpool destroy -f TB2
- shut down and reinstall TB4 drives
- reformat the TB2 drives on another machine
- Reinstall the TB2 drives and create a new pool using disk id's (not sd*)
Does this seem reasonable? Is there an easier way?
Thanks to anyone who can help me out of this mess.