9

Using Linux, I have several backup levels. One of them is a periodical sector by sector copy (using dd) of my laptop harddisk to an external USB disk. Yes, I have other backups too, like remote rsync.

This approach (the disk dd) is OK when cloning a HDD with no LVM volumes, since I can plug the external disk anytime and mount the partitions simply mounting /dev/sdb* instead of /dev/sda*. Trivial and handy.

Today I moved ALL my harddisk (including the /boot) to LVM. Everything works fine. I will stress it for a couple of days, and then I will do a sector by sector copy to my external harddisk.

Now I have a problem, I guess.

If in the future I plug the external USB HDD to recover any file, the OS will detect a duplicate LVM configuration, with the same name and the same UUID. Even doing a vgrename (which LVM would be renamed, the internal HDD or the external HDD?), the cloned UUID will not change. Is there any command to change name and UUID? Ideally I would clone the HDD and then change the LVM group name and its UUID, but I don't know how to do it.

Another related issue would be...

In the past I have booted my laptop using the external disk, using the BIOS boot menu and changing GRUB entries manually to boot from /dev/sdb instead of /dev/sda. But now my current GRUB configuration boots directly from a LVM logical volume, something like: set root='(LVM-root)' in my grub.cfg. So... What is going to happen with duplicated volumes?

Any suggestion?

I guess I could repartition my external harddisk and change backup strategy from dd to rsync, but this disk has windows installed too, and I really would like to have a physical "real" copy.

1
  • 1
    Maybe you could try some LVM mirroring tricks. Mar 11, 2011 at 9:40

4 Answers 4

4

I agree with one of the previous answers -- running dd on the whole disk is not a great solution here. If the reason you're stuck on dd has to do with backing up another OS, then you can use dd only for those other OS partitions, and rsync for the linux partitions.

Part of the problem is that if you run dd of the whole disk while you have filesystems up and mounted from it, there is no way to ensure the backup is consistent. It may work 9 times out of 10, but under any load, the resultant backup could be corrupted. The only way to ensure this doesn't happen is to unmount all filesystems and deactivate all volume groups for the entire time that the dd is running. Not very convenient if your / is mounted from LVM.

In any event, if you insist on sticking with this approach, the tool you're looking for to rename a duplicated vg is called "vgimportclone". It's intended for use with device level snapshots, but it will work with dd as well.

2
  • 1
    Of course, the "dd" is done booting from a livecd. The disk is not mounted.
    – jcea
    Dec 29, 2011 at 4:52
  • Using "dd" has two problems, beside the LVM "cloning" effect: 1. Downtime (in my laptop, about FOUR hours. I do the backup when I don't need the computer, like when dining out) and 2. If the source disk has a read error the copy will fail and the destination (backup) disk will be inconsistent and unusable to recover the read-error in the master disk. This problem is MAJOR, but can easily solved using TWO backup disks, alternating backups on them. This way, if current backup fails and you can't not use it, you have the consistent previous backup.
    – jcea
    Dec 29, 2011 at 4:56
4

Closing this issue, after a year. I have written the answer in my personal website. I hope it is useful to somebody else.

I use several levels of backup, from rsync to an external disk to rsync+ZFS over a remote computer. One of the backups I do is a -from time to time- "dd" from my laptop harddisk to an external USB attached (connected only for the duration of the backup procedure). I do this, for instance, before taking my laptop for a trip.

This procedure worked well for ages. Until I started using LVM on my laptop.

If you use LVM on your computer, you copy your harddisk using dd (booting from a LiveCD) and you plug the external harddisk while the computer is running normally, later, YOU WILL CORRUPT YOUR DATA, EVEN LOSE ALL YOUR FILESYSTEM BEYOND ANY REPAIR!!!.

Why?. Because when you plug the external harddisk the OS will see the same LVM configuration in both disks and will naively assume that both disks are the same, accessible via a multipath link, so it will spread reads and writes between both disks, irredeemably corrupting BOTH. You will trash both your live data and your backup!.

This happened to me once. I lost data. I had other backups, for a few days before, so I didn't lose anything very important, but it was quite annoying and exposed a flaw in my backup strategy.

Solution: Make sure the LVM configuration in the backup disk is different. The problem is... How to do it?

I posted the question on superuser.com (Stack Exchange) but I didn't get any good answer. So I developed my own procedure and, after more than a year of battletesting it, I am posting it here and closing the original question:

  • Be sure that if something goes bad, your computer will not boot automatically. In my case, my laptop doesn't boot automatically if the power goes off and on again. Moreover, the harddisk is encrypted, so it will ask for a password.

    This is necessary because you can corrupt BOTH disks if you reboot with the backup disk attached before completing the procedure.

  • Boot from a Live CD. You can not do a "dd" from a live partition, if you expect to be able to recover your data :-).

    Something to try in the future would be to use LVM snapshots to be able to do the backup while I am using the computer, but then I have the duplicate LVM issue, that I am trying to avoid here. Another option would be to reconfigure LVM to mirror the data live to the external harddisk, and break the mirror after the sync is complete. But that sounds risky, and wouldn't backup my Windows partition or the data I don't keep in LVM volumes.

  • After booting the LiveCD, plug the external USB harddisk.

  • Login as "root" and execute vgchange -a n. This command will disable LVM in both disks. I execute the command a couple of times, to be sure.

  • Be sure /dev/sda is the source (internal harddisk) and /dev/sdb is the destination (USB external harddisk). For instance, do dd if=/dev/sdb of=/dev/null and check which harddisk LED blinks.

  • When you are sure about which disk is which, you do the copy with dd if=/dev/sda of=/dev/sdb bs=65536. With my configuration, the backup takes four hours. My internal harddisk is 500GB, and my USB is copying at 35MB/s. I do this at night, while I sleep.

    Your external USB harddisk must be, AT LEAST, as big as your internal harddisk. It is obvious, isn't it?

  • After this copy is done, you have an exact clone of your internal harddisk. The backup is done. But now you will have an issue if you ever plug this harddisk to your computer while you are working with your internal harddisk, as already described. We need to change the LVM configuration.

  • Now, after the "dd" is done, do "sync" and unplug the external UDB harddisk.

  • You execute pvchange --uuid /dev/sda*. This command will change the UUID of all the physical volumes in your internal harddisk. This command is safe even if you have partitions that are not physical volumes for LVM, because they will be automatically and safely skipped.

    The system knows which partitions are physical volumes because the partition type and because you executed "pvcreate" when you created the LVM.

  • You execute vgchange -u LVM. This command will change the UUID of the LVM volume group. My volume group is called "LVM", by the way.

  • You execute vgscan. This command will scan internal harddisk (the only currently attached) and will locate your LVM volume group there.

  • You execute vgrename LVM LVM2, to change the name of your Volume Group.

  • Now you plug your external USB harddisk.

  • You "vgscan" again, this time to locate the Volume Group in the external USB harddisk. Now you will have two Volume Groups. One called "LVM2" in your internal harddisk and other called "LVM" in your USB external harddisk.

  • Rename the Volume Group in the external USB harddisk with vgrename LVM LVM_BACKUP.

  • And rename the Volume Group in the internal harddisk back to the original name: vgrename LVM2 LVM.

  • You are done. You can review the situation with vgdisplay -v.

    You will see that Logical Volumes UUIDs are the same in both Volume Groups, but that doesn't seem to cause any problem, and I don't know how to change them.

  • Unplug your external USB harddisk, store it in a safe place, reboot your computer, eject the LiveCD and go back to business.

1

Studing the issue carefully, I think the right approach would be:

  1. Plug the external HD.

  2. Desactivate the LVM management of the second HD: "vgchange -a n /dev/LVM2".

  3. Clone the harddisk, with "dd".

  4. For every physical volume, generate new UUIDs for the clone disk, with "pvchange -u /dev/hdb*".

  5. Change the VG UUID with "vgchange -u LVM". This will change the UUID of one of the VG, could be the original or the clone. It doesn't matter. Being different is enough.

  6. Change the name of the VG of the cloned disk from "LVM" to "LVM2", with "vgrename". I don't know if this would work without a previous "vgscan" to "discover" the new VG.

Would this work?.

6
  • That first vgchange won't work, I think you first have to use the filter as I described at serverfault.com/a/670986/270721 Nov 12, 2015 at 20:17
  • I have been using the accepted response procedure for years.
    – jcea
    Nov 13, 2015 at 23:52
  • I suppose the catch is that it works if you can actually deactivate both volume groups - if you're not using the original at all. Nov 14, 2015 at 9:23
  • In my accepted checklist the very first step is "boot from a live CD"... So you can deactivate all volume groups.
    – jcea
    Nov 15, 2015 at 23:54
  • Ah, then that makes sense. There are situations where people are able to make clones while the systems are online, though that's more of a topic for the ServerFault/UnixLinux site. Nov 16, 2015 at 15:48
1

I don't really think dd is the best option for backups in most cases, for a number of reasons. One option as someone mentioned is to use LVM's mirroring. You can disconnect the external drive and when you reattach it it will resync without having to copy 100% of everything including empty sectors like dd would. But personally I think even better/simpler is to just use rsync to copy to your external drive. Not only will it be by faaaaaar the fastest, your backup is also immediately usable without any extra steps required at all. Combine this with LVM snapshots on the external drive and you have multiple point in time copies all of which can be used anytime without any extra trickery.

1
  • Tentacle, the "dd" is the last of THREE different backups schemas I have. The other two are rsync based. I need "dd" from time to time because my internal disk has several OSs installed.
    – jcea
    Mar 30, 2011 at 11:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .