33

Is there a way in Ubuntu to rollback or undo the last upgrade after doing an apt-get upgrade if you don't like the results?

7
  • Not with out a lot of work, that wouldn't be worth it. Can you save your /home and start over? What didn't you like?
    – user10547
    Sep 9, 2009 at 22:31
  • I haven't done it yet, but this is a production server and I need to be sure there's no incompatibilities with our custom PHP/MySQL/Apache2 setup, and get back quickly if there is. I'd like to upgrade because I think it will fix another problem I'm having.
    – jjclarkson
    Sep 9, 2009 at 22:45
  • Unless you did dist-upgrade, I don't see what results there are not to like, since those are most likely security updates.
    – LiraNuna
    Sep 9, 2009 at 22:48
  • If you have a custom install, it's your responsibility to save patches and apply them against the current version.
    – LiraNuna
    Sep 9, 2009 at 22:51
  • 2
    If it's a server, try a dev server before upgrading the production one
    – solarc
    Oct 3, 2009 at 2:27

12 Answers 12

9

I had to do this today on my Debian system. First, I identified the time range when the offending upgrade happened, and retrieved the log entries giving the old and new version numbers of the upgraded packages:

$ awk '$1=="2016-03-20" && $3=="upgrade"' /var/log/dpkg.log
2016-03-20 16:58:22 upgrade libwebkitgtk-3.0-0:amd64 2.4.9-3 2.4.10-1
2016-03-20 16:58:24 upgrade libjavascriptcoregtk-3.0-0:amd64 2.4.9-3 2.4.10-1
2016-03-20 16:58:26 upgrade traceroute:amd64 1:2.0.22-1 1:2.1.0-1
2016-03-20 16:58:33 upgrade ethtool:amd64 1:4.2-1 1:4.5-1
2016-03-20 16:58:34 upgrade libsdl1.2debian:amd64 1.2.15+dfsg1-3 1.2.15+dfsg1-4
2016-03-20 16:58:34 upgrade subversion:amd64 1.9.3-2+b1 1.9.3-3
2016-03-20 16:58:36 upgrade libsvn1:amd64 1.9.3-2+b1 1.9.3-3
2016-03-20 16:58:56 upgrade linux-image-amd64:amd64 4.3+70 4.4+71
2016-03-20 16:58:56 upgrade linux-libc-dev:amd64 4.3.5-1 4.4.6-1
2016-03-20 16:59:03 upgrade amd64-microcode:amd64 2.20141028.1 2.20160316.1

Next, I tried to find the still-cached package files on disk (luckily for me I hadn't run autoclean):

$ awk '$1=="2016-03-20" && $3=="upgrade" {gsub(/:/, "%3a", $5); split($4, f, ":"); print "/var/cache/apt/archives/" f[1] "_" $5 "_" f[2] ".deb"}' /var/log/dpkg.log | xargs -r ls -ld
ls: cannot access '/var/cache/apt/archives/ethtool_1%3a4.2-1_amd64.deb': No such file or directory
-rw-r--r-- 1 root root   28820 Dec 18  2014 /var/cache/apt/archives/amd64-microcode_2.20141028.1_amd64.deb
-rw-r--r-- 1 root root 1978874 Dec 10 18:22 /var/cache/apt/archives/libjavascriptcoregtk-3.0-0_2.4.9-3_amd64.deb
-rw-r--r-- 1 root root  185006 Mar 12 00:41 /var/cache/apt/archives/libsdl1.2debian_1.2.15+dfsg1-3_amd64.deb
-rw-r--r-- 1 root root 1317644 Mar  3 11:30 /var/cache/apt/archives/libsvn1_1.9.3-2+b1_amd64.deb
-rw-r--r-- 1 root root 7679400 Dec 10 18:22 /var/cache/apt/archives/libwebkitgtk-3.0-0_2.4.9-3_amd64.deb
-rw-r--r-- 1 root root    6108 Dec 14 06:59 /var/cache/apt/archives/linux-image-amd64_4.3+70_amd64.deb
-rw-r--r-- 1 root root 1075506 Feb  7 21:36 /var/cache/apt/archives/linux-libc-dev_4.3.5-1_amd64.deb
-rw-r--r-- 1 root root  983174 Mar  3 11:30 /var/cache/apt/archives/subversion_1.9.3-2+b1_amd64.deb
-rw-r--r-- 1 root root   53376 Feb 28 18:35 /var/cache/apt/archives/traceroute_1%3a2.0.22-1_amd64.deb

It looks like I don't have the older ethtool package for some reason. Still, let's carry on by force-installing the older package files:

$ sudo dpkg -i /var/cache/apt/archives/amd64-microcode_2.20141028.1_amd64.deb /var/cache/apt/archives/libjavascriptcoregtk-3.0-0_2.4.9-3_amd64.deb /var/cache/apt/archives/libsdl1.2debian_1.2.15+dfsg1-3_amd64.deb /var/cache/apt/archives/libsvn1_1.9.3-2+b1_amd64.deb /var/cache/apt/archives/libwebkitgtk-3.0-0_2.4.9-3_amd64.deb /var/cache/apt/archives/linux-image-amd64_4.3+70_amd64.deb /var/cache/apt/archives/linux-libc-dev_4.3.5-1_amd64.deb /var/cache/apt/archives/subversion_1.9.3-2+b1_amd64.deb /var/cache/apt/archives/traceroute_1%3a2.0.22-1_amd64.deb
dpkg: warning: downgrading amd64-microcode from 2.20160316.1 to 2.20141028.1
(Reading database ... 139632 files and directories currently installed.)
Preparing to unpack .../amd64-microcode_2.20141028.1_amd64.deb ...
Unpacking amd64-microcode (2.20141028.1) over (2.20160316.1) ...
dpkg: warning: downgrading libjavascriptcoregtk-3.0-0:amd64 from 2.4.10-1 to 2.4.9-3
Preparing to unpack .../libjavascriptcoregtk-3.0-0_2.4.9-3_amd64.deb ...
Unpacking libjavascriptcoregtk-3.0-0:amd64 (2.4.9-3) over (2.4.10-1) ...
dpkg: warning: downgrading libsdl1.2debian:amd64 from 1.2.15+dfsg1-4 to 1.2.15+dfsg1-3
Preparing to unpack .../libsdl1.2debian_1.2.15+dfsg1-3_amd64.deb ...
Unpacking libsdl1.2debian:amd64 (1.2.15+dfsg1-3) over (1.2.15+dfsg1-4) ...
dpkg: warning: downgrading libsvn1:amd64 from 1.9.3-3 to 1.9.3-2+b1
Preparing to unpack .../libsvn1_1.9.3-2+b1_amd64.deb ...
Unpacking libsvn1:amd64 (1.9.3-2+b1) over (1.9.3-3) ...
Preparing to unpack .../libwebkitgtk-3.0-0_2.4.9-3_amd64.deb ...
Unpacking libwebkitgtk-3.0-0:amd64 (2.4.9-3) over (2.4.9-3) ...
dpkg: warning: downgrading linux-image-amd64 from 4.4+71 to 4.3+70
Preparing to unpack .../linux-image-amd64_4.3+70_amd64.deb ...
Unpacking linux-image-amd64 (4.3+70) over (4.4+71) ...
dpkg: warning: downgrading linux-libc-dev:amd64 from 4.4.6-1 to 4.3.5-1
Preparing to unpack .../linux-libc-dev_4.3.5-1_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.3.5-1) over (4.4.6-1) ...
dpkg: warning: downgrading subversion from 1.9.3-3 to 1.9.3-2+b1
Preparing to unpack .../subversion_1.9.3-2+b1_amd64.deb ...
Unpacking subversion (1.9.3-2+b1) over (1.9.3-3) ...
dpkg: warning: downgrading traceroute from 1:2.1.0-1 to 1:2.0.22-1
Preparing to unpack .../traceroute_1%3a2.0.22-1_amd64.deb ...
Unpacking traceroute (1:2.0.22-1) over (1:2.1.0-1) ...
Setting up amd64-microcode (2.20141028.1) ...
update-initramfs: deferring update (trigger activated)
amd64-microcode: microcode will be updated at next boot
Setting up libjavascriptcoregtk-3.0-0:amd64 (2.4.9-3) ...
Setting up libsdl1.2debian:amd64 (1.2.15+dfsg1-3) ...
Setting up libsvn1:amd64 (1.9.3-2+b1) ...
dpkg: dependency problems prevent configuration of libwebkitgtk-3.0-0:amd64:
 libwebkitgtk-3.0-0:amd64 depends on libwebkitgtk-3.0-common (>= 2.4.9); however:
  Package libwebkitgtk-3.0-common is not installed.

dpkg: error processing package libwebkitgtk-3.0-0:amd64 (--install):
 dependency problems - leaving unconfigured
Setting up linux-image-amd64 (4.3+70) ...
Setting up linux-libc-dev:amd64 (4.3.5-1) ...
Setting up subversion (1.9.3-2+b1) ...
Setting up traceroute (1:2.0.22-1) ...
update-alternatives: using /usr/bin/traceroute.db to provide /usr/bin/traceroute (traceroute) in auto mode
update-alternatives: using /usr/bin/lft.db to provide /usr/bin/lft (lft) in auto mode
update-alternatives: using /usr/bin/traceproto.db to provide /usr/bin/traceproto (traceproto) in auto mode
update-alternatives: using /usr/sbin/tcptraceroute.db to provide /usr/sbin/tcptraceroute (tcptraceroute) in auto mode
Processing triggers for libc-bin (2.22-3) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for initramfs-tools (0.123) ...
update-initramfs: Generating /boot/initrd.img-4.4.0-1-amd64
Errors were encountered while processing:
 libwebkitgtk-3.0-0:amd64

As the error message said, one of my packages depended on a -common package just before the upgrade, but the upgrade removed it (and apt-get can't find it any more). Luckily, its package file is still in /var/cache/apt so I can just add it to the list and try again:

$ ls -ld /var/cache/apt/archives/libwebkitgtk-3.0-common*
-rw-r--r-- 1 root root 452278 Dec 10 18:22 /var/cache/apt/archives/libwebkitgtk-3.0-common_2.4.9-3_all.deb
$ sudo dpkg -i /var/cache/apt/archives/amd64-microcode_2.20141028.1_amd64.deb /var/cache/apt/archives/libjavascriptcoregtk-3.0-0_2.4.9-3_amd64.deb /var/cache/apt/archives/libsdl1.2debian_1.2.15+dfsg1-3_amd64.deb /var/cache/apt/archives/libsvn1_1.9.3-2+b1_amd64.deb /var/cache/apt/archives/libwebkitgtk-3.0-0_2.4.9-3_amd64.deb /var/cache/apt/archives/linux-image-amd64_4.3+70_amd64.deb /var/cache/apt/archives/linux-libc-dev_4.3.5-1_amd64.deb /var/cache/apt/archives/subversion_1.9.3-2+b1_amd64.deb /var/cache/apt/archives/traceroute_1%3a2.0.22-1_amd64.deb /var/cache/apt/archives/libwebkitgtk-3.0-common_2.4.9-3_all.deb
(Reading database ... 139632 files and directories currently installed.)
Preparing to unpack .../amd64-microcode_2.20141028.1_amd64.deb ...
Unpacking amd64-microcode (2.20141028.1) over (2.20141028.1) ...
Preparing to unpack .../libjavascriptcoregtk-3.0-0_2.4.9-3_amd64.deb ...
Unpacking libjavascriptcoregtk-3.0-0:amd64 (2.4.9-3) over (2.4.9-3) ...
Preparing to unpack .../libsdl1.2debian_1.2.15+dfsg1-3_amd64.deb ...
Unpacking libsdl1.2debian:amd64 (1.2.15+dfsg1-3) over (1.2.15+dfsg1-3) ...
Preparing to unpack .../libsvn1_1.9.3-2+b1_amd64.deb ...
Unpacking libsvn1:amd64 (1.9.3-2+b1) over (1.9.3-2+b1) ...
Preparing to unpack .../libwebkitgtk-3.0-0_2.4.9-3_amd64.deb ...
Unpacking libwebkitgtk-3.0-0:amd64 (2.4.9-3) over (2.4.9-3) ...
Preparing to unpack .../linux-image-amd64_4.3+70_amd64.deb ...
Unpacking linux-image-amd64 (4.3+70) over (4.3+70) ...
Preparing to unpack .../linux-libc-dev_4.3.5-1_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.3.5-1) over (4.3.5-1) ...
Preparing to unpack .../subversion_1.9.3-2+b1_amd64.deb ...
Unpacking subversion (1.9.3-2+b1) over (1.9.3-2+b1) ...
Preparing to unpack .../traceroute_1%3a2.0.22-1_amd64.deb ...
Unpacking traceroute (1:2.0.22-1) over (1:2.0.22-1) ...
Selecting previously unselected package libwebkitgtk-3.0-common.
Preparing to unpack .../libwebkitgtk-3.0-common_2.4.9-3_all.deb ...
Unpacking libwebkitgtk-3.0-common (2.4.9-3) ...
Setting up amd64-microcode (2.20141028.1) ...
update-initramfs: deferring update (trigger activated)
amd64-microcode: microcode will be updated at next boot
Setting up libjavascriptcoregtk-3.0-0:amd64 (2.4.9-3) ...
Setting up libsdl1.2debian:amd64 (1.2.15+dfsg1-3) ...
Setting up libsvn1:amd64 (1.9.3-2+b1) ...
Setting up linux-image-amd64 (4.3+70) ...
Setting up linux-libc-dev:amd64 (4.3.5-1) ...
Setting up subversion (1.9.3-2+b1) ...
Setting up traceroute (1:2.0.22-1) ...
update-alternatives: using /usr/bin/traceroute.db to provide /usr/bin/traceroute (traceroute) in auto mode
update-alternatives: using /usr/bin/lft.db to provide /usr/bin/lft (lft) in auto mode
update-alternatives: using /usr/bin/traceproto.db to provide /usr/bin/traceproto (traceproto) in auto mode
update-alternatives: using /usr/sbin/tcptraceroute.db to provide /usr/sbin/tcptraceroute (tcptraceroute) in auto mode
Setting up libwebkitgtk-3.0-common (2.4.9-3) ...
Setting up libwebkitgtk-3.0-0:amd64 (2.4.9-3) ...
Processing triggers for libc-bin (2.22-3) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for initramfs-tools (0.123) ...
update-initramfs: Generating /boot/initrd.img-4.4.0-1-amd64

Success! Actually this didn't solve my problem. But it successfully downgraded the packages, QED.

6

I also had to undo an package upgrade today on a couple of Debian servers. I successfully reverted the packages to the last version by using aptitude, whereas the following awk command was very helpful.

(In the below command replace the date string with the date of the day from which on you want to revert the upgrades)

awk 'BEGIN{ start="0" } { if($0 ~ /Log started: 2017-06-20/) { start="1"} if ( start == "1" && $0 ~ /Unpacking.*over/) {gsub(/[\s\t)( ]+/,"",$5); printf("%s=%s ", $2 , $5)}}' /var/log/apt/term.log

review the output to verify that these are the packages and versions to be reverted. Then use apt or aptitude to do the downgrade of the listed packages:

apt-get install [paste output here]

or

aptitude install [paste output here]

I hope that this is also a useful time saver for others.

3
  • Thanks. It gives me Unable to find a version "5.4.0.60.63" for the package "linux-generic", is there any way to resolve that? aptitude update does not help.
    – jangorecki
    Jan 18, 2021 at 16:35
  • I suspect that linux-generic=5.4.0.60.63 is no longer available in the official package sources, and that you have cleaned your local apt package cache. The solution for this dilemma depends on your specific linux distribution. If you are using Debian you could try using the snapshot archive which provides a kind of wayback-machine to old packages which are no longer available otherwise: snapshot.debian.org Jan 19, 2021 at 6:52
  • This was astoundingly helpful.
    – Br.Bill
    Jan 26 at 23:13
5

aptitude gives you access to all versions of a package if available according to Debian package management.

1
  • 2.7.3 might help you downgrade to stable. 2.7.16 explains saving and restoring dpkg state. dpkg-repack allows you to "compress" a single package. Nov 2, 2012 at 20:50
5

I came across Hartman's Blogstatic Blog: How to Undo an Update in Ubuntu Lucid

The first step to undoing the offending update was to find out what updates it was exactly. After searching some forums I came across a way to see my update history: Open synaptic package manager ("sudo synaptic" in the terminal). From the menu bar, click File -> History and you will see all your updates sorted by date.

Unfortunately I had installed about 20 updates today, and I didn't know which one had caused the problem. By searching through each of the packages named in the History list, I was able to downgrade a few at a time until the problem was solved and I had identified the offending update. To do this:

Use the search bar to find the package you want to downgrade. Once you've found what you're looking for, click on the package to select it. From the menu bar, click Package -> Force Version and select the previous version of the package from the drop down menu. Click the "Apply" button to apply the downgrade.

2
  • 1
    I am expecting everyone here to know the basics on how to look up the last offending update:
    – Manav Brar
    Jun 14, 2012 at 20:56
  • 2
    What was that edit about? Do you have another question? Do you want us to play a guessing game?
    – slhck
    Jun 15, 2012 at 7:15
2

I believe not, aside from taking a full backup of the relevant filesystems (those that contain /, /bin, /lib, /sbin, /usr, /var, /etc and /boot (which may all be on on filesystem) and your boot record) so you can roll the machine back afterwards.

3
  • I could (and probably will) take a bare metal backup, but I wanted a software option that would let me undo the upgrade in a faster time frame. Restoring from the bare metal backup could take several hours.
    – jjclarkson
    Sep 9, 2009 at 22:47
  • 1
    You could make sure you have copies of all the packages you are about to upgrade in their previous versions (they are probably still sat in your apt cache somewhere in /var) and any relevant config files stored away. You could then try force a roll back by explicitly telling dpkg to install those versions. You might have some work to do afterwards when you want the normal upgrades to happen, so it isn't something I'd recommend. Sep 9, 2009 at 23:20
  • A backup made with rsync (timestamping enabled) can be restored quite quickly, compared to a backup made with tar. Oct 6, 2012 at 10:48
2

I run my linux servers in a virtualized environment and run a shapshot just before an apt-get upgrade, or any major 3rd party updates/upgrades for that matter.

Then if something goes wrong, I simply revert and life goes on until I can find out more information.

This came in very handle when I upgraded my Ubuntu box to 12.04, and somehow MySQL was completely non-functional after the upgrade. I rolled back, found the answer later, reran the upgrade, fixed MySQL, and life was good.

1

You could try checkinstall

After you ./configure; make your program, CheckInstall will run make install (or whatever you tell it to run) and keep track of every file modified by this installation, using the excelent installwatch ...

So maybe you could tell it to run aptitude safe-upgrade and it would keep track of every modification made by the upgrade.

1

There's a project called Nexenta that combines the OpenSolaris kernel with the Ubuntu userspace. It provides a tool to integrate Solaris's ZFS and Debian's apt in order to provide an undo button for upgrades. See here: http://www.nexenta.org/os/TransactionalZFSUpgrades

More generally, what you need is a versioning file system. Btrfs for Linux is in development.

1

Using Apt-Undo is a possible option, but it can only work if you are using it to installed and uninstall packages. It won't help if you've already uninstalled software the normal way.

http://www.ubuntugeek.com/apt-undo-a-simple-way-of-undoing-apt-actions.html http://lkubuntu.wordpress.com/2011/07/27/apt-undo-a-simple-way-of-undoing-apt-actions/

1

DPkg::Pre-Install-Pkgs and ZFS on Linux snapshots

If you installed your operating system on a ZFS file system (e.g., ZFS on Linux), you can configure apt-get to run zfs snapshot before it installs or upgrades anything, which will back up your file system instantaneously. It may work with backup mechanisms other than ZFS snapshots, but I'll leave that testing to others.

The Script

Create a file like

/etc/apt/apt.conf.d/71backup

with contents

// Tell `apt-get' to take a ZFS snapshot before installing or upgrading a
// set of packages:
DPkg::Pre-Install-Pkgs {"/sbin/zfs snapshot rpool/ROOT/debian@apt-get_$(date '+%Y-%m-%d-%H%M')";};

where rpool/ROOT/debian should be replaced by the name of the ZFS file system to which your operating system is mounted. You gave it that name when you first installed your OS, and it can be found under attribute NAME with the command

# zfs list -t filesystem
NAME ...
...
rpool/ROOT/debian ...
...

But does it work?

You'd better take a snapshot before following the configuration advice of some random internet person:

# zfs snapshot rpool/ROOT/debian@$(date '+%Y-%m-%d-%H%M%S')_test
# zfs list -t snapshot | grep rpool
...
rpool/ROOT/debian@2018-08-01-230001_test

There it is. Should error: fn_borked soon occur, you may return your system to its blissful current state with

# zfs rollback rpool/ROOT/debian@2018-08-01-230001_test

Now try it out by installing two tiny games that are likely available in your repo:

# apt-get install tanglet sudoku
...
# zfs list -t snapshot | grep apt\-get
rpool/ROOT/debian@apt-get_2018-08-02-033614

That snapshot contains your file system as it was before the two games were installed.

# exit
$ sudoku

Fun times for grandma, but you hate sudoku.

$ sudo -i
# zfs rollback rpool/ROOT/debian@apt-get_2018-08-02-033614
# exit
$ sudoku
-bash: /usr/games/sudoku: No such file or directory
$ tanglet
-bash: tanglet: command not found

After multiple snapshots have been taken, you may roll back to any earlier one by adding the -r flag. In our case, for instance, try

# zfs -r rollback rpool/ROOT/debian@2018-08-01-230001_test

Be warned, however, that not only will this return your file system to the state it was in when rpool/ROOT/debian@2018-08-01-230001_test was taken, but it will also delete irretrievably all later snapshots. If you followed along with this post, the snapshot rpool/ROOT/debian@apt-get_2018-08-02-033614 would now be gone.

I tested apt-get upgrade on a Debian GNU/Linux with

# apt-get -t=oldstable install tanglet sudoku
...
# apt-get upgrade
...
# zfs list -t snapshot | grep apt\-get

It works. One snapshot was created for the install command, another for the upgrade command.

Caveat: I tested this for the first time today and know very little about apt's inner workings. Should this break something for you or entail risks my benewbed mind hasn't considered, please comment about it below.

0

I have successfully done that few times but I won't recommended it. This is what I did (if I recall correctly):

1) Remove any non-official software which are not included in the default ubuntu repositories (it may not be required, but I suggest it as they may get in your way).

2) Change your /etc/apt/sources.list (and sources.list.d/*) to the previous version (comment all non-official repositories).

3) apt-get update / aptitude update

4) Using aptitude, downgrade core packages (like X11, libraries, etc). It will start firing a lot of broken packages... so you will need to solve each case (you need to know how to do that in aptitude). The way you downgrade it is by going to the description panel and installing the version (by pressing +) at the bottom.

5) Repeat #4 until all software belongs to your target version (check on version column in aptitude).

These are the reason why I don't recommend this method:

  • It takes a lot of time (its a painfully process)
  • Some application may not work properly (as they may still have the most recent configuration). In that case you will need to "purge" and reinstall.
  • There are system-related updates that won't work after downgrade
  • Its highly risky as you may end-up with an unusable system

I would highly recommend to do a clean install and move your configuration little by little. It also takes time, but at the end you have a stable version.

The reason I did it was mainly as experimentation and as a result of desperation.

0

When using Debian there is a tool apt-revert where you have a CLI to select which apt call should be reverted.

This can, of course, only work in general, if all old package versions are still available, even if already deleted locally. This is possible for Debian with snapshots.debian.org which apt-revert uses.

If something like this exists for Ubuntu, too, it probably wouldn't be a lot of work to adapt apt-revert to also work on Ubuntu.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .