Query dpkg for All Packages Installed/Removed from Default Image

Question

Is there any way to query dpkg to tell what packages have been added/removed since the base install of a system?

I am running Debian 5.

Answer 1

As far as I can tell there's not a straight-forward way to do that. dpkg database contains only current state of packages so any history would have to be deduced or parsed from logs.

---

Short answer (aka one-liner for the lazy)

dpkg-query -f='${Package}\n' -W  | grep -vx "`grep 'Setting up' /var/log/installer/syslog | cut -d' ' -f8`"

Explanation

1. Getting a list of packages installed by the installer

debian-installer writes its logs to /var/log/installer. There's a syslog file there and we can grep that to see what packages were installed:

grep 'Setting up' /var/log/installer/syslog

Note that we are looking for Setting up or Unpacking rather than eg. Selecting since we want all packages, including those installed as dependencies. We need to extract package names from that:

grep 'Setting up' /var/log/installer/syslog | cut -d' ' -f8

2. Getting a list of currently installed packages

Now we need a list of currently installed packages to do a reverse grep against. We want only package names, eg. no descriptions to avoid filtering out too much so plain dpkg -l is not great. We can use dpkg-query instead:

dpkg-query -f='${Package}\n' -W

3. Filtering out packages installed at startup

All that's left to do is to filter out contents of the 1st list from the 2nd list. For that we use grep -vx. -v is for "reverse", -x (or --line-regex) is for "exact". The latter only matches whole lines which makes sure that we filter out eg. apt but not aptitude.

So here's the whole thing put together:

 # currently installed
 current="`dpkg-query -f='${Package}\n' -W`"
 # base install (let's also remove duplicates to make verification easier)
 base="`grep 'Setting up' /var/log/installer/syslog | cut -d' ' -f8 | sort | uniq`"
 # your answer
 extra="`echo "$current" | grep -vx "$base"`"
 removed="`echo "$base" | grep -vx "$current"`"

4. Verifying results

To check if we got the right packages we can count the lines:

 echo "$base" | wc -l         # 160
 echo "$current" | wc -l      # 1677
 echo "$extra" | wc -l        # 1517
 echo "$removed" | wc -l      # 0 (I used "netinst")

---

Update: if /var/log/installer is not there

/var/log/installer is removed from the system if package installation-report is purged. In that case we can get try and get the list of packages that were loaded by the installer from the oldest /var/log/dpkg.log.* file.

There's 2 issues with that:

1. there is no guarantee that the info we need is still there since these files are rotated (still there in my case - install on 2010-10-01 and lots of updates since).
2. the oldest file will probably include logs of packages being installed after the initial install.

There's nothing we can do about (1). With (2) we can manually (by timestamps perhaps) establish which package concludes the installation by debian-installer. In my case it appears to have been os-prober (which I believe might be common in some cases due to its relation to grub).

Here's example code to get $base using this method:

 last_installed="os-prober"
 base_dpkglog="`zgrep ' install ' dpkg.log.8.gz | cut -d' ' -f4 | while read pkg; do 
     [ "$pkg" != "$last_installed" ] && echo $pkg || { echo $pkg; break; }
 done | sort | uniq`"

And a quick verification:

 echo "$base_dpkglog" | wc -l                    # 159
 echo "$base" | wc -l                            # 160
 diff -d <(echo "$base") <(echo "$base_dpkglog") # 31d30
                                                 # < dpkg