38

(This question is very similar to 10458. It was suggested that Fedora and Ubuntu/Debian are different enough to warrant different answers.)

As I use any Ubuntu setup I gradually install a number of packages over and above the baseline installation. If I reinstall, or if I need to install a new machine, I usually want to reinstall those specific packages, and I want to do it fast to get back to work with a minimum of hassle. As far as I've seen all of the package managers (apt-get, aptitude and synaptic) can tell me which packages are installed, and they all have logs (albeit different ones for each tool, which is a hassle). But none of them can tell me which packages I've installed, as opposed to their dependencies or system updates. Even the logs are tricky in that I'm not entirely sure what I should be extracting from them, or how to integrate them (in the case of the various apt family tools). This means that each time I re-install, or even just backup, I'm not sure how to re-create that list.

I'm not necessarily expecting any of the tools to do this for me, but if they don't I'm looking for workarounds. Even patterns to grep for, good rules of thumb, or a clear idea of what exactly is being logged, would be useful. There may not be a "best answer" here but good ones would be very helpful.

Most of the answers below provide some approximation of what I am looking for, and are useful to some degree. The chosen one is the one that comes closest to a reasonably automatic way of re-installing my tools on a new system, even with all its caveats.

Improve this question
3
  • You're not likely to get one easily-shared answer for all Linux distros. Package management is a big part of what distinguishes different Linux distros.
    – Telemachus
    Jul 17, 2009 at 13:27
  • Telemachus - True. And it may make sense to split this into two questions. But it seemed a fairly specialized question, and I use both systems, so I didn't want to narrow it down too much in advance. It looks like most of the answers here are for dpkg/apt, so a separate question for rpm/yum may make sense.
    – quark
    Jul 17, 2009 at 17:13
  • Switch to NixOS :) (just trolling).
    – Alexey
    Feb 15, 2017 at 10:13

8 Answers 8

Reset to default
31

On any Debian based machine, this is one common way to duplicate a package set. On the old machine:

dpkg --get-selections "*" > my_favorite_packages

Copy the file my_favorite_packages to the new machine (a thumb drive is a good option, but scp also works fine). Then run this sequence (with root privileges):

apt-get update
dpkg --set-selections < my_favorite_packages
apt-get -u dselect-upgrade

This doesn't get you only the packages you installed. It also gets their dependencies, etc. Also, if the repositories between the two machines are different, all bets are off.

As far as logs, apt-get keeps a log at /var/log/apt/history.log (thanks to Tshepang for updating this in a comment); dpkg does (at /var/log/dpkg.log), but it's famously hard to parse and can only be read with root privileges; aptitude has one at /var/log/aptitude and you can page through it with regular user privileges.

As far as I can tell, you are right that none of these logs track specifically what you installed as opposed to auto-installed dependencies. You can get that information, however, from an aptitude search. Search for all installed packages that were also installed automatically:

aptitude search '~i ~M'

If you want only the ones you installed (not the auto-dependencies), negate the ~M:

aptitude search '~i !~M'

If you want that formatted so that you have only the names of packages and the word "install", aptitude can do that too. This gives you a list ready to feed to dpkg --get-selections:

aptitude search '~i !~M' -F "%p install"

(I've got nothing on RedHat or RedHat-based systems. Sorry. There really is no one answer for Linux per se since package management is a big part of what makes different distros different.)

Improve this answer
4
  • Seems like a combination of your advice and Ludwig's might do the trick: aptitude can generate a script to feed to dpkg so that's automatible which is a serious win. And if one were to do it on the vanilla machine, the difference in lists is close enough to what I'm asking for to be practically useful.
    – quark
    Jul 22, 2009 at 0:09
  • 3
    Note that APT now keeps a log in "/var/log/apt/history.log", and is used by apt-get, synaptic, and aptitude (so far as I've seen). This is since early 2010.
    – tshepang
    Feb 14, 2011 at 21:21
  • The dpkg.log statements don't appear to be true in Ubuntu 14.04 as any user I can get my selections easily, not trivial, but not too difficult. awk '$3 != "install" { next } ; { gsub(/:.+/, "", $4) ; print $4 }' /var/log/dpkg.log | sort | uniq
    – Steve Buzonas
    Apr 14, 2015 at 11:21
  • Actually, I realize it's not as easy as I initially thought, the flaw in the previous awk script pays no attention to uninstalled packages. The following awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u does.
    – Steve Buzonas
    Apr 14, 2015 at 13:54
7

Use dpkg -l '*' > jaunty.original to remeber all installed packages on a freshly installed system.

After you have installed all your additional packages do dpkg -l '*' > mysystem.2009017.

The additional packages are just the difference: diff jaunty.original mysystem.2009017

Improve this answer
3
  • 3
    The basic idea is strong: use command line to dump the list of currently installed applications, and then use command line to install those packages on a new machine. You can get pretty creative and specific with the approach.
    – pcapademic
    Jul 16, 2009 at 23:13
  • 1
    I prefer dpkg --get-selections
    – CesarB
    Jul 17, 2009 at 11:19
  • While this doesn't track the packages I added distinct from their dependencies, it definitely generates a useful list.
    – quark
    Jul 22, 2009 at 0:03
3

Aptitude is actually quite good at this. Aptitude does know when something was installed by hand or by dependency and you can tell it to remove things that are no longer needed and were only installed because something else depended on it always keeping up your system as small as possible.

There's a handful of packages that make up an Ubuntu installation, ubuntu-minimal, ubuntu-desktop, ubuntu-server and so on. If you tell Aptitude to mark those as manually installed and remove everything else, then you end up with the minimum amount possible of packages.

I explain how to do all that in two posts at my blog: Cleaning up a Debian GNU/Linux and Cleaning up a Debian GNU/Linux (or Ubuntu), reprise. In short, the answer you are looking for is:

aptitude search ~i | grep -v "i A"

The last time I worked with that, if you used apt-get, then it didn't work. That's why I always recommend aptitude and as far as I know, Debian is deprecating apt-get in favor of aptitude.

I don't know how to do it on Fedora and you should probably separate than into a different question. Fedora and Ubuntu are different operating systems and should be treated as such (even if they share their kernel and some other stuff).

Improve this answer
9
  • 2
    I think that you can get that information without needing grep: aptitude search '~i !~M' should do the trick.
    – Telemachus
    Jul 17, 2009 at 10:57
  • 1
    Also, apt-get isn't deprecated. Debian recommends aptitude for package management on the command line, but that's a far cry from deprecating apt-get.
    – Telemachus
    Jul 17, 2009 at 11:17
  • There is something subtle here. Looking for "A" in the third column does seem to mark packages that I know are installed as dependencies. But it clearly isn't catching all of them: most of the list was definitely not installed by my explicit request.
    – quark
    Jul 21, 2009 at 23:57
  • @Telemachus. Your command and the one with the pattern do not do exactly the same thing: the two lists have different contents. I don't know enough about aptitude to tell you why though.
    – quark
    Jul 22, 2009 at 0:00
  • @Pablo: The links to your log seem broken. If you can fix them I'd definitely want to read them.
    – quark
    Jul 22, 2009 at 0:00
2

On debian apt-show-versions shows the versions of the installed packages.

Improve this answer
1
  • Noted. This doesn't appear to be installed (by default) on Ubuntu.
    – quark
    Jul 21, 2009 at 23:50
2

When just using dpkg you don't know whether the package was manually installed by the user or automatically (as a dependency or during the initial OS install). If you want to retain that information, you need to get a list of only the packages that were actually manually installed.

For that, you can use either of these two one-liners. Both yield the exact same output on my machine and are more precise than all solutions proposed up until now in this question. They are a combination of the two answers (1) and (2). Note that I originally posted this answer here.

Using apt-mark:

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

Using aptitude:

comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

Very few packages still fall through the cracks, although I suspect these are actually installed by the user, either right after the installation through the language localization setup or e.g. through the Totem codec installer. Also, the linux-header versions also seem to accumulate, even though I've only installed the non version-specific metapackage. Examples:

libreoffice-help-en-gb
openoffice.org-hyphenation
gstreamer0.10-fluendo-mp3
linux-headers-3.13.0-29

How does it work

  1. Get the list of manually installed packages. For aptitude, the additional sed strips out remaining whitespace at the end of the line.
  2. Get the list of packages installed right after a fresh install.
  3. Compare the files, only output the lines in file 1 that are not present in file 2.

Other possibilities don't work as well:

  • Using the ubuntu-14.04-desktop-amd64.manifest file (here for Ubuntu 14.04) instead of /var/log/installer/initial-status.gz. More packages are shown as manually installed even though they are not.
  • Using apt-mark showauto instead of /var/log/installer/initial-status.gz. apt-mark for example doesn't include the xserver-xorg package, while the other file does.

Both list more packages than the above solution.

Improve this answer
1

On apt-based systems look at /var/log/apt/term.log. For me, there's a pretty clear line to draw where the installation ended and where my installations began.

Improve this answer
2
  • Less useful for me, because there is an intermixing of manual installs and system updates. Also depending on your setup term.logs will eventually be outdated and deleted, so it won't go back as far as I need it to.
    – quark
    Jul 21, 2009 at 23:49
  • For anyone trying this, note that wading through apt's log seems to be a lot more work than other options discussed here. It's certainly not automatic to extract a package list from the log.
    – quark
    Jul 21, 2009 at 23:55
1

From man aptitude-create-state-bundle:

aptitude-create-state-bundle produces a compressed archive storing the files that are required to replicate the current package archive state.

This will retain the same information that aptitude has on which packages were manually installed.

It's meant to be used with aptitude-run-state-bundle:

aptitude-run-state-bundle unpacks the given aptitude state bundle created by aptitude-create-state-bundle(1) to a temporary directory, invokes on it with the supplied , and removes the temporary directory afterwards.

Improve this answer
0

I am biased, and the solution I present is not always possible, but I got tired of this situation. The result is that I don't install anything anymore with the update/package manager tools.

I took a quite hard route though (I had strict requirements for versions). I created a huge makefile which downloads, compiles and install in my home directory every package (program, library, whatever) I need. I developed it stepwise, piece by piece. The makefile downloads and compiles everything, even the compilers.

When I move to a new system, or reinstall, I just copy the makefile (plus some supporting stuff), run make world and come back the next day.

For some programs I develop (so I have control on), I use a tool I programmed, chestnut package manager. Sort of like .app folders on MacOSX. Everything is in the package, so I know what is installed at any time, and I know it's self contained and self sufficient (except for system libs)

Improve this answer
5
  • You could just put the package manager install commands in a script and have the same effect; assuming that the code you need is packaged. Your approach looks very similar to gentoo's.
    – wcoenen
    Jul 17, 2009 at 12:52
  • 1
    Nice to know about. Looks like a lot of extra work beyond a default Ubuntu/Debian system. I can see manually maintaining some packages, but maintaining all of them this way is more work than I want to do.
    – quark
    Jul 21, 2009 at 23:51
  • Yes, but with the additional issue that the ubuntu/fink/darwinports stuff do not work cross platform everywhere (I was on a digital and an IBM sp4 once). I don't claim this is a nice way to go. I just say that it does the job, albeit in a ugly, smelly way, and I keep full control of what goes on my system.
    – Stefano Borini
    Jul 22, 2009 at 0:30
  • Of course, I could decide one of these days to actually take a serious look to emerge and rework everything with it.
    – Stefano Borini
    Jul 22, 2009 at 0:31
  • This path is more common these days when you consider tools like chef and puppet.
    – Steve Buzonas
    Apr 14, 2015 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.