Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two text files on Linux. One contains a list of valid IDs. E.g:


The other contains a list of invalid IDs. But, some of these also appear on the list of valid IDs, in this example "efgh":


How can I easily construct a text file that contains all the lines from the invalid list that do not appear in the valid list? That is, I want to end up with a text file that has:


I'd like either some Linux commandline magic of some clever Vim trickery. Thanks in advance!

share|improve this question
up vote 9 down vote accepted

I think comm can help you do what you want.

It is a less known command line tool that shall be present in most Linux systems.

share|improve this answer
Just what I needed. I used comm -2 -3 invalid.txt valid.txt. – Sander Marechal Jul 7 '10 at 8:53

Assuming you have files named valid and invalid, you can sort them:

sort valid > valid-sorted; sort invalid > invalid-sorted

then compare:

diff valid-sorted invalid-sorted

The output will be (assuming your example data):

< abcd
< ijkl
> mnop
> qrst

The stuff with '<' is only in the first file, the stuff with '>' only in the second (invalid).

Thus diff valid-sorted invalid-sorted |grep '^>' should give you the strings appearing in invalid but not in valid.

Stripping out the leading '>' is left as an exercise for the reader ;-).

Caveat: This will not work if your input files actually contain '>' at the start of a line. You'll have to do some tricking then...

share|improve this answer

I have my own utility for that, which relies on another; they also work on unsorted input. Basic usage is

except file1 file2 > file2-except-any-lines-in-file1
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.