Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

On Linux I have two text files with russian UTF8 words sorted with sort -u (actually I used :%sort u in Vim, it produces same results).

One of the files dict.txt contains around 700000 words of my custom dictionary. Another file bad-words.txt contains often mistyped words.

I'd like to remove all words found in bad-words.txt from dict.txt.

I know that a perl script using a hash could do that, but I'm after a Unix one liner.

Ist there please a Unix command (maybe something using diff?) to perform this task? And I hope diff won't be confused by the large number of lines - because "Beyond compare" program on Windows is...

share|improve this question

comm will do this.

comm -3 dict badwords

or to sort on the fly:

comm -3 <(sort dict-input) <(badwords)

share|improve this answer
Unfortunately it complains comm: file 1 is not in sorted order (despite me presorting it) and produces strange 2-columns output... – Alexander Farber Mar 20 '13 at 21:28
comm expects the file to be sorted in ASCIIbetical order, e.g. LANG=C sort. – grawity Mar 20 '13 at 22:43
@grawity: That advice saved my day :) Thanks! – altanis Sep 9 '14 at 9:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .