Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

On Linux I have two text files with russian UTF8 words sorted with sort -u (actually I used :%sort u in Vim, it produces same results).

One of the files dict.txt contains around 700000 words of my custom dictionary. Another file bad-words.txt contains often mistyped words.

I'd like to remove all words found in bad-words.txt from dict.txt.

I know that a perl script using a hash could do that, but I'm after a Unix one liner.

Ist there please a Unix command (maybe something using diff?) to perform this task? And I hope diff won't be confused by the large number of lines - because "Beyond compare" program on Windows is...

share|improve this question

comm will do this.

comm -3 dict badwords

or to sort on the fly:

comm -3 <(sort dict-input) <(badwords)

share|improve this answer
Unfortunately it complains comm: file 1 is not in sorted order (despite me presorting it) and produces strange 2-columns output... – Alexander Farber Mar 20 '13 at 21:28
comm expects the file to be sorted in ASCIIbetical order, e.g. LANG=C sort. – grawity Mar 20 '13 at 22:43
@grawity: That advice saved my day :) Thanks! – altanis Sep 9 '14 at 9:05

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.