Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

What's the Unix command to take a text file and strip away whitespace and punctuation, leaving only the words, one word per line?

share|improve this question

1 Answer 1

up vote 5 down vote accepted

This is already answered here on stackoverflow for shell commands that does this.

Alternatively you can also use vim to do this as outlined in this post on stackoverflow.

The top answer given by rampion on how to using shell:


You could use grep:

  • -E '\w+' searches for words
  • -o only prints the portion of the line that matches
% cat temp
Some examples use "The quick brown fox jumped over the lazy dog,"
rather than "Lorem ipsum dolor sit amet, consectetur adipiscing elit"
for example text.
# if you don't care whether words repeat
% grep -o -E '\w+' temp
Some
examples
use
The
quick
brown
fox
jumped
over
the
lazy
dog
rather
than
Lorem
ipsum
dolor
sit
amet
consectetur
adipiscing
elit
for
example
text

If you want to only print each word once, disregarding case, you can use sort

  • -u only prints each word once
  • -f tells sort to ignore case when comparing words
# if you only want each word once
% grep -o -E '\w+' temp | sort -u -f
adipiscing
amet
brown
consectetur
dog
dolor
elit
example
examples
for
fox
ipsum
jumped
lazy
Lorem
over
quick
rather
sit
Some
text
than
The
use

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.