Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Does there exist a command line utility to check for and ideally fix grunge in text files? e.g.

  • Inconsistent or non-native line endings (lf vs cr/lf)

  • Extra whitespace at end of lines

  • Spurious byte order markings (Visual Studio sometimes inserts these, even though they are neither necessary nor desirable in UTF-8 files)

And possibly other things I hadn't thought of? (Closest I can find so far is something called tab2space, but the primary function of that is as it says on the tin, to expand tabs, and that's something I don't want.)

share|improve this question
1  
which OS shall it be? – akira Feb 21 '11 at 11:17
    
Windows or Linux. (Though I would normally expect a utility like that, that doesn't need to do any platform specific stuff, to be cross-platform or at least easily portable.) – rwallace Feb 21 '11 at 11:34
up vote 3 down vote accepted

To convert to/from various line endings there are programs called unix2dos/dos2unix or todos/fromdos.

To remove extra whitespace at the end of the line you can use sed s/[ \t]*$// (do this step after correcting line endings).

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .