Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have several times had a text document which I need to apply several hundred find/replaces on. These find/replaces do not follow a pattern which regex can be reasonably applied to, and need to be applied in order. Previously I've resorted to doing them by hand after much searching, but is there a better way?

share|improve this question
up vote 1 down vote accepted

Please correct me if I've misunderstood your question but from your description, I take it to mean you have a single (possibly very large) Ascii .txt document and that when you say the changes must be applied "in order", you mean you'd like to do the first search/replace on the entire document, then the second search/replace on the entire document and so on.

Perhaps the easiest solution would be to create file (call it sedscript) containing a sed script, one line per change. Here's an example. The g at the end means "global", i.e., replace all occurrences, not just the first occurrence on any given line.


You could then run this as follows:

sed -f sedscript infile.txt > outfile.txt

If you're satisfied with the output, simply mv it back over the top of the input:

mv outfile.txt infile.txt

If you're on a Linux machine, that comes with sed. If you're on Windows, you can get sed (and mv) with either Cygwin or my own Hamilton C shell (including the free version).


Since you would also like to consider matches that span line ends, then, yes, one way to do that is to replace all the line ends with a special character or string, do the search/replace operations you intend, then put the line ends back when you're done.

The easiest way to do the line end conversions with sed is in separate pipeline stages. In my example here, I've replaced the \r\n sequences with a # but could be a completely arbitrary string (but it's easier if you can use a single character.)

sed 's/\r\n/#/' infile.txt | s -f sedscript | sed 's/#/\r\n/g' > outfile.txt

Inside your sedscript file, you'd then search/replace on both variations, with a space between the words or whatever you've replaced it with.

If you're able to use just a single character and don't need a multicharacter string to guarantee uniqueness, you can use \(...\) notation to create a tagged regular expression around [...] list of characters that might separate a word. Whatever it matches can be inserted into the replace string as \1.

Here's a screenshot how this might work.

Line breaks with sed

share|improve this answer
You understand correctly. Will this work with tab and newline characters as part of the find/replace? And, I'm fairly new to/uncomfortable with command line, so it would help even more if you give exact commands to use if the file I want to run the replaces on is located at C:\Users\Eric\Documents\unformatted.txt – ete Mar 12 '13 at 17:05
It would work with tab characters in either the search or replace strings but not with embedded newline characters. (Is matching strings with embedded newlines a requirement?) Replace infile.txt with your actual path, C:\Users\Eric\Documents\unformatted.txt. – Nicole Hamilton Mar 12 '13 at 20:54
Okay, replacing newlines is required for many of the find/replaces, but I could maybe work around it with so long as I can add newlines in with a find/replace, is there a way to do that? And thanks. – ete Mar 12 '13 at 21:05
Are the newlines in there simply to wrap the text to some convenient margins or because each line is a separate record? – Nicole Hamilton Mar 12 '13 at 21:29
The newlines to be removed are part of the formatting which I want to remove, I'm not certain which that falls under. – ete Mar 12 '13 at 23:41

Normally programs are good at applying a single search-and-replace combination to multiple files, not the other way around.

To me it seems like your best bet is using a word processor program with macro capabilities. Then you can write a function which performs the replacing, and then call it hundred of times with your search/replace strings. Macro-enabled word processors are, for example LibreOffice Writer and Microsoft Word.

share|improve this answer
This seems like a reasonable approach, notepad++ (my usual text editor) seems to have the functions, but I'm not sure how to make it replace newline characters correctly (extended mode makes replacing things WITH newline work fine, but not replacing newlines). – ete Mar 12 '13 at 17:09

FART should be able to help you. Just create a batch file with multiple FART calls, and if you want to replace newlines (\r\n), enable the use of C-style syntax via the -C or --c-style parameters.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .