I have a big text file (about 2GB). I want to do five search and replace actions on the same file, and would like to do this in one command. Normally I use vim, open the file, do one replace action, then the next, etc. There is one catch, as I noticed that after three or four searches vim crashes because of memory issues.

Here are two examples of the command I use in Vim:


What is the best way to handle this?


I would use sed like this :

sed -i "s/www\.abcdef/www.test.abcdef/g;s/www\.kmlnop/www.test.klmnop/g;" yourfile.txt

-i option stands for "in place" replacement. You can tell sed to create a backup of your file providing an extension to this option ( -i.bak will backup yourfile.txt as yourfile.txt.bak ).

|improve this answer|||||
  • That's quick! Not only your answer ;-) but this script with 5 search and replaces is about 10x faster as just opening the file in vim. One thing confused me though. At first I thought the .bak file would be the edited file, but it's the original of course. – SPRBRN Jul 11 '13 at 12:36
  • Ten search and replace actions (with thousands of hits) in a 2GB file in one go, no memory problems. Less than two minutes on an average desktop - super! – SPRBRN Jul 11 '13 at 12:54
  • One question... You escape the dots in the replace string. Is this necessary? – SPRBRN Jul 11 '13 at 13:00
  • 1
    You're welcome @rxt :) Actually, you're right, you can use non-escaped dots in the replacement string in sed. I tried, and it works. There's a good thread in Unix & Linux Stackexchange, and the accepted answer not mention dots as characters to escape. – ssssteffff Jul 11 '13 at 13:08
  • 2
    @rxt you said replace string, sorry, no you don't need to escape them there. – terdon Jul 11 '13 at 13:09

If you have many more search patterns, you could save them in a file and read the substitutions from there. For example, say these are the contents of replacements.txt:

www\.abcdef www.test.abcdef 
www\.klmnop www.test.klmnop

You can then read a list of N replacements and replace them with this:

while read from to; do
  sed -i "s/$from/$to/" infile.txt ; 
done < replacements.txt 


  • This assumes your search strings do not contain spaces and any strange characters need to be escaped in replacements.txt.
  • It will run one sed per replacement which may take a while if you have many replacement operations.
  • It can deal with an arbitrary number of replacements (thousands or millions or whatever) as long as you don't mind that it will take a bit more time.

Another option would be to write the above as a sed script:


You can then run the script on your file and it will make all the replacements in one go:

sed -f replace.sed infile.txt 
|improve this answer|||||
  • +1 for the ,,other option''. Could be handy to have the replacements stored in a file! (I hope I'll remember that...) – mpy Nov 17 '13 at 11:09
  • +1 for the "other option" also because it uses native functionality rather than a custom script, so is more portable/shareable – David Cook Oct 20 '17 at 1:10
  • @DavidCook thanks, but it is no more native or portable than the other. The first approach is using a POSIX shell loop, it is exactly as portable as the second. It will just be much slower since it uses a shell loop. – terdon Oct 20 '17 at 7:25
  • You're right, what I meant is that the sed script file format is more portable, because it uses builtin sed functionality rather than a script, which would have to be shared alongside the replacements.txt file. Nevertheless, they are both great options! – David Cook Oct 21 '17 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.