Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a big text file (about 2GB). I want to do five search and replace actions on the same file, and would like to do this in one command. Normally I use vim, open the file, do one replace action, then the next, etc. There is one catch, as I noticed that after three or four searches vim crashes because of memory issues.

Here are two examples of the command I use in Vim:

:%s/www\.abcdef/www.test.abcdef/g 
:%s/www\.klmnop/www.test.klmnop/g

What is the best way to handle this?

share|improve this question

2 Answers 2

up vote 4 down vote accepted

I would use sed like this :

sed -i "s/www\.abcdef/www.test.abcdef/g;s/www\.kmlnop/www.test.klmnop/g;" yourfile.txt

-i option stands for "in place" replacement. You can tell sed to create a backup of your file providing an extension to this option ( -i.bak will backup yourfile.txt as yourfile.txt.bak ).

share|improve this answer
    
That's quick! Not only your answer ;-) but this script with 5 search and replaces is about 10x faster as just opening the file in vim. One thing confused me though. At first I thought the .bak file would be the edited file, but it's the original of course. –  SPRBRN Jul 11 '13 at 12:36
    
Ten search and replace actions (with thousands of hits) in a 2GB file in one go, no memory problems. Less than two minutes on an average desktop - super! –  SPRBRN Jul 11 '13 at 12:54
    
One question... You escape the dots in the replace string. Is this necessary? –  SPRBRN Jul 11 '13 at 13:00
1  
You're welcome @rxt :) Actually, you're right, you can use non-escaped dots in the replacement string in sed. I tried, and it works. There's a good thread in Unix & Linux Stackexchange, and the accepted answer not mention dots as characters to escape. –  ssssteffff Jul 11 '13 at 13:08
2  
@rxt you said replace string, sorry, no you don't need to escape them there. –  terdon Jul 11 '13 at 13:09

If you have many more search patterns, you could save them in a file and read the substitutions from there. For example, say these are the contents of replacements.txt:

www\.abcdef www.test.abcdef 
www\.klmnop www.test.klmnop

You can then read a list of N replacements and replace them with this:

while read from to; do
  sed -i "s/$from/$to/" infile.txt ; 
done < replacements.txt 

NOTES:

  • This assumes your search strings do not contain spaces and any strange characters need to be escaped in replacements.txt.
  • It will run one sed per replacement which may take a while if you have many replacement operations.
  • It can deal with an arbitrary number of replacements (thousands or millions or whatever) as long as you don't mind that it will take a bit more time.

Another option would be to write the above as a sed script:

s/www\.abcdef/www\.test\.abcdef/g;
s/www\.kmlnop/www\.test\.klmnop/g;
s/aaaa/bbbb/g;
s/cccc/dddd/g;
s/eeee/ffff/g;

You can then run the script on your file and it will make all the replacements in one go:

sed -f replace.sed infile.txt 
share|improve this answer
    
+1 for the ,,other option''. Could be handy to have the replacements stored in a file! (I hope I'll remember that...) –  mpy Nov 17 '13 at 11:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.