This is mostly off-topic, but you could use
find -maxdepth 1 -type f -name '*.txt' | xargs python -c '
import fileinput
for line in fileinput.input(inplace=True):
print line.replace("blah", "blee"),
'
The main benefit here (over ... xargs ... -I {} ... sed ...
) is speed: you avoid invoking sed
10 million times. It would be faster still if you could avoid using Python (since python is kind of slow, relatively), so perl might be a better choice for this task. I'm not sure how to do the equivalent conveniently with perl.
The way this works is that xargs
will invoke Python with as many arguments as it can fit on a single command line, and keep doing that until it runs out of arguments (which are being supplied by ls -f *.txt
). The number of arguments to each invocation will depend on the length of the filenames and, um, some other stuff. The fileinput.input
function yields successive lines from the files named in each invocation's arguments, and the inplace
option tells it to magically "catch" the output and use it to replace each line.
Note that Python's string replace
method doesn't use regexps; if you need those, you have to import re
and use print re.sub(line, "blah", "blee")
. They are Perl-Compatible RegExps, which are sort of heavily fortified versions of the ones you get with sed -r
.
edit
As akira mentions in the comments, the original version using a glob (ls -f *.txt
) in place of the find
command wouldn't work because globs are processed by the shell (bash
) itself. This means that before the command is even run, 10 million filenames will be substituted into the command line. This is pretty much guaranteed to exceed the maximum size of a command's argument list. You can use xargs --show-limits
for system-specific info on this.
The maximum size of the argument list is also taken into account by xargs
, which limits the number of arguments it passes to each invocation of python according to that limit. Since xargs
will still have to invoke python quite a few times, akira's suggestion to use os.path.walk
to get the file listing will probably save you some time.
sed
for each file. I'm not sure if there's a way to open, edit, save, and close a series of files insed
; if speed is essential you may want to use a different program, perhaps perl or python.sed
is probably faster than launchingpython
orperl
as well, except if you do everything in that interpreter.