Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I need to add tags like text of file in thousand of files in a directory and I tried it using cat and outputing it to a stream of file using

for file in *
do
    cat ../gau > temp;  //gau contain format i need to append in each file
    echo $file >>temp;
    cat ../gau_ >>temp ;//contains </DOCID>
    cat $file >>temp;  
    cat ../gau1  >> temp;  //this contain last sentence </DOC>
    cat temp > $file
done

but doing this is very slow. Can someone please tell me a better and more efficient way to do this? Is it possible to do using C? How can we open files in batches and then process them and put back as it can speed up this process since opening and writing file is a bottle neck I suppose.

Is there and premade program (which is efficient and fast) to do this job as we are short on time.

share|improve this question
    
Please don't cross-post. Also, you should ask a moderator to link your accounts if you're unable to do it yourself. –  Dennis Williamson Mar 15 '11 at 16:19

2 Answers 2

You may see a lot of improvement by only calling cat once per file:

for file in *
do
    echo "$file" | cat ../gau - ../gau_ "$file" ../gau1 > temp && mv "$file"
done

The dash in the arguments of cat picks up the filename from the pipeline. All the arguments are conCATenated together into the output file.

share|improve this answer

Some optimizations to your existing method:

Some of the cat operations can be replaced with echo:

echo "</DOCID>" >> temp

This will append </DOCID> to the temp file without having to read it from a file first. The same can be done for any of the cats from the gau* files if the content is not too long.

Rename the temp file instead of catting the data to the original file:

mv temp $file

This is an almost instant operation and doesn't involve reading all the data and then writing it all over again.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.