Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

For some reason this script outputs three files for each original, rather than one.

Must have made some trivial error - I'm new to this!

Much obliged if anyone could explain why this happens.

Script:-

for f in *.txt
do
   noOfRows=$(cat $f | wc -l)
   relevantRows=$(expr $noOfRows - 5)
   head -n $relevantRows $f | tee ${f%.txt}-Amended.txt
done

Result of ls command:-

E12-5_F2_NEG-Amended-Amended-Amended.txt  E12-5_M3_POS-Amended-Amended-Amended.txt
E12-5_F2_NEG-Amended-Amended.txt          E12-5_M3_POS-Amended-Amended.txt
E12-5_F2_NEG-Amended.txt                  E12-5_M3_POS-Amended.txt
E12-5_F2_NEG.txt                          E12-5_M3_POS.txt
E12-5_F2_POS-Amended-Amended-Amended.txt  E12-5_M4_NEG-Amended-Amended-Amended.txt
E12-5_F2_POS-Amended-Amended.txt          E12-5_M4_NEG-Amended-Amended.txt
E12-5_F2_POS-Amended.txt                  E12-5_M4_NEG-Amended.txt
E12-5_F2_POS.txt                          E12-5_M4_NEG.txt
E12-5_F5_NEG-Amended-Amended-Amended.txt  E12-5_M4_POS-Amended-Amended-Amended.txt
E12-5_F5_NEG-Amended-Amended.txt          E12-5_M4_POS-Amended-Amended.txt
E12-5_F5_NEG-Amended.txt                  E12-5_M4_POS-Amended.txt
E12-5_F5_NEG.txt                          E12-5_M4_POS.txt
E12-5_F5_POS-Amended-Amended-Amended.txt  E12-5_M7_NEG-Amended-Amended-Amended.txt
E12-5_F5_POS-Amended-Amended.txt          E12-5_M7_NEG-Amended-Amended.txt
E12-5_F5_POS-Amended.txt                  E12-5_M7_NEG-Amended.txt
E12-5_F5_POS.txt                          E12-5_M7_NEG.txt
E12-5_M3_NEG-Amended-Amended-Amended.txt  E12-5_M7_POS-Amended-Amended-Amended.txt
E12-5_M3_NEG-Amended-Amended.txt          E12-5_M7_POS-Amended-Amended.txt
E12-5_M3_NEG-Amended.txt                  E12-5_M7_POS-Amended.txt
E12-5_M3_NEG.txt                          E12-5_M7_POS.txt

Many thanks, Adam

share|improve this question

2 Answers 2

this script outputs three files for each original ... Much obliged if anyone could explain why this happens.

Since E12-5_F2_NEG-Amended.txt ends in .txt it will be picked up by your script next time you run it.

The triple results indicate you ran your script three times whilst debugging it.

If the script output to $f.new rather than to ${f%.txt}-Amended.txt, you wouldn't have this problem.

Alternatively put rm *Amended.txt at the start of the program. If you have a very large number of files an a directory this can be slow on older Unix variants.

Another option is to output the files into a subdirectory (so something like "new/${f%.txt}.Amended.txt")

share|improve this answer

You can do what your script is trying to achieve in a single line:

head --lines=-5 input.txt > output.txt

In a for loop:

for f in *.txt; do head --lines=-5 "$f" > "${f%.txt}-Amended.txt"; done

You can use -n -5 instead of --lines=-5 to save on typing if you want.

As RedGrittyBrick points out, the reason you have three files per input is probably because you ran the script multiple times, and since the outputs end with .txt, they were picked up by the *.txt glob of the successive scripts.

Now I will critique your specific script.

noOfRows=$(cat $f | wc -l)

This is a truly useless use of cat; rather than cat $f | wc -l, use wc -l "$f". It's probably not so important in this particular script, but it's good not to develop bad habits. Speaking about bad habits: Always quote variables, e.g. "$f". This will make sure the file name is treated as a single argument even if it contains whitespace.

relevantRows=$(expr $noOfRows - 5)

There's nothing actually wrong here, though I would generally prefer to use something like

relevantRows=$((noOfRows-5))

AFAIK there's no performance difference between the two, but I find the bash way more visually pleasing; and more importantly, the way I described is defined in POSIX, and is thus more portable. Within bash only (so don't use this if you'll need to port the script to a different shell), the best way to do this in a script would be to use let:

let noOfRows-=5

...which would subtract 5 from the number contained in the variable $noOfRows, meaning that there's no need to create the variable $relevantRows.

head -n $relevantRows $f | tee ${f%.txt}-Amended.txt

This is the correct thing to do if you want to have the output displayed on the command-line as well as putting it into your output file. Otherwise, just use > to redirect stdout to a file.

share|improve this answer
1  
The $(()) way is POSIX, and it's actually the let method that is a non-standard Bash addition. expr is also less defined in function than the POSIX $(()) construction, so the latter should be preferred if portability is needed (which often is a good guideline for well formed scripts and avoiding bad habits). See e.g. wiki.bash-hackers.org/commands/builtin/… . –  Daniel Andersson Jan 27 '13 at 15:39
    
@Daniel thank you for that info, incorporated into the answer –  evilsoup Jan 27 '13 at 16:18
    
Thanks for your help - and for the critique! –  user193170 Jan 27 '13 at 16:41
    
Excellent critique, and I agree with every bit of it, but you omitted actually answering the original question (“why is there triplicate output”). See @RedGrittyBrick’s answer for that. –  kopischke Jan 28 '13 at 15:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.