-1

I have 5000 txt files with the different data below that. I need a shell script to copy only line 11 (last line) of them into a single file and sort them from smallest to largest.

for example:

file1.txt

1KE5.pdb
USER_CHARGES
INVALID_CHARGES
@<TRIPOS>ATOM
ATOM      1  N   MET A   1   40.880  54.110  11.190  1.00  0.00
ATOM      8  HB1 MET A   1   38.760  53.510   9.880  1.00  0.00
ATOM      9  HB2 MET A   1   39.700  52.020   9.980  1.00  0.00
1            40.7               

file2.txt

1KW5.pdb                     
USER_CHARGES                 
INVALID_CHARGES              
@<TRIPOS>ATOM                
ATOM      6  HA  MET A   1   39.020  54.080  12.120  1.00  0.00
ATOM      7  CB  MET A   1   39.050  52.700  10.580  1.00  0.00
ATOM      8  HB1 MET A   1   38.760  53.510   9.880  1.00  0.00
2            33.7               

file3.txt

1KW5.pdb                     
USER_CHARGES                 
INVALID_CHARGES              
@<TRIPOS>ATOM                
ATOM      4  H3  MET A   1   40.580  54.900  10.580  1.00  0.00
ATOM      5  CA  MET A   1   39.750  53.360  11.780  1.00  0.00
ATOM      6  HA  MET A   1   39.020  54.080  12.120  1.00  0.00
3            54.2               

I need the output to be:

Final.txt

1       40.7
2       33.7
3       54.2

I need a script to resolve this problem. Thank you for any help you can give.

  • 1
    if the line you want is always last, you can always tail -n 1 filename >> output.txt each file. wrap it in a find command that knows how to select your 5000 files, and it should be pretty easy. – Frank Thomas Jul 13 '16 at 12:59
  • 2
    Please note that Super User is not a script writing service. If you tell us what you have tried so far (including any scripts you are using) and where you are stuck then we can try to help with specific problems. You should also read How do I ask a good question?. – DavidPostill Jul 13 '16 at 14:07
0

You can use find to get the input files, tail to get the end of the files, and sort to sort them. For example:

find /path/to/ -name "file*.txt" -type f -exec tail -n1 {} \; | sort --numeric-sort --output output.txt

Where /path/to/ is the path to your files, file*.txt is the pattern of your file names, -type f finds only files, not directories, -n1 tells to tail to return 1 line, and output.txt is the output file.

  • Thanks for everybody help, this works but the values start from column 7, with decimal after numbers (e.g. 1.000 2.000 3.0000) in the output file. I want to be written values from first col without zero digits. – H. aryapour Jul 14 '16 at 13:03
2

Depending on which you think is more consistent, the 11th line, or the last line, you could use either sed or tail, respectively. I would prefer sed as it handle multiple files more cleanly, and also ensures that files with fewer lines are included. So, then, you would just need a list of your files. This could be done with globbing, if they are all in the same directory, or with find if they are nested below that.

So, with find and sed:

find /path/to/your/files -type f -name '*.txt' -exec sed -n 11p {} + >> output.txt

The same with tail, note the use of \; instead of + so that each file is handled separately:

find /path/to/your/files -type f -name '*.txt' -exec tail -n1 {} \; >> output.txt

Or perhaps globbing:

sed -n 11p /path/to/your/files/*.txt >> output.txt
0

** I need a shell script to copy only ... last line of them into a single file and sort them from smallest to largest.

Since your example files don't have a line 11, I have worked on the basis of your stated "(last line)".

  • multiple text files with a common naming pattern.
  • last line
  • sort
  • place in a file

To gather the last lines and sort them

$ tail -q -n 1 file*.txt | sort
1            40.7
2            33.7
3            54.2

and, to sort the results and put the results in a file

$ tail -q -n 1 file*.txt | sort > final.txt
$ cat final.txt
1            40.7
2            33.7
3            54.2

This assumes the files are all in one directory.

If the files are in subdirectories one-level deep, change file*.txt to */file*.txt

If the files are in multiple nested directories of varying depths, or otherwise in multiple locations, you should use the find command as described in other answers.

$ find . -name 'file*.txt' -exec tail -q -n 1 {} \; | sort
1            40.7
2            33.7
3            54.2

where the file names can include spaces, which used to be a problem.

$ ls -F *.txt
file1.txt  file2.txt  file three.txt  final.txt

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.