I have a directory structure with 14 directories containing a bunch of files containing data in a three-column format (separated with tabs). I intended to use find and awk to extract the second column from each of those files and output it with the same filename but under a different root folder. Here the sketch of my directory.

data/all -> AA, AB, AC, AD ... (A* being folders containing the files with data stored in a 3-column format, e.g. AA100.txt, AA101.txt ...)

i want to have the modified (one-column) files with the same name, but all under a new root directory data/pos (as opposed to data/all/) -> AA, AB, AC, AD ... (again, each containing A*100.txt, A*101...)

My try was to use find -exec and to give it the awk command, but I'm having issues with outputting the file to the right place.

when being in data/all/

find * -type f -exec awk '{print$2}' '{}' > ../pos/'{}' \;

However {} as wildcard for the input file doesn't seem to work when outputting the file?

What am I doing wrong? (I'm on a ubuntu server btw)

3 Answers 3


You could try without find, if all you want is all the files. While in data/all/, run this:

for file in ./*; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done

If you want to cover the files in the whole hierarchy under /data/all, you can enable the globstar option if you are using bash (I believe this would "just work" on zsh) and then use ** to match all files:

shopt -s globstar
for file in ./**; do awk '{print$2}' "$file" > "../pos/$(basename $file)"; done
  • Using for */* did the trick for me though, the globstar option wasn't available for me.
    – conipo
    Dec 19, 2013 at 19:44

What am I doing wrong?

You are using the redirection > ../pos/'{}' as if it was handled by find or awk but redirections are handled by the shell. In your case it means that you can redirect only output of the whole find (not output of awk).

Note that you do not usually need to use wildcard like * for the starting path of find. Is the common way find . what you wanted to do or is there any reason for find *?


Here we will keep the flexibility of find in contrast to the solution by Jacobo de Vera. Run awk in a shell loop:

find . -type f -print0 |
  while read -r -d $'\0' x; do
    awk '{print $2}' "$x" > "../pos/$(basename "$x")"

The original way with -exec will be less efficient because for every file a shell will be launched in addition to awk and the multi-level escaping is pretty complicated here:

find . -type f -exec sh -c 'awk "{print \$2}" "{}" > "../pos/{}"' \;

There could also be an alternative solution of doing the redirection inside awk.

  • I used the wildcard * in order to avoid the avoid the ./ in the path (as it gave me an error message that it could not find the specified path, which was something like pos/./AA/AA101.txt -> I had assumed the error to be there, that's why I used the * instead)
    – conipo
    Dec 19, 2013 at 19:42

find /path -exec ls -l {} \; | awk '{print$1}'


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .