I am parsing text on the command line. How can I take the output of one command and use it in two places in the next command?

some_command | head -n $( awk '/search_term/ {print FNR} output_of_some_command ) output_of_some_command

I am truncating the output of some_command using head, but I need to search the output of some_command (using awk in this case) to find the line number at which to truncate.

Can I do this, or do I need to write to a temporary file?

  • Thinking this through I could use sed instead of head to truncate the text... but I still wonder if it could be done this way. – Ashley G Oct 15 '14 at 22:07

For this particular task, you could use sed instead of head – or you could use awk instead of head:

some_command | awk '{print} /search_term/{exit}'

Print lines, but, when you find one that matches the search term, exit (and don’t process any more lines after that).

A more general solution to the problem you describe is

some_command | tee >(command2) | command3


some_command | tee >(command2) >(command3)

These will work in bash, but probably not too many other shells (including sh).  >(command) gives you a file handle to a pipe to a process running command.  As you probably know, tee lets you take a data stream and write it multiple places.

But your problem is subtlety more complicated than what you said.  You’re not trying to do

                                          A→B and A→C in parallel

you’re trying to do

                A→B→C and A→C in parallel

where the head command (corresponding to C in the diagram) takes input from “some_command” (A) but also has a command-line argument set from the output of awk (B).  I suspect that a general solution for that would be much trickier.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.