I've got a program that can take input from stdin and produces output in the stdout like this:

cat initial_input.txt | myprogram myindex myoptions - | filter_output > outfile.txt

The program loads a data structure (myindex) before starting to process the input, and this can take quite a while. Part of the output is amenable to be fed back to the program. I want to pipe part of the stdout back into the stdin that the program takes, so that it is appended at the end of the initial input the program was called with, without interrupting the initial program execution, since it takes time to initially load myindex into it.

Is there a way to do that?

1 Answer 1


You might be able to accomplish this with tail:

tail -f initial_input.txt -n 100000 | myprogram myindex myoptions - | filter_output >> initial_input.txt

The important bits here are:

  • tail; This works just like cat, but (with the -f option) it "follows" the end of the file, so as the file grows, it dumps more data to stdout. The -n option specifies how many lines to dump initially (by default I think it only does 10), so you'll need that number to be at least as large as the original number of lines in the file.
  • ; Notice at the end I use >> instead of > as in your question. This tells the shell to append to, rather than over-write the existing file.

Please be aware that this could potentially be a "dangerous" thing to do, if you're not in a tightly controlled environment. And what I mean by "tightly controlled" is that you know that nobody else will be reading from or writing to your initial_input.txt file. If more than one process is trying to write to that file at once, you'll get garbage from it. But it sounds like you're probably working on files that only you ever modify, and you put them there yourself, so you ought to be safe.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .