Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Is there already a program that reads multiple pipes or file descriptors and writes to the standard output (not splitting lines).

Like cat, but reading all files simultaneously and preserving lines.

Something like:

until(all fds are EOF) {
    select(multiple fds);
    read a line from triggered fd;
    write the line to the stdout;
}

It is needed to avoid coding of select/epoll loops or using multithreading in simple programs. Like "select loop for bash".

Update: I've also created a similar question on unix.SE.

share|improve this question
    
Please describe what you're trying to accomplish. This almost sounds like an XY problem. –  Dennis Williamson Feb 23 '11 at 18:33
    
@Dennis Williamson, Every time I want to write a oneliner that shares output the problem arises. Program with even loop is sometimes too big to simple one-shot tasks. –  Vi. Feb 23 '11 at 18:40
1  
Please be more specific. Can you post an example? –  Dennis Williamson Feb 23 '11 at 18:41
    
@Dennis Williamson, I have a stateful program that reads data from /dev/input/whatever and issues HTTP requests and starts other programs. And now I want it to handle multiple input devices (but don't want to think about FD loops or threading). I want solution that can be applied to any of such problems. Like pipe demultiplexor/multiplexor. –  Vi. Feb 23 '11 at 18:46

3 Answers 3

A alternative using xargs(1) and tail(1):

Create the data sources:

$>touch datafile.txt
$>mkfifo pipe1
$>mkfifo pipe2

Start tail processes with xargs:

$>echo datafile.txt pipe1 pipe2 | xargs -n 1 -P 0 tail -f >> datasink.txt

Push data into sources:

$>echo dataset1 > pipe1
$>echo dataset2 > datafile.txt
$>echo dataset3 > pipe1
$>echo dataset4 > pipe2

Show data from sink:

$>cat datasink.txt 
dataset1
dataset2
dataset3
dataset4
share|improve this answer
up vote 0 down vote accepted

Created a program to do just it: fdlinecombine. It reads specified file descriptors and outputs the data linewise.

Checked some corner cases, tested a bit. Hoping it will be useful for somebody too.

share|improve this answer

A partial solution could be to use:

tail -fq file1 file2 file3 | grep -v ^$ | yourprogram

I name this a partial solution because the grep removes the empty line that tail adds every times it changes file. If your input contains empty lines, they are removed.

share|improve this answer
    
Clever, a +1 from me. –  Nifle Feb 23 '11 at 20:14
    
Fails: mkfifo pipe1 pipe2 && tail -fq pipe1 pipe2; then writing to pipe2 hangs. –  Vi. Feb 24 '11 at 10:38
    
tail -f on multiple pipes works in general bad unless it had read each of them at least once. –  Vi. Feb 24 '11 at 10:45
    
Also it is slow. –  Vi. Mar 1 '11 at 15:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.