Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

This first pipeline works fine (printing "c"):

echo "a" | sed 's/a/b/' | sed 's/b/c/'

This one does not do what I expect (nothing gets printed when I feed an "a" into my fifo ):

mkfifo fifo;
cat fifo | sed 's/a/b/' | sed 's/b/c/' 

However, if I remove the second "sed" command from the latter pipeline, I do get a "b" printed. I think my understanding of pipes and redirects must be too simplistic. Can someone explain to me how to fix the 2nd case so that I can run two successive commands on the contents of the fifo?

(note this isn't a problem specific to fifo, the same behavior occurs with netcat too. I posted a similar question about netcat but got no answers)

share|improve this question
2  
Your code works for me. Maybe, you could spell out more detail on how you "feed an 'a'" into your fifo. –  John1024 Aug 4 at 6:34
    
As @John1024 said, WorksForMe. –  Bob Aug 4 at 6:35
    
Me too, no problem whatsoever. –  MariusMatutiae Aug 4 at 6:36
    
echo "a" >fifo –  Sridhar-Sarnobat Aug 4 at 6:36
    
It seems this is buffering related. What version of sed are you using? Mine is GNU sed version 4.2.1 –  Sridhar-Sarnobat Aug 4 at 6:40

1 Answer 1

up vote 8 down vote accepted

Use the -u (--unbuffered) option of sed, so it writes it's output, instead of collecting it first for a fast, large write later.

cat fifo | sed -u 's/a/b/' | sed -u 's/b/c/'

cat is always unbuffered.

If it's not about sed, but some program with output buffering, but without options to disable it, there are tools that can help to force buffering off - see man unbuffer and man stdbuf.

If it's a shell script, it should be unbuffered by default: when a command creates output in the script, and exits, the output is flushed. Otherwise, you use unbuffer inside the shell script.

For background, see this overview of stdio buffering.

share|improve this answer
    
Suppose instead of sed, I had my own custom script. What would I do in this case to avoid the buffering? Or should it be unbuffered by default? –  Sridhar-Sarnobat Aug 4 at 6:42
    
Just some more followup - if I wanted to use perl regular expressions with buffering, it would go something like nc -k -l 4458 | perl -ne 's{HTTP}{FTP}g; $|++;print $_' | perl -ne 's{FTP}{RTSP}g; $|++;print $_' –  Sridhar-Sarnobat Aug 5 at 4:14
    
Yes, what do you mean? Is there a problem, or is it an example? Oh, and you can use ssed -R if you like sed with perl regular expressions. –  Volker Siegel Aug 5 at 4:32
    
Sorry, that was not a question. I apologize for not being clear, it was just another example of the same concept. Unfortunately it's too late for me to edit that comment. I didn't know sed could understand Perl regex. Perhaps I should start using that instead from now on. –  Sridhar-Sarnobat Aug 5 at 4:35
    
Oh, no problem! –  Volker Siegel Aug 5 at 4:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.