Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want netcat to pipe the output of a tcp/ip connection to commands that examine the output and split it into distinct files separated by the strings "msg" and "/msg" from the output, something like

nc host port | while :
do
split 'msg' '/msg' > timestamp.txt
done

but I can't get the syntax right. In an ideal world, I'd also like the whole of the output of nc appending to another file, say big_data.txt. Maybe I should use grep?

I know how to send nc output to a file and can work out how to split it, but the file will keep appending for 12 hours each day. Also, I think it's more efficient to split the data as it arrives.

share|improve this question

I did it all in R.

    stream_processor <- function(host_,port)
     con2 <- socketConnection(host=host_, port=port_)
    # as non-blocking, may need to loop for input
    readLines(con2)
    while(isIncomplete(con2)) {
        Sys.sleep(1)
        z <- unlist(readLines(con2,1000))
     write(z, file=paste(make.names(date()),'.stream',sep=''))
  }
}
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .