I want to split and GZip a large file, and this answer appeared to be what I'm looking for, and it seemed like a very useful way of doing things I never thought of, so I'd like to generalize it; the only problem is: it doesn't appear to work.
Say I want to split my input and process it further (I know
split but I want to pipe it around in my script directly!)
read to read a line into a variable
#!/bin/bash printf " a \n b \n c \n d " | for ((i = 0 ; i < 2 ; i++)) ; do echo "<< $i >>" for ((j = 0 ; j < 2 ; j++)) ; do read l echo "$l" done done
<< 0 >> a b << 1 >> c d
Which is almost what I want, apart from the fact that it trims the spaces from the start and end (and maybe modifies the line in other ways? Will it work with arbitrary UTF-8 encoded content?) edit solved
And I imagine it might be quite slow. edit Benchmarked it: at least 3000x slower.
So I tried to pipe it through
head (I get the result with using
awk as the answer suggests, it doesn't appear to do anything differently)
#!/bin/bash printf " a \n b \n c \n d " | for ((i = 0 ; i < 2 ; i++)) ; do echo "<< $i >>" head -n 2 done
<< 0 >> a b << 1 >>
And stops because
head apparently closes its input on exit. I haven't found a program that doesn't do this, and maybe it's actually enforced by the system? (I'm on OS X)
head -n 2 <&0 which (according to the bash docs) copies the file descriptor first doesn't work either.
Do I have to use a named pipe? Is there some incantation to make this work?