Bash one-liner to read stdin and print last 10 matches each for 2 separate patterns, without using tmp files / awk / perl / etc

Question

EDIT: Generally speaking, I'm asking for an elegant way to duplicate an input stream, process it with 2 or more sets of commands, then merge the output of these commands. Emphasis on elegant.

Real-world use case: In a CI pipeline, a bash script is doing git fetch --tags which outputs a long list of tags. The tags all look like desktop-... or mobile-.... We don't really need to see tags from months or years ago, so I'd like to avoid printing the whole list, and instead print just the last 10 (each) desktop and mobile tags. I can imagine a scenario where there would be more than 2 platforms, so in general case we would like to be able to filter an arbitrary number of groups of 10.

Rules:

• Has to be a neat and elegant bash one-liner, not a script (use some advanced bash features or (more or less common) helper utilities to reduce the amount of code)
• Should not use temporary files or something else that requires cleanup
• No piping to awk, perl or similar
• No complicated bash logic (e.g. counter variables and a while read loop)
• Has to work both in the terminal and when detached from a tty (like in the CI env)
• Has to match at least 2 patterns, and print 10 last results for each
• The results for each pattern should not be intermingled, but let's say this requirement is loose, as long as the solution allows to | sort -V the complete output as the last step
• Bonus points if the solution can be extended to 3+ patterns

For simplicity of testing, I deleted all the local tags, dumped the output of git fetch --tags to a file called tags, and am just using cat tags during my attempts. To make it easier for you to test possible solutions, here's a simple script that generates a reasonably looking mock input:

for ((i=0; i<100; ++i)); do echo "$((1+RANDOM/5000)).0.$((RANDOM/5000))"; done | sort -V | while read v; do [[ $((RANDOM%2)) == 1 ]] && echo -n "mobile-" || echo -n "desktop-"; echo $v; done >tags

Note: The solution shouldn't refer to the file directly. If it was ok, you could just grep it twice or more to get 10 matches of each pattern. But in the use case I described the command git fetch --tags runs only once (even if it was run more than once, it would output nothing on the second run, since the tags would already be fetched by the first run). So it's only allowed to use this input file once like so: cat tags. This is intended to emulate the real-world use case described above.

My best attempt so far:

cat tags | tee >(grep '\<desktop-' | tail -n 10) | grep '\<mobile-' | tail -n 10

The problem with the "desktop" process substitution is: its stdout is null, so the output is lost. All the examples of process substitution that I've seen redirect the output such commands to a file. I feel like there's gotta be some nice way to "merge" such output back into a single stream, but I couldn't find this way so far.

Solutions that produce the desired output but don't meet my rules for being neat and elegant (both of them are a bit long:

• Some typical imperative programming, the kind of code you would write if you were using a "real" programming language. It's not in the spirit of UNIX shell scripting, which is using pipes and standard commands, each doing one job (and do it well) to achieve the desired end result.

d=0; m=0; tac tags | while read l; do [[ $d -lt 10 && $l =~ ^desktop- ]] && { echo $l; let ++d; }; [[ $m -lt 10 && $l =~ ^mobile- ]] && { echo $l; let ++m; } done | sort -V

• Uses pipes and bash's process substituion, but uses a temporary file. "Proper" usage of temp files generally involves mktemp, cleaning up afterwards, handling abnormal script termination (using trap to do cleanup on SIGHUP/SIGINT).

cat tags | tee >(grep '\<mobile-' | tail -n 10 >mobile-tags) | grep '\<desktop-' | tail -n 10 && cat mobile-tags && rm -f mobile-tags

Answer 1

Ok, got it this morning, plumbing:

the idea is to dup()licate the output on both STDERR and STDOUT with tee and parse it in //:

Here we go, could be a one liner as well, on multiline for readability:

git fetch --tags | tee /dev/stderr 2> >(
grep -w mobile   | tail -n10) 1> >(
grep -w desktop  | tail -n10) 

Or with the help or your brilliant comments (kind of brainstorming):

git fetch --tags | tee  /dev/fd/{10,11,12} 10> >(
grep -w desktop  | tail -n 3) 11> >(
grep -w mobile   | tail -n 3) 12> >(
grep -w foobar   | tail -n 3) 1>/dev/null

Could be simplified as:

git fetch --tags | tee >(
grep -w mobile   | tail -n3) >(
grep -w desktop  | tail -n3) >(
grep -w foobar   | tail -n3) > /dev/null

Related: http://mywiki.wooledge.org/ProcessSubstitution

Answer 2

Using grep and the good switches:

$ grep -Ewm20 'desktop|mobile' tags | sort
desktop-10.5.0
desktop-11.5.0
desktop-12.6.0
desktop-13.6.0
desktop-14.7.0
desktop-15.7.0
desktop-16.8.0
desktop-17.8.0
desktop-18.9.0
desktop-19.9.0
mobile-15.6.0
mobile-16.6.0
mobile-17.7.0
mobile-18.7.0
mobile-19.8.0
mobile-20.8.0
mobile-21.9.0
mobile-22.9.0
mobile-23.10.0
mobile-24.10.0

Answer 3

What about this one liner?

d=( $(grep desktop tags) ) m=( $(grep mobile tags) ); printf '%s\n' "${d[@]:0:10}" "${m[@]:0:10}" | sort -rV