Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to pass curl the output from awk

./ | grep 'resource' | awk '{print $4}' | curl OUTPUT_FROM_AWK | grep myString

How can I achieve this?!

share|improve this question
up vote 3 down vote accepted

Use xargs.

xargs utility [argument ...]

The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments.

There are more parameters and options than in this shortened form, of course.

A general example using curl:

$ echo "" | xargs curl
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<H1>302 Moved</H1>
The document has moved
<A HREF="">here</A>.

In your specific case, it'd look similar to the following:

./ | grep 'resource' | awk '{print $4}' | xargs curl | grep myString
share|improve this answer
Great! However in my case this doesn't work. If piping to "xargs echo" it also doesn't work. I guess output of jspider is too fast or sth.. – Stephan Kristyn Dec 2 '11 at 17:13
In that case, consider adding to your question or creating a new question dealing with your specific problem. I don't have this specific issue, e.g. curl -s "" | grep -E 'href="http://.*stackexchange\.com' | sed 's|^.*<a href="http://\([^"]*\)">.*$|http://\1|g' | grep -v "<" | xargs curl -s | grep "<title>" works fine for me. (Yeah I know the code's extremely hacky). – Daniel Beck Dec 2 '11 at 18:10
Oh my.. I have to de-cypher that sed part in order to understand what you are doing, still it isn't working with jspider. I will start using perl now. – Stephan Kristyn Dec 5 '11 at 9:39
@StephanKristyn I extract URLs from hyperlinks, and since it doesn't work well enough, using grep -v I then remove all remaining lines with HTML tag brackets. Just a simple example that shows the approach works in general. – Daniel Beck Dec 5 '11 at 9:51

Try this, it's untested but should work.

for a in $(./ | grep 'resource' | awk '{print $4}'); do curl $a | grep myString; done
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.