Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a file that has lines that look like this:

"http://api.domain.com/path/to/resource?format=json" > output.xml

and I execute the following from my command line:

cat file | xargs -L 1 -I {} wget {}

Unhappily, the command looks lik it executes, but no file ever gets created. When I use the command:

cat file | xargs -L 1 -I {} echo {}

It prints the lines of the file without the double quotes. I think this may be the source of the problem. How do I get it to include them?

How can correctly I execute the wget command?

share|improve this question
    
The initial command executes, but it won't really do what you want. As the whole line is passed as one argument to wget (or echo, for that matter), your redirection operator is just "lost" inside this one argument. The quotes are not the initial problem here. The question is how to solve it in an elegant way. –  slhck Feb 13 '12 at 21:02
    
In your case, why not prepend each line with wget and execute it? –  slhck Feb 13 '12 at 21:03
1  
Are you sure wget is the tool you are looking for here? Did you mean to include the -O option to specify the output file? Or maybe you want curl? –  Zoredache Feb 13 '12 at 21:31
add comment

1 Answer 1

up vote 1 down vote accepted

Quotes, redirection operators, etc. are typically parsed by your shell before the argv[] array is constructed. > is stripped away; the shell performs redirection. Quotes are stripped away; the shell performs word splitting.

With xargs, the entire input line goes directly to argv[], with the quotes and the operators being passed to wget, which has no idea what to do with them.

To parse such input files as you have, you'll need to get help from the shell, in two possible ways.


< file xargs -d '\n' -I {} sh -x -c "wget {}"

This way, file is being read line by line without allowing xargs to do any splitting (-d '\n') and passed to the sh shell as part of a command line. The -x option to sh lets you see commands it executes.


while read -r line; do
    eval "wget $line"
done < file

Here the same is done using only builtins of the current shell – either sh, bash or compatible.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.