0

Im trying to use either XARGS or FOR in a single line to accept a list of URLS as arguments, this fails since a file name can't be a URLs. We tried doing this with FOR and XARGS, but it fails:

$for i in $(cat example.txt);do echo $i > $i; done
bash: http://example.site.com: No such file or directory
bash: https://secureexample.com: No such file or directory

$cat example.txt | xargs -I {} sh -c "echo  {} > {}"
sh: 1: cannot create http://example.site.com: Directory nonexistent
sh: 1: cannot create https://secureexample.com: Directory nonexistent

is it possible to remove the forward slashes from the argument in the second occurrence(name of the output file) so that ill get a distinctive file per host, for example:

http.example.site.com
https.secureexample.com

My knowledge in single-line bash is pretty limited, sadly, that's the company's preference. i tried using curly braces on the filename and change the value with SED, AWK and even CUT, but as i said, my knowledge is too limited. I also tried multiple arguments with XARGS and FOR but i also failed spectacularly to make that work.

I was able to circumvent the issue by creating separate files with URIs instead of URLs, and then supplement http:// and https://, but that means double the files and double the commands, example:

$cat http_example.txt
example.site.com
secureexample.com 
$cat http_example.txt | xargs -I {} sh -c "echo  http://{} > {}"
$ls -la
-rw-r--r-- 1 root root   24 Mar 22 08:06 example.site.com
-rw-r--r-- 1 root root   25 Mar 22 08:06 secureexample.com

Is there any way to make this work in a single line bash command whilst using a file with URLS? thank you!

4
  • Difficult to understand what is your question. Can you make it clearer? – user596332 Mar 22 '20 at 13:14
  • I edited it, i hope that makes it clearer. i may need to edit the title as well, but i can't figure out a way to explain it properly. – Giladiald Mar 22 '20 at 13:30
  • Yes, I think I get it now. Do you really need the leading http(s). in the resulting file name? Or would example.site.com be enough? – user596332 Mar 22 '20 at 13:33
  • Yes, i can do it without (see the circumvent portion). All of the tools we use in my company need complete URLS. – Giladiald Mar 22 '20 at 13:42
2
while read -r i;do echo "$i">"${i#http*://}";done<example.txt

This removes the leading http(s):// part, creating example.site.com and secureexample.com files.

The ${i#http*://} is a construct that removes from $i the leading part matching http*:// (* is a wildcard).


while read -r i;do echo "$i">"$(echo "$i"|sed 's|://|.|')";done<example.txt

This one creates files http.example.site.com and https.secureexample.com.

The sed is replacing :// of the input file lines with ..

5
  • It works wonderfully, my machine was acting up. it works very well, thank you so much. – Giladiald Mar 22 '20 at 14:53
  • i see what went wrong, if there is a / at the end of the URL im getting errors, is this fixable? even if not that's fine ofcourse – Giladiald Mar 22 '20 at 15:12
  • 1
    Yes! In the last command, just change the second $i${i%/} (right before sed). – user596332 Mar 22 '20 at 15:25
  • First of all, thank you. you can stop answering whenever you want, you've done more than enough. i need just a tiny bit more help with the SED part. it doesn't work with hosts that reside on other ports, so website.com:8080 wont work. again, fill free to stop answering you have done more than enough – Giladiald Mar 22 '20 at 15:43
  • Glad to help! What exactly do you mean with doesn't work? It creates the file website.com:8080, you want it without :8080? – user596332 Mar 22 '20 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.