Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to download about 100 packages so I'm using wget-list to make it easier. My question however, is once I've made the list (I assume it's in a .txt format), is there a way I can insert comments into it that wget will ignore? Something like this:

#This is a comment
http://someurl.com
http://anotherurl.com
share|improve this question
up vote 1 down vote accepted

It doesn't look like it:

If --force-html is not specified, then file should consist of a series of URLs, one per line.

You could try HTML style comments: <!-- Comment --> - maybe those get interpreted as comments, although I wouldn't count on it.

You could also use the --force-html parameter and feed it HTML - a format in which you'd have all freedom to comment as much as you like. The downside is that it adds a lot of clutter:

<!-- This is a comment -->
<a href="http://someurl.com"></a>
<a href="http://anotherurl.com"></a>
share|improve this answer
    
Yeah HTML makes it way too messy. Thanks for the clarification. – n0pe Apr 2 '11 at 13:01

Just put comments in your without notation, wget will simply pick them up as invalid URLs.

share|improve this answer

You can pipe through grep or sed to remove comments:

grep -v '^#' ~/list.wget | wget -i- -c -B http://base.url.if_needed
share|improve this answer

I tested wget using the list available at https://en.wikipedia.org/wiki/Comparison_of_programming_languages_%28syntax%29#Inline_comments

I discovered that wget does not support a comment character. However, the following generate quick "Invalid URL" errors:

:  Test comment 1   list: Invalid URL :  Test comment 1: Scheme missing
:: Test comment 2   list: Invalid URL :: Test comment 2: Scheme missing
#  Test comment 3   list: Invalid URL http://#  Test comment 3: Invalid host name
// Test comment 4   list: Invalid URL // Test comment 4: Scheme missing

These were not listed on the Wikipedia article but also cause quick "Invalid URL" errors

/ Test comment 1    list: Invalid URL / Test comment 1: Scheme missing
[ Test comment 1    list: Invalid URL http://[ Test comment 1: Unterminated IPv6 numeric address
@ Test comment 1    list: Invalid URL http://@ Test comment 1: Invalid user name
? Test comment 1    list: Invalid URL http://? Test comment 1: Invalid host name

The remaining comment lead-in character strings all caused wget to attempt to resolve a domain name using DNS resulting in at least eight lines of error output.

I also discovered that wget scans the entire list file and builds a list of URLs to fetch before it starts fetching. For example if you have a list file containing:

# test comment 1
# test comment 2
http://superuser.com/questions/265711/commenting-in-a-wget-list

# test comment 3
# test comment 4
# test comment 5
# test comment 6

The wget output is:

list: Invalid URL http://# test comment 1: Invalid host name
list: Invalid URL http://# test comment 2: Invalid host name
list: Invalid URL http://# test comment 3: Invalid host name
list: Invalid URL http://# test comment 4: Invalid host name
list: Invalid URL http://# test comment 5: Invalid host name
list: Invalid URL http://# test comment 6: Invalid host name
--2015-08-19 14:03:55--  http://superuser.com/questions/265711/commenting-in-a-wget-list
Resolving superuser.com (superuser.com)... 190.93.247.58, 190.93.244.58, 141.101.114.59, ...
Connecting to superuser.com (superuser.com)|190.93.247.58|:80... connected.
HTTP request sent, awaiting response... 200 OK
<snip>

Thus, while : :: # / // [ @ ? can all safely be used as comment lead-in characters the resulting errors will be output first and will not be in-line with the wget attempts to fetch pages.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.