Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm using wget --mirror --html-extension --convert-links to mirror a site, but I end up with lots of filenames in the format post.php?id=#.html. When I try to view these in a browser it fails, because the browser ignores the query string when loading the file. Is there any way to replace the ? character in the filenames with something else?


The answer of --restrict-file-names=windows worked correctly. In conjunction with the flags --convert-links and --adjust-extension/-E (formerly named --html-extension, which also works but is deprecated) it produces a mirror that behaves as expected.

wget  --mirror --adjust-extension --convert-links --restrict-file-names=windows http://www.example
share|improve this question
up vote 12 down vote accepted

See the --restrict-file-names option. While not exactly intended for this particular purpose, --restrict-file-names=windows will probably help you along:

--restrict-file-names=modes

Change which characters found in remote URLs must be escaped during generation of local filenames. [...]

When "windows" is given, Wget escapes the characters \, |, /, :, ?, ", *, <, >, and the control characters in the ranges 0--31 and 128--159. In addition to this, Wget in Windows mode uses + instead of : to separate host and port in local file names, and uses @ instead of ? to separate the query portion of the file name from the rest. Therefore, a URL that would be saved as www.xemacs.org:4300/search.pl?input=blah in Unix mode would be saved as www.xemacs.org+4300/search.pl@input=blah in Windows mode.

share|improve this answer

Your browser will view it fine if you use an URL like

file:///tmp/example.com/post.php%3Fid=1.html

instead of

file:///tmp/example.com/post.php?id=1.html

Note: if you're having trouble with internal links from downloaded files, it'd be because you terminated wget before it was done with the downloading. Since you specified --convert-links and --html-extension (only applies when those are given), wget would normally fix the links to use %3F instead of ?; however, it does this at the end, after it's finished downloading; if it has been interrupted, it will not have fixed any of the links, and you're left in this predicament. Of course, you can always write a script to go through and fix the links, but...

share|improve this answer
    
this is not quite correct, wget, with no interruption, for a URL like http://site.com/article.cgi?25 will be saved as article.cgi?25.html see section 2.7 at gnu.org/software/wget/manual/wget.html#HTTP-Options – Tzury Bar Yochay Jul 31 '12 at 7:22

wget does not have an option to modify the saved name. What you'll probably need to do is create a script to go through and replace ? with _ or something similar. Wget alone cannot do this.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .