There is -p
/--page-requisites
parameter (with -r
together) which would download the most external URLs necessary to properly display a given HTML page (unless they're excluded in robots file).
More about this can be read in the manual (man wget
):
It's worth knowing that Wget's idea of an external document link is any URL specified in an <A>
tag, an <AREA>
tag, or a <LINK>
tag other than <LINK REL="stylesheet">
.
Since Wget does not ordinarily distinguish between external and inlined documents, one is generally left with "leaf documents" that are missing their requisites.
Please note that Wget has an internal table of HTML tags/attribute pairs that it considers when looking for linked documents during a recursive retrieval. To add one, try using --follow-tags=list
(comma-separated list), opposite of --ignore-tags=list
.
This list of tags is probably defined in html-url.c
and it goes as follow:
/* For tags handled by tag_find_urls: attributes that contain URLs to
download. */
static struct {
int tagid;
const char *attr_name;
int flags;
} tag_url_attributes[] = {
{ TAG_A, "href", ATTR_HTML },
{ TAG_APPLET, "code", ATTR_INLINE },
{ TAG_AREA, "href", ATTR_HTML },
{ TAG_BGSOUND, "src", ATTR_INLINE },
{ TAG_BODY, "background", ATTR_INLINE },
{ TAG_EMBED, "href", ATTR_HTML },
{ TAG_EMBED, "src", ATTR_INLINE | ATTR_HTML },
{ TAG_FIG, "src", ATTR_INLINE },
{ TAG_FRAME, "src", ATTR_INLINE | ATTR_HTML },
{ TAG_IFRAME, "src", ATTR_INLINE | ATTR_HTML },
{ TAG_IMG, "href", ATTR_INLINE },
{ TAG_IMG, "lowsrc", ATTR_INLINE },
{ TAG_IMG, "src", ATTR_INLINE },
{ TAG_INPUT, "src", ATTR_INLINE },
{ TAG_LAYER, "src", ATTR_INLINE | ATTR_HTML },
{ TAG_OBJECT, "data", ATTR_INLINE },
{ TAG_OVERLAY, "src", ATTR_INLINE | ATTR_HTML },
{ TAG_SCRIPT, "src", ATTR_INLINE },
{ TAG_TABLE, "background", ATTR_INLINE },
{ TAG_TD, "background", ATTR_INLINE },
{ TAG_TH, "background", ATTR_INLINE }
};