The spider tag has no wiki summary.
0
votes
0answers
48 views
Looking to backup content from online class [closed]
Possible Duplicate:
How can I download an entire website?
Basically, I took an online class, and now I want to spider the content, and back it up locally. Any Suggestions on ...
1
vote
2answers
133 views
Web spider crawler - recommendations needed
I'm looking for a robust web spider/web crawler to crawl webpages in the background.
It needs to run in the background and spider pages continuously.
It needs to be fast and memory efficient.
can ...
0
votes
0answers
77 views
web spidering/crawling, can i do it or just search engines?
i already had a question answered about web-scraping with wget. but as i read a little more, i realize i may be looking for a web-crawling program. particularly the part about web-crawlers being able ...
0
votes
2answers
99 views
How does developers get your search criteria direct in google [closed]
Sometimes when I do a search for specific content, my exact search phrase will be visible in Google on some site which does not contain any information. It looks like the hack in to your search query ...
1
vote
1answer
111 views
What is the best way to archive (spider) a site that is going to be removed? [closed]
Possible Duplicate:
How can I download an entire website
Three different blogs that I read have recently announced that they are going to be discontinued and removed from the web. Although ...
0
votes
1answer
168 views
wget: Turn Off Forced .html Retreival
When performing a recursive download, I specify a pattern via the -R parameter for wget to reject, but if this file is a HTML file, wget downloads the file regardless of whether or not it matches the ...
2
votes
2answers
86 views
Tool to recursivly convert a HMTL file to PDF?
Are there any tools which not only convert a HTML file to PDF but also follow links, so that in the end I get 1(!) PDF file which contains all html files?
0
votes
1answer
29 views
Spider a website with variations in the POST parameters
I need an online or offline app that can take a URL, vary the input value for one parameter according to rules set by the user, and then download each varied url it gets.
For example, if I give it ...
1
vote
1answer
154 views
Looking for web spider/download program which can use existing browser cookies and can process Javascript
I'm looking for a program to spider a web site* and download the content to disk. I've got the following requirements though which appear to trip up the programs I've tried:
The site's logon ...
