The spider tag has no wiki summary.
0
votes
1answer
111 views
Web crawler - scan and map website and log all links that have “particular-string” in them
Is there a tool that would scan a website and log all links that have a particular string in them?
I see Teleport Pro, but it seems like it does make a copy of the site, but I only need links.
0
votes
0answers
50 views
Looking to backup content from online class [closed]
Possible Duplicate:
How can I download an entire website?
Basically, I took an online class, and now I want to spider the content, and back it up locally. Any Suggestions on ...
1
vote
2answers
180 views
Web spider crawler - recommendations needed
I'm looking for a robust web spider/web crawler to crawl webpages in the background.
It needs to run in the background and spider pages continuously.
It needs to be fast and memory efficient.
can ...
0
votes
0answers
99 views
web spidering/crawling, can i do it or just search engines?
i already had a question answered about web-scraping with wget. but as i read a little more, i realize i may be looking for a web-crawling program. particularly the part about web-crawlers being able ...
1
vote
1answer
143 views
What is the best way to archive (spider) a site that is going to be removed? [closed]
Possible Duplicate:
How can I download an entire website
Three different blogs that I read have recently announced that they are going to be discontinued and removed from the web. Although ...
0
votes
1answer
189 views
wget: Turn Off Forced .html Retreival
When performing a recursive download, I specify a pattern via the -R parameter for wget to reject, but if this file is a HTML file, wget downloads the file regardless of whether or not it matches the ...
2
votes
2answers
109 views
Tool to recursivly convert a HMTL file to PDF?
Are there any tools which not only convert a HTML file to PDF but also follow links, so that in the end I get 1(!) PDF file which contains all html files?
0
votes
1answer
62 views
Spider a website with variations in the POST parameters
I need an online or offline app that can take a URL, vary the input value for one parameter according to rules set by the user, and then download each varied url it gets.
For example, if I give it ...
1
vote
1answer
180 views
Looking for web spider/download program which can use existing browser cookies and can process Javascript
I'm looking for a program to spider a web site* and download the content to disk. I've got the following requirements though which appear to trip up the programs I've tried:
The site's logon ...
