30 reputation
9
bio website
location
age
visits member for 2 years, 4 months
seen Jun 16 '12 at 11:10

Feb
11
awarded  Popular Question
Dec
20
awarded  Popular Question
May
8
revised Wget - if / else download condition?
removed -i option / explanation
May
8
revised Wget - if / else download condition?
removed -i option / explanation
May
8
awarded  Commentator
May
8
comment Wget - if / else download condition?
to keep the html files i would now simply use the option -A ogg,htm,html in the first place.
May
8
comment Wget - if / else download condition?
most excellent, thank you! sorry, I didn't watch out, you were right.
Apr
28
awarded  Teacher
Apr
17
accepted Parsing out specific URLs from local html file
Apr
17
comment Parsing out specific URLs from local html file
okay, thanks for this solution. --- the grep i am looking for would have been grep "^http://.*superuser\.com/" filewithlinksfromsuperuser.txt >> su-subdomainlinks.txt and grep "^http://superuser\.com/\users/[0-9]\{3,6\}" filewithlinksfromsuperuser.txt >> all su-users-links.txt
Apr
17
comment Parsing out specific URLs from local html file
oic, thanks, I'm sorry, overread that. I have to combine the two then. I tried grep "^http://*superuser\.com/" filewithlinksfromsuperuser.txt >> allsuperuserlinks.txt but it only got me the superuser.com links, when i wanted the subdomains blog.superuser.com and meta.superuser.com too... How do i use the asterisk right? (insisting because it was part of my question, sorry :) Any way to grep links that have a certain amount of numbers in it, like www.foo.com/12345 and www.foo.com/subfolder/123456. Or should i open a new question?
Apr
17
revised Parsing out specific URLs from local html file
emphasis on html parsing
Apr
17
comment Parsing out specific URLs from local html file
yes, thanks, i mean, doing it with a simple bash script, using awk, sed, or something else (as implied by the tags). ----- I tried your suggestion on my saved html-page, but it didn't work. My script above parses (reads out) every URL that it finds inside the a href tags of the html file and copies them to a plain txt file, line by line. You might want to try our scripts to see what i mean. For example, save this page, then only keep the links that follow meta.superuser.com or superuser.com/users for a start...
Apr
17
revised Parsing out specific URLs from local html file
specified bash requirement
Apr
17
comment Parsing out specific URLs from local html file
would need it with bash so i can merge it easily with other scripts i wrote, thanks. I'm pretty new to this...
Apr
17
comment I think my HD is Failing. How can I confirm whether or not it is?
if there is other important stuff on it, you could try total commanders batch copy process over night, and tell it to skip all files it can't read (you'll have to check every now and then for the first hour probably, if there are popups how to go on, but after that it does everything reliable till the end, or until it gets to a physical part that makes the HDD hang up. totalcommander.com --- please don't forget to vote or accept. Hungry for badges :)
Apr
17
awarded  Analytical
Apr
17
revised Parsing out specific URLs from local html file
added 2 characters in body
Apr
17
asked Parsing out specific URLs from local html file
Apr
17
comment Wget - if / else download condition?
(...) the bogus scripts in `/cgi-bin', you can specify: wget -I /people,/cgi-bin host/people/bozo" ---- (apparently it also says -D makes only sense when using with -H (allowing wget "spanning", to follow links/downloads to other domains) - it still makes directories for other domains which i didn't want, though)