Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to write a shell script to grab the dynamic URL which ComboFix is located at at BleepingComputer.com/download/combofix

However, for some reason I can't seem to get my regex to match the download link of the "click here" if the download doesn't work. I used a regex tester and it said I matched the link, but I can't seem to get it to work when I execute it, it turns up an empty result. Here's my entire script:

 #!/bin/bash
 # Download latest ComboFix from BleepingComputer
 wget -O Listing.html "http://www.bleepingcomputer.com/download/combofix/" -nv
 downloadpage=$(sed -ne 's@^.*<a href="\(http://www[.]bleepingcomputer[.]com/download/combofix/dl/[0-9]\+/\)" class="goodurl">.*$@\1@p' Listing.html)
 echo "DL Page: $downloadpage"
 secondpage="$downloadpage"
 wget -O Download.html $secondpage -nv
 file=$(sed -ne 's@^.*<a href="\(http://download[.]bleepingcomputer[.]com/dl/[0-9A-Fa-f]\+/[0-9A-Fa-f]\+/windows/security/anti[-]virus/c/combofix/ComboFix[.]exe\)">.*$@\1@p' Download.html)
 echo "File: $file"
 wget -O "ComboFix.exe" "$file" -nv
 rm Listing.html
 rm Download.html
 mkdir Tools
 mv "ComboFix.exe" "Tools/ComboFix.exe" -f

The first two downloads work successfully, and I end up with: http://www.bleepingcomputer.com/download/combofix/dl/12/

But it fails to match the final sed that will give me the download link.

The code it's supposed to match is:

<a href="http://download.bleepingcomputer.com/dl/6c497ccbaff8226ec84c97dcdfc3ce9a/5058d931/windows/security/anti-virus/c/combofix/ComboFix.exe">click here</a>

SOLUTION:

For anyone interested, the final code that worked was:

#!/bin/bash
# Download latest ComboFix from BleepingComputer
wget -O Download.html "http://www.bleepingcomputer.com/download/combofix/12" -nv
file=$(sed -ne 's@^.*<a href=\x27\(http://download[.]bleepingcomputer[.]com/dl/[0-9A-Fa-f]\+/[0-9A-Fa-f]\+/windows/security/anti[-]virus/c/combofix/ComboFix[.]exe\)\x27>.*$@\1@p' Download.html)
echo "File URL: $file"
wget -OSN "ComboFix.exe" "$file" -nv
share|improve this question

2 Answers 2

up vote 1 down vote accepted

That's funny: I just went to http://www.bleepingcomputer.com/download/combofix/dl/12/ and found

<a href='http://download.bleepingcomputer.com/dl/587c6038..../..../ComboFix.exe'>click here</a>.

i.e., the URL is delimited by single quotes (') rather than double quotes (") (which is what your sed command is looking for).

share|improve this answer
    
How very strange, I'm viewing it in Chrome and I'm seeing double quotes –  user901398 Sep 19 '12 at 2:47
    
You're right though, that's the problem, I viewed the Download.html file and it has single quotes –  user901398 Sep 19 '12 at 2:53

There is no need to download the first page as the second page is static. It will always be:

http://www.bleepingcomputer.com/download/combofix/dl/12/

It is also strongly suggested that you use the -N flag on wget to check timestamps. This will only download ComboFix if its newer then the one on your computer.

Also make sure your script only runs once or twice a day. There is no need to run it more than that.

By limiting the requests and using the -N, you should avoid being automatically banned by our scripts.

Thanks

share|improve this answer
    
I wasn't certain if the first one was static or not, since otherwise the number seems unnecessary, thanks for letting me know. Currently the script only runs once a day at night, I'll be sure to add the -N to it, thanks for the tip and inside information :) –  user901398 Sep 20 '12 at 7:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.