Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to scrape Brenda Enzymes for a bunch of molecule files with a script. If I just download by file URL (it's a PHP redirect), I get a 0Kb file, so I think they're checking for cookies and/or referer.

Then I did this:

1st request

$ wget --cookies=on --keep-session-cookies --save-cookies=cookies.txt "http://www.brenda-enzymes.info/php/search_result.php4?a=54&W%5B2%5D=&T%5B2%5D=1&Search=Search&l=100&os=1&RNV=1&RN=&T%5B0%5D=2&W%5B1%5D=3.4.21.5&T%5B1%5D=1&V%5B3%5D=1&W%5B3%5D=&T%5B3%5D=1&V%5B4%5D=1&W%5B4%5D=&T%5B4%5D=2&V%5B5%5D=1&W%5B5%5D=&T%5B5%5D=2&V%5B6%5D=1&W%5B6%5D=&T%5B6%5D=2&W%5B7%5D=&T%5B7%5D=2&FNV=1&orderDesc=1&orderByHTMLField=IC50+Value"

2nd request

$ wget --referer="http://www.brenda-enzymes.info/php/search_result.php4?a=54&W%5B2%5D=&T%5B2%5D=1&Search=Search&l=100&os=1&RNV=1&RN=&T%5B0%5D=2&W%5B1%5D=3.4.21.5&T%5B1%5D=1&V%5B3%5D=1&W%5B3%5D=&T%5B3%5D=1&V%5B4%5D=1&W%5B4%5D=&T%5B4%5D=2&V%5B5%5D=1&W%5B5%5D=&T%5B5%5D=2&V%5B6%5D=1&W%5B6%5D=&T%5B6%5D=2&W%5B7%5D=&T%5B7%5D=2&FNV=1&orderDesc=1&orderByHTMLField=IC50+Value" --cookies=on --load-cookies=cookies.txt --keep-session-cookies --save-cookies=cookies.txt -O test.mol http://www.brenda-enzymes.info/brenda_ligand/download_ligand_molfile.php4?LigandID=298989

Still nothing, the resulting test.mol is an empty file.

However, if I follow the first URL in a browser, and then copy-n-paste the second URL, it does download an actual 3Kb-ish .mol file.

What am I missing? Thanks.

share|improve this question
    
testing your two wget works on my linux host... perhaps you have to delete cookies.txt ... i get a test.mo1 of 2168 Bytes. –  philippe lhardy Mar 17 '13 at 16:24
add comment

1 Answer

A better way to do what you are attempting is to use SOAP queries. According to their documentation, the Brenda site offers SOAP functionality.

I don't know what type of information you are looking for. The following is an example of a Perl script to get the Km value for the molecule with an EC number of 1.1.1.1:

#!/usr/bin/perl -w
use SOAP::Lite;

$resultString = SOAP::Lite
-> uri('http://www.brenda-enzymes.info/soap2')
-> proxy('http://www.brenda-enzymes.info/soap2/brenda_server.php')
-> getKmValue("ecNumber*1.1.1.1#organism*Homo sapiens#")
-> result; 

print $resultString;

Have a look through the documentation, you should be able to find what you need.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.