2

I'm trying to scrape Brenda Enzymes for a bunch of molecule files with a script. If I just download by file URL (it's a PHP redirect), I get a 0Kb file, so I think they're checking for cookies and/or referer.

Then I did this:

1st request

$ wget --cookies=on --keep-session-cookies --save-cookies=cookies.txt "http://www.brenda-enzymes.info/php/search_result.php4?a=54&W%5B2%5D=&T%5B2%5D=1&Search=Search&l=100&os=1&RNV=1&RN=&T%5B0%5D=2&W%5B1%5D=3.4.21.5&T%5B1%5D=1&V%5B3%5D=1&W%5B3%5D=&T%5B3%5D=1&V%5B4%5D=1&W%5B4%5D=&T%5B4%5D=2&V%5B5%5D=1&W%5B5%5D=&T%5B5%5D=2&V%5B6%5D=1&W%5B6%5D=&T%5B6%5D=2&W%5B7%5D=&T%5B7%5D=2&FNV=1&orderDesc=1&orderByHTMLField=IC50+Value"

2nd request

$ wget --referer="http://www.brenda-enzymes.info/php/search_result.php4?a=54&W%5B2%5D=&T%5B2%5D=1&Search=Search&l=100&os=1&RNV=1&RN=&T%5B0%5D=2&W%5B1%5D=3.4.21.5&T%5B1%5D=1&V%5B3%5D=1&W%5B3%5D=&T%5B3%5D=1&V%5B4%5D=1&W%5B4%5D=&T%5B4%5D=2&V%5B5%5D=1&W%5B5%5D=&T%5B5%5D=2&V%5B6%5D=1&W%5B6%5D=&T%5B6%5D=2&W%5B7%5D=&T%5B7%5D=2&FNV=1&orderDesc=1&orderByHTMLField=IC50+Value" --cookies=on --load-cookies=cookies.txt --keep-session-cookies --save-cookies=cookies.txt -O test.mol http://www.brenda-enzymes.info/brenda_ligand/download_ligand_molfile.php4?LigandID=298989

Still nothing, the resulting test.mol is an empty file.

However, if I follow the first URL in a browser, and then copy-n-paste the second URL, it does download an actual 3Kb-ish .mol file.

What am I missing? Thanks.

  • testing your two wget works on my linux host... perhaps you have to delete cookies.txt ... i get a test.mo1 of 2168 Bytes. – philippe lhardy Mar 17 '13 at 16:24
  • Make sure your cookies.txt is blank when trying the 1st request, the existing cookie may be confusing the server. – LawrenceC Sep 9 '15 at 12:41
2

A better way to do what you are attempting is to use SOAP queries. According to their documentation, the Brenda site offers SOAP functionality.

I don't know what type of information you are looking for. The following is an example of a Perl script to get the Km value for the molecule with an EC number of 1.1.1.1:

#!/usr/bin/perl -w
use SOAP::Lite;

$resultString = SOAP::Lite
-> uri('http://www.brenda-enzymes.info/soap2')
-> proxy('http://www.brenda-enzymes.info/soap2/brenda_server.php')
-> getKmValue("ecNumber*1.1.1.1#organism*Homo sapiens#")
-> result; 

print $resultString;

Have a look through the documentation, you should be able to find what you need.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.