My users will be uploading products on my website which are registered by a government organisation. This government organisation has created an online database of these products which is free to search (but has not provided the ability to download). I was wondering if there was any way to automatically search this other database before my users have upload their product as a means to check that their product actually exists? This is in order to create integrity on the website.

I understand how the match it but was wondering if it was possible to automatically search online database from another site and pull data from it?

Cheers in advance

  • 1
    Is the database web based? Check the terms of service, if they don't forbid it you could do web scrapping. In some cases you can programmatically emulate the visiting of a website. The best case would if the government provided an API then someone could write a program to do it easily. – cybernard May 12 '16 at 0:47
  • Yes that database is web based their website hosts a search engine for it, thanks for the web scrapping tip! – Philayyy May 12 '16 at 0:57

You have not provided specifics, so I can't answer with specifics, but CURL is very often used to do this kind of "scraping" - it basically lets you control how to get web requests, and can even do things like log you in, set a browser string, handle HTTPS, follow redirects etc.

If your curent site uses PHP, there are PHP modules for it, so its easy to integrate.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.