-1

We have setup one of web site on server and site build in php and symfony framework, As my requirement is to prevent Google from indexing and below of my robot.txt and Is prevent using .htaccess?

User-agent: *
Disallow:

So How to to prevent it and test?

  • Last time I had to do something like this was after seeing them in my website stats. So if you don't see them in a while consider job done, maybe :| – pun Jun 29 '16 at 11:23
4

Your file should be named: robots.txt If you can see the content of the file on line : http://youwebsite/robots.txt, then it should be working. Otherwise you can test it with this google tool: https://support.google.com/webmasters/answer/6062598?hl=en

  • Problem is that, get 404 file not found but file is exists, not works; How can I prevent Google from indexing in my project, I'm using 1and1.com panel, So Where to change it? -i'm confuses. pls just give a way... – Nullpointer Jun 29 '16 at 13:30
  • What file shows a 404 error? The robots.txt one? Be carefull about the 's' because in your first post you named it robot.txt – hyde Jun 30 '16 at 12:27
  • It's my bit mistake on htaccess file, So that not working but Now I have two option for this 1) robots.txt and .htaccess ( htmlremix.com/seo/… )for prevent indexing ; So which one should better ? – Nullpointer Jun 30 '16 at 12:33
  • The robots.txt solution is the best beaucoup every search engine will check it. Whereas in the .htaccess you will need to specify every search engine user agent. And if one is changing you will have to change the name in your .htaccess – hyde Jun 30 '16 at 12:38

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.