up vote 0 down vote favorite
share [g+] share [fb]

Google indexed a page on a site of mine that was not linked from any other page, ever. No one has ever put a link to it, and the directory contents were not browsable. How could this happen? I thought crawlers have no way to include a page that is not linked.

link|improve this question
74% accept rate
1  
Super User is intended for computer hardware and software related questions only. Web services such as Google search are considered off-topic. Please see the FAQ – heavyd Apr 29 '10 at 17:52
Where should I put this question then? – Yar Apr 29 '10 at 17:56
1  
Super User is not for every kind of question. Take a look at this question for ideas. meta.stackoverflow.com/questions/41570/… Also this list may be useful. meta.stackexchange.com/questions/4/list-of-stackexchange-sites – heavyd Apr 29 '10 at 18:00
feedback

closed as off topic by heavyd, Gnoupi, th3dude, quack quixote Apr 29 '10 at 18:13

Questions on Super User are expected to generally relate to computer software or computer hardware, within the scope defined in the faq.

1 Answer

up vote 1 down vote accepted

You didn't link to it, but how certain are you that no one else did? Someone sharing it on Facebook, Twitter, a public chat log, their personal site, etc. could do the trick.

If you don't want Google to index a page, block the Googlebot via robots.txt or some other technique. Relying solely on obscurity will bite you in the ass.

link|improve this answer
good point, thanks for that. I do not want to be ass-bitten. – Yar Apr 29 '10 at 17:57
Found the link to it, thanks again. – Yar Apr 29 '10 at 18:17
@ceejayoz, my point was just to make sure that somebody somewhere had to share the link to get it indexed in Google (which was the case: in fact it was me in a past life). I was kind of checking the rules of the universe, too. At the same time, I did some investigation into making stuff not web-visible using GoDaddy, which was a useful result of your answer. Thanks again. – Yar May 4 '10 at 14:24
feedback

Not the answer you're looking for? Browse other questions tagged or ask your own question.