-4

Is there a place to download a list of newly emerging websites every day like search engines do? When I set up a new website, I do not create any backlink for it. How does google know my website and crawl it? Does it use some kind of whois database? But I do not think there is a whois database that allows you to query every day.

7
  • This would be the purpose of DNS. How Google functions would not be known to anyone outside of Google. Furthermore, questions about how Google Search works (which is a web application) is out of scope here at Super User. Commentary isn’t required to issue a downvote
    – Ramhound
    Jan 22, 2021 at 6:04
  • If you disagree with my assertion then you can bring up the topic at meta.superuser but I won’t spend time justifying my close vote in the comment section
    – Ramhound
    Jan 22, 2021 at 6:09
  • Google Search tag description is clear in this regard though
    – Ramhound
    Jan 22, 2021 at 6:11
  • 2
    Your question isn’t close at this time. Others might disagree with my assertion. However, making personal attacks, because you disagree with a downvote isn’t acceptable behavior. I won’t be returning to this question in effort to deescalate the current conversation. Most of your questions have been upvoted so your complaint doesn’t make sense
    – Ramhound
    Jan 22, 2021 at 6:13
  • 1
    You submitted this question on the wrong community. Read the Google Search tag description. If you are question banned on Severfault then there is a reason that is likely justified. You would have to submit a question to meta.serverfault to better understand the reason. (You likely have too many deleted questions)
    – Ramhound
    Jan 22, 2021 at 6:20

1 Answer 1

2

There is no authorative place to download a list of new websites, and contrary to some comments new sites are not discovered because of DNS.

Sites are typically found via a link to them or a request to the search engine. Many systems automatically "ping" Google when the site is installed or content is added to ask for an update.

2
  • I was thinking it would not be that much data to watch for changes to various major DNS providers. I mean at the end of the day we would just be guessing how Google’s secret sauce works :$
    – Ramhound
    Jan 22, 2021 at 7:11
  • @Ramhound Other then requests to 8.8.8.8 and maybe DoH I don't see how Google could ethically achieve this through DNS.
    – davidgo
    Jan 22, 2021 at 7:40

Not the answer you're looking for? Browse other questions tagged .