3

I'm trying to accomplish this task, I have an IP and with the IP I want to make an HTTP request.

Let's say that I have the stackoverflow.com IP (151.101.193.69) and I want to make an HTTP request at the website with python using the requests module. I wrtie this code

import requests
response = requests.get("http://151.101.193.69:80")
response.text

that returns the html code of the page. Doing so I get a page that says

Fastly error: unknown domain: 151.101.193.69. Please check that this domain has been added to a service.

Details: cache-mxp19846-MXP

But if I do this with the Google IP it works and the request returns the Google home page

import requests
response = requests.get("http://216.58.205.174:80")
response.text
2
  • Try this in your browser 151.101.193.69:80 The result you are getting in python is the same as in the browser So this is not a python issue, and not a question of how to make an HTTP request. The HTTP request is fine. There's some issue at the server end – barlop May 19 '19 at 12:21
  • And the question is completely wrong because you know very well how to make a request to an IP since you managed it for google's IP:PORT – barlop May 19 '19 at 12:22
8

Yes, that's normal.

HTTP requests always include a "Host:" header indicating what name was actually used in the URL. For example, accessing http://superuser.com will send a header Host: superuser.com; and if you're using http://151.101.193.69, the header will of course show Host: 151.101.193.69 (or even be completely absent).

The important bit is that many HTTP servers share the same IP address for many different domains (aka virtual hosting), and they rely on this header to know which domain you're trying to access.

For example, all Stack Exchange domains – superuser.com, stackoverflow.com, serverfault.com, diy.stackexchange.com, … – share the exact same set of IP addresses. The only way for the webserver to distinguish between all these websites is by looking at the "Host" HTTP header.

(Additionally, these addresses actually belong to Fastly CDN, not to the real webservers. Large CDNs such as Fastly or CloudFlare might use the same IP address for hundreds of domains from various different customers.)

Finally, it is up to the server to decide what it'll do with "Host" headers that it doesn't recognize. Some servers return an error page stating so, other servers return the "first" domain that they have.


So in order to make a successful request, you'd need to do this:

requests.get("http://151.101.193.69", headers={"Host": "stackoverflow.com"})

But that's a silly way of saying requests.get("http://stackoverflow.com"). The only time you might need this method is when you have an IP address that doesn't match the DNS information (e.g. if you're trying to bypass DNS).

(And it will cause you more problems as soon as you try to access a HTTPS website, because the domain from the URL is used for checking TLS certificates, too.)

2
  • and if i know only the ip without knowing the host? – Mat.C May 19 '19 at 12:46
  • Then you usually cannot access the website, and hopefully the post explains why you cannot. (It might work with dedicated webservers which literally have just that one website, but it usually won't work with webservers which have multiple.) – user1686 May 19 '19 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.