Yes, that's 100% true, but if you think about it, your Internet service provider (ISP) could redirect the requests to another server as well. Even if your website had a certificate that is accepted by a certificate authority (CA), the CA could, in conjunction with your Internet service provider, dynamic DNS provider, or if you had a static IP, regular DNS provider could redirect the requests to another server and the certificate could be accepted for the other server.
Unfortunately, that's the way the Internet and the Web work. In your computer is a list of sources to accept certificates from, and depending on where your computer's operating system comes from, the list is provided and can be changed at any time by the vendor (Microsoft, Apple, etc.)
If you trust them and the CA, ISP, and DNS provider, you're fine. And it's very rare for any issues relating to this to happen, but they have happened before. https://www.eff.org/deeplinks/2011/08/iranian-man-middle-attack-against-google
Sorry if you're disappointed in the answer... But that's how things work online.