-3

When I am trying to access transfermarkt.com, transfermarkt.co.uk and transfermarkt.de from my home network on any unit (PC, mobile, etc.) it gives me a page saying:

403 forbidden nginx

If I try to use mobile data instead or connect to a different Wi-Fi it works fine.

I think Transfermarkt blocked me because I have been scraping their site, so it probably detected that a lot of traffic was coming from an ISP over a couple of days and then somehow blocked it (I wasn't blocked instantly it was a couple of days after). Maybe they have blacklisted some kind of ID of my router or ISP, is there some way to renew that, like renewing an IP address on a PC so I won't be blacklisted anymore?

The only similar question I was able to find was this:

Why am I forbidden from accessing this website from my home internet connection, but can access it from my cell phone?

And there wasn't really any answers I could use, I seriously doubt I could convince transfermarkt to unblock me.

Forgive me if I sound ignorant, my question is just if there is a way to bypass the ban?

  • They most likely just blocked your external IP. – MoonRunestar Feb 2 '18 at 15:14
  • 3
    Your ISP is not blocked bu most likely your individual IP address has been blocked. How to get a new IP address is determined by how your ISP works. Some ISPs assign you a static IP and the only way you might be able to change that is by asking them if you can have it changed. Some other ISPs allow you to reset your modem—power it off, wait a few minutes, and then power it back up—and that should be good. But at the end of the day, the website is blocking you so if you get a new IP address and engage in the same behavior you might be banned again. – JakeGould Feb 2 '18 at 15:15
  • 3
    Also, you don’t know if this website ban is permanent or temporary. Meaning if you wait a few days, you might find your address unbanned. But again, it all depends on the way the site you are accessing handles bans and lifting bans. – JakeGould Feb 2 '18 at 15:16
  • The IP address of my PC isn't blocked. I can access the website just fine if I connect to another network. If I try to use any device even my phone on my home network it doesn't work. – Martin Feb 2 '18 at 15:23
  • 1
    “The IP address of my PC isn't blocked.” You have no idea how IP addresses work. Your home network IP address is what is blocked. Anything that uses that IP address will be blocked. If your PC, phone, table or anything uses another external IP address, you can then connect. This is purely an issue with your IP address on your modem/router. – JakeGould Feb 2 '18 at 16:13
1

Your external IP is blocked, fixes may include:

  1. Rebooting your router - you may get a different public IP after a reboot.
  2. Contacting your ISP to get a new IP - some ISP's will change your IP if you just request it.
  3. Using a vpn or other tunnelling service to hide your external IP - many free tunnelling services exist, however be sure of the reputation of any service you use as they can see all your internet traffic.
  4. Using tor
  5. Using a mobile connection such as a mobile hotspot, or setting up internet sharing via a mobile phone
  6. Using a web cafe.
2

I have been scraping their site

Did you get permission?

The content of this website is protected by copyright. Reprint, inclusion in online services, Internet and duplication on data carriers such as CD-ROM, DVD-ROM, etc., may only be made in part, with the prior written consent of Transfermarkt. Commercial re-marketing of the content is prohibited.

(their legal notice)

My IP has been blocked ... Is there a way I can bypass this ban?

Yes, both legal and not-legal.

You could explain to them why you need their data wholesale, what you intend to do with it, and get their permission.

Bypassing a ban based on IP-addresses is simply a matter of obtaining a different IP-address, this might be as simple as rebooting a router or switching ISP. By doing so you are likely in breach of the sites terms and conditions though.

  • 1
    According to whatismyipaddress.com I have a static IP address, so I don't think I can change my IP address by resetting my router. Why has no one mentioned the possibility of using VPN? I tested it and it worked. Too bad they all either cost money or put restrictions and give you malware and sell your private data. – Martin Feb 2 '18 at 16:36
  • 1
    Oh and no I haven't gotten their permission. I have never heard of anyone ever asking for permission before scraping a bit of data off a random site and I can't imagine they would allow it. I'm not overloading the servers, or scraping all data and reselling it or anything. I just need some data for a neural network (small private project), that's all. – Martin Feb 2 '18 at 16:39
  • @Martin A website owner can do whatever they want. If your external IP address is blocked, it’s blocked. This question should be closed and this answer only states what has been stated in the comments: This issue is too idiosyncratic and too broad. You cannot “get around” something like this. You are blocked. Either you contact the site owner and ask why and see if they can remove it or you can see about getting a new IP address. But if you engage in the same behavior you will be blocked. You are not entitled to access to a privately run site for just scraping purposes. – JakeGould Feb 2 '18 at 16:41
  • Downvoted as it doesn't actually explain how to bypass a block in any meaningful way. I realise this maybe a moral quandary. – djsmiley2kStaysInside Feb 2 '18 at 16:58
  • 1
    @Martin “This is not idiosyncratic, I assume many different websites can block your router IP and there are multiple ways to bypass it.” That is why this is idiosyncratic. Utterly nobody here can tell you how your ISP works nor can anyone tell you how the website you are attempting to scrape enforces access. “Scraping in itself is not illegal…” Nobody is saying this is “illegal” but rather a website owner can do whatever they want so if they want to block scrapers, then they can and there is no way around this. All downvotes are justified because utterly nobody here can help you fix this. – JakeGould Feb 2 '18 at 17:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.