-3

The title is not a typo. When I am trying to access transfermarkt.com transfermarkt.co.uk and transfermarkt.de from my home network on any unit (PC, mobile, etc.) it gives me a page saying "403 forbidden nginx". If I try to use mobile data instead or connect to a different wifi it works fine.

I think transfermarkt blocked me because I have been scraping their site, so it probably detected that a lot of traffic was coming from an ISP over a couple of days and then somehow blocked it (I wasn't blocked instantly it was a couple of days after). Maybe they have blacklisted some kind of ID of my router or ISP, is there some way to renew that, like renewing an IP address on a PC so I won't be blacklisted anymore?

The only similar question I was able to find was this:

Why am I forbidden from accessing this website from my home internet connection, but can access it from my cell phone?

And there wasn't really any answers I could use, I seriously doubt I could convince transfermarkt to unblock me.

Forgive me if I sound ignorant, my question is just if there is a way to bypass the ban?

Improve this question
9
  • They most likely just blocked your external IP.
    – user487867
    Feb 2, 2018 at 15:14
  • 3
    Your ISP is not blocked bu most likely your individual IP address has been blocked. How to get a new IP address is determined by how your ISP works. Some ISPs assign you a static IP and the only way you might be able to change that is by asking them if you can have it changed. Some other ISPs allow you to reset your modem—power it off, wait a few minutes, and then power it back up—and that should be good. But at the end of the day, the website is blocking you so if you get a new IP address and engage in the same behavior you might be banned again.
    – Giacomo1968
    Feb 2, 2018 at 15:15
  • 3
    Also, you don’t know if this website ban is permanent or temporary. Meaning if you wait a few days, you might find your address unbanned. But again, it all depends on the way the site you are accessing handles bans and lifting bans.
    – Giacomo1968
    Feb 2, 2018 at 15:16
  • The IP address of my PC isn't blocked. I can access the website just fine if I connect to another network. If I try to use any device even my phone on my home network it doesn't work.
    – Martin
    Feb 2, 2018 at 15:23
  • 1
    “The IP address of my PC isn't blocked.” You have no idea how IP addresses work. Your home network IP address is what is blocked. Anything that uses that IP address will be blocked. If your PC, phone, table or anything uses another external IP address, you can then connect. This is purely an issue with your IP address on your modem/router.
    – Giacomo1968
    Feb 2, 2018 at 16:13

2 Answers 2

Reset to default
1

Your external IP is blocked, fixes may include:

  1. Rebooting your router - you may get a different public IP after a reboot.
  2. Contacting your ISP to get a new IP - some ISP's will change your IP if you just request it.
  3. Using a vpn or other tunnelling service to hide your external IP - many free tunnelling services exist, however be sure of the reputation of any service you use as they can see all your internet traffic.
  4. Using tor
  5. Using a mobile connection such as a mobile hotspot, or setting up internet sharing via a mobile phone
  6. Using a web cafe.
Improve this answer
1

I have been scraping their site

Did you get permission?

The content of this website is protected by copyright. Reprint, inclusion in online services, Internet and duplication on data carriers such as CD-ROM, DVD-ROM, etc., may only be made in part, with the prior written consent of Transfermarkt. Commercial re-marketing of the content is prohibited.

(their legal notice)

My IP has been blocked ... Is there a way I can bypass this ban?

Yes, both legal and not-legal.

You could explain to them why you need their data wholesale, what you intend to do with it, and get their permission.

Bypassing a ban based on IP-addresses is simply a matter of obtaining a different IP-address, this might be as simple as rebooting a router or switching ISP. By doing so you are likely in breach of the sites terms and conditions though.

Improve this answer
7
  • 1
    According to whatismyipaddress.com I have a static IP address, so I don't think I can change my IP address by resetting my router. Why has no one mentioned the possibility of using VPN? I tested it and it worked. Too bad they all either cost money or put restrictions and give you malware and sell your private data.
    – Martin
    Feb 2, 2018 at 16:36
  • 1
    Oh and no I haven't gotten their permission. I have never heard of anyone ever asking for permission before scraping a bit of data off a random site and I can't imagine they would allow it. I'm not overloading the servers, or scraping all data and reselling it or anything. I just need some data for a neural network (small private project), that's all.
    – Martin
    Feb 2, 2018 at 16:39
  • @Martin A website owner can do whatever they want. If your external IP address is blocked, it’s blocked. This question should be closed and this answer only states what has been stated in the comments: This issue is too idiosyncratic and too broad. You cannot “get around” something like this. You are blocked. Either you contact the site owner and ask why and see if they can remove it or you can see about getting a new IP address. But if you engage in the same behavior you will be blocked. You are not entitled to access to a privately run site for just scraping purposes.
    – Giacomo1968
    Feb 2, 2018 at 16:41
  • Downvoted as it doesn't actually explain how to bypass a block in any meaningful way. I realise this maybe a moral quandary.
    – djsmiley2kStaysInside
    Feb 2, 2018 at 16:58
  • 1
    @Martin “This is not idiosyncratic, I assume many different websites can block your router IP and there are multiple ways to bypass it.” That is why this is idiosyncratic. Utterly nobody here can tell you how your ISP works nor can anyone tell you how the website you are attempting to scrape enforces access. “Scraping in itself is not illegal…” Nobody is saying this is “illegal” but rather a website owner can do whatever they want so if they want to block scrapers, then they can and there is no way around this. All downvotes are justified because utterly nobody here can help you fix this.
    – Giacomo1968
    Feb 2, 2018 at 17:45

Not the answer you're looking for? Browse other questions tagged .