1

Background: I have TP-Link Archer D7 router which uses a local web page for its configuration. I would like to download two given pages automatically using wget (as a daily precaution to check whether config was changed by somebody) -- for this I need direct, given, address of a page.

Some pages show you all the time the main address, despite internally they navigate among various pages. The problem is, that you cannot go to page X, because even if you are at page X, the url states you are at main page.

My case is such, I am looking at advanced status page of my router, yet, the URL still shows http://192.168.1.1/ (main page of the router). So for example if I copy&paste this url to another tab, I will get main page, instead of status. I peeked (using firebug) what page is fetched when I switch to the status page -- http://192.168.1.1/main/status.htm?_=1442919208105. However when I enter this page in URL box, I get error 403 Forbidden. It is not authentication problem because it is the same browser.

My question is -- what do I miss? What should I do, to use direct urls?

4
  • Can you use the tamperdata plugin or equivalent and look at what HTTP data is being sent?
    – user193661
    Sep 22, 2015 at 12:42
  • @user193661, sure, firebug can show HTTP traffic as well. What I should look for? As I already wrote, I found out single GET for the address which interests me, but when I copy&paste it I get error 403. Sep 22, 2015 at 13:21
  • 1
    You need to "replay" the request in the browser, so the correct headers are used.
    – user193661
    Sep 22, 2015 at 13:27
  • @user193661, thank you. With help of this post superuser.com/questions/584918/… and then Header Hacker from Chrome I was able to get the direct page (this TP-Link stuff is ordeal...). The problem was with missing Referer header when accessing the page directly. Could you please post your comment as an answer so I could accept it? Sep 22, 2015 at 16:24

2 Answers 2

1

If you unexpectedly get 403 HTTP response, the web server may be checking for certain headers. So you should check that you are using the correct HTTP headers. You can do this with a web browser plugin like Tamper Data for Firefox or Header Hacker for Google Chrome.

Example HTTP GET request and response:

  5:40:08.691[72ms][total 72ms] Status: 302[Moved Temporarily]
  GET http://192.168.1.1/9 Load Flags[LOAD_DOCUMENT_URI  LOAD_INITIAL_DOCUMENT_URI  ] Content Size[-1] Mime Type[text/html]
     Request Headers:
        Host[192.168.1.1]
        User-Agent[Mozilla/5.0]
        Accept[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
        Accept-Language[en-US,en;q=0.5]
        Accept-Encoding[gzip, deflate]
        Referer[http://192.168.1.1/]
        Cookie[session_id=1]
        Connection[keep-alive]
     Response Headers:
        Content-Type[text/html]
        Cache-Control[public]
        Pragma[cache]
        Expires[Tue, 22 Sep 2015 13:10:12 GMT]
        Date[Tue, 22 Sep 2015 12:40:12 GMT]
        Last-Modified[Tue, 22 Sep 2015 12:40:12 GMT]
        Accept-Ranges[bytes]
        Connection[close]
        Location[/page9]
0
0

Option 1, Hover over links

In all modern browsers you can hover over the link and it will show you the page URL, like this: cPanel 1 cPanel 2 and harvest the URL that way

Option 2, Inspect element

You could inspect element like so: Inspect Element 1 Inspect Element 2 and copy that URL but add http://192.168.1.1 at the start

Option 3, Show only this frame

As far as I know, this only works in Firefox. Since these are iframes, you can right click in the frame and select Open Frame in New Tab and harvest the URL as below: View frame

3
  • He already knows what page will be fetched "I peeked (using firebug) what page is fetched when I switch to the status page -- 192.168.1.1/main/status.htm?_=1442919208105. However when I enter this page in URL box, I get error 403 Forbidden."
    – DavidPostill
    Sep 22, 2015 at 11:32
  • @DavidPostill I've seen this before where the status.htm is fetching something else entirely
    – td512
    Sep 22, 2015 at 11:33
  • Thank you, but (1) I don't have direct URL given somehow in the main page, this what I would like to read; in this case it is javascript:void(0), (2) same as (1), (3) it does not have to be iframe, modern, dynamic pages go beyond frames, and unfortunately it is my case. Sep 22, 2015 at 11:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .