Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am trying to use wget to download my private github pages, but I cannot get past the login screen.

How do I send the login/password using post data on the login page and then download the actual page as an authenticated user?

Here is the command I am trying to run with the output:

 wget --save-cookies cookies.txt \
 --post-data 'login=myUserName&password=myPassowrd' \
 https://github.com/login

wget output:

Resolving github.com... 207.97.227.239
Connecting to github.com|207.97.227.239|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2012-11-23 19:58:13 ERROR 403: Forbidden.

I tried the following command too:

 wget --save-cookies cookies.txt \
 --post-data 'authenticity_token=sPV07gM2/OHYDAT99WmawItd8R7hiTaJnBAs/b3zN9Y=&login=myUserName&password=myPassowrd' \
 https://github.com/login

Here is the form html-code of the login page https://github.com/login

  <form accept-charset="UTF-8" action="/session" method="post"><div style="margin:0;padding:0;display:inline"><input name="authenticity_token" type="hidden" value="sPV07gM2/OHYDAT99WmawItd8R7hiTaJnBAs/b3zN9Y=" /></div>      <h1>
        Sign in
          <a href="https://github.com/plans">(Pricing and Signup)</a>
      </h1>
    <div class="formbody">
        <label for="login_field">
          Username or Email<br />
          <input autocapitalize="off" autofocus="autofocus" class="text" id="login_field" name="login" style="width: 21em;" tabindex="1" type="text" />
        </label>

        <label for="password">
          Password <a href="/sessions/forgot_password">(forgot password)</a>
          <br />
          <input autocomplete="disabled" class="text" id="password" name="password" style="width: 21em;" tabindex="2" type="password" />
        </label>

        <label class='submit_btn'>
                              <input name="commit" tabindex="3" type="submit" value="Sign in" />
        </label>
    </div>
</form>
share|improve this question
add comment

1 Answer 1

up vote 5 down vote accepted

You're making several mistakes by not doing what your browser would do:

  • You need to send the POST request with login credentials to the form action, i.e. https://github.com/session.
  • You need to provide all form parameters, including the percent-encoded hidden form parameter authenticity_token.
  • You need to provide the session cookies set by /login.

The only thing not required I'd have expected is setting the referer.


What you need to do:

$ wget --keep-session-cookies --save-cookies cookies.txt -O login.rsp https://github.com/login
$ grep authenticity_token login.rsp

This will request the login page, store the session, and print the CSRF token hidden form value (plus some surrounding HTML).

Now login after percent-encoding all parameters, especially the value of the hidden form parameter authenticity_token which often contains punctuation:

 $ wget --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt --post-data='login=USERNAME&password=PASSWORD&authenticity_token=TOKEN_VALUE_PRINTED_BY_GREP_THEN_PERCENT_ENCODED' https://github.com/session

You'll get bounced around a bit, and will end up on https://github.com, just like when logging in in the browser.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.