Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to use Wget to download my private GitHub pages, but I cannot get past the login screen.

How do I send the login/password using post data on the login page and then download the actual page as an authenticated user?

Here is the command I am trying to run with the output:

 wget --save-cookies cookies.txt \
 --post-data 'login=myUserName&password=myPassword' \

Wget output:

Connecting to||:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2012-11-23 19:58:13 ERROR 403: Forbidden.

I tried the following command too:

 wget --save-cookies cookies.txt \
 --post-data 'authenticity_token=sPV07gM2/OHYDAT99WmawItd8R7hiTaJnBAs/b3zN9Y=&login=myUserName&password=myPassword' \

Here is the form HTML code of the login page,

<form accept-charset="UTF-8" action="/session" method="post"><div style="margin:0;padding:0;display:inline"><input name="authenticity_token" type="hidden" value="sPV07gM2/OHYDAT99WmawItd8R7hiTaJnBAs/b3zN9Y=" /></div> 
    <h1>Sign in <a href="">(Pricing and Signup)</a> </h1>
    <div class="formbody">

        <label for="login_field">
            Username or Email<br />
            <input autocapitalize="off" autofocus="autofocus" class="text" id="login_field" name="login" style="width: 21em;" tabindex="1" type="text" />

        <label for="password">
            Password <a href="/sessions/forgot_password">(forgot password)</a>
            <br />
            <input autocomplete="disabled" class="text" id="password" name="password" style="width: 21em;" tabindex="2" type="password" />

        <label class='submit_btn'>
            <input name="commit" tabindex="3" type="submit" value="Sign in" />
share|improve this question
up vote 8 down vote accepted

You're making several mistakes by not doing what your browser would do:

  • You need to send the POST request with login credentials to the form action, i.e.
  • You need to provide all form parameters, including the percent-encoded hidden form parameter authenticity_token.
  • You need to provide the session cookies set by /login.

The only thing not required I'd have expected is setting the referer.

What you need to do:

$ wget --keep-session-cookies --save-cookies cookies.txt -O login.rsp
$ grep authenticity_token login.rsp

This will request the login page, store the session, and print the CSRF token hidden form value (plus some surrounding HTML).

Now login after percent-encoding all parameters, especially the value of the hidden form parameter authenticity_token which often contains punctuation:

 $ wget --load-cookies cookies.txt --keep-session-cookies --save-cookies cookies.txt --post-data='login=USERNAME&password=PASSWORD&authenticity_token=TOKEN_VALUE_PRINTED_BY_GREP_THEN_PERCENT_ENCODED'

You'll get bounced around a bit, and will end up on, just like when logging in in the browser.

share|improve this answer
this doesn't work anymore getting "ERROR 422: Unprocessable Entity." – Luca G. Soave Jun 2 '15 at 12:01
@LucaG.Soave It's to be expected that the specifics of a non-API like this change after more than 2.5 years. But the basic advice should still hold -- do what your browser would do. – Daniel Beck Jun 2 '15 at 14:32

You must log in to answer this question.

protected by bwDraco May 11 '15 at 23:10

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged .