9

I am looking for a tool (possibly on Linux) that will allow me to make a manual HTTP or HTTPS request. By manual, I really mean it: I should be able to feed it a text file which looks like

POST /foo HTTP/1.1
Host: www.example.com
Accept: text/xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-gb,en;q=0.5
Content-Type: text/plain
Content-Length: 11

Hello world

and a destination URL (www.example.com/foo), and send the request to the URL. At most, it would be useful if the Content_Length header was automatically computed.

I would be able to write such a tool using some library like httplib in Python, but the very reason I need it is to do manual investigation when something goes wrong with such libraries.

9

wget has a --post-file option which should work for you.

Edit: Also, there's Ncat, which you would use in a similar fashion to Randolf Richardson's telnet suggestion, except that it also supports SSL/HTTPS:

ncat -C --ssl www.example.com 443 < input.txt > output.txt
  • 5
    If you need HTTPS, use openssl instead of netcat: openssl s_client -connect <address>:443 – Ambroz Bizjak Jul 18 '11 at 22:24
  • wget --post-file will add headers and use the specified file only for the body of the request. – David Balažic Apr 3 '18 at 18:16
6

For HTTP (not HTTPS), one alternative to the "wget" command that comes to mind is to use telnet as follows:

  • telnet hostname 80 < input.txt > output.txt

The file "input.txt" is your list of pre-set commands that you wish to feed to the host at hostname and the file "output.txt" will store the response.

  • 1
    This made me wonder if netcat supports SSL, and lo-and-behold, there's Ncat, an SSL-enabled netcat clone. – Chris Acheson Jul 18 '11 at 22:15
  • @Chris Acheson (+1): That's fantastic! Please feel free to copy what I wrote in a separate answer and adapt it to a solution that utilizes NetCat (I'll vote for your answer if you do this). – Randolf Richardson Jul 18 '11 at 22:38
1

You can issue a GET request with OpenSSL:

openssl s_client -quiet -connect cdn.sstatic.net:443 <<eof
GET /stackexchange/js/universal-login.js HTTP/1.1
Connection: close
Host: cdn.sstatic.net

eof

Note that you can also use "HTTP/2", but be careful because some servers (e.g. github.com) do not support it.

0

For me, it worked creating a request file (Example: request.txt)

POST /foo HTTP/1.1
Host: www.example.com
Accept: text/xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-gb,en;q=0.5
Content-Type: text/plain; charset=utf-8
Content-Length: 11
Connection: close

Hello world

And then call the openssl s_client command:

cat request.txt | openssl s_client -quiet -connect www.example.com:443

However, be careful about some points:

  • The file must be encoded properly, specially the content body. Better if you include a header Content-type: text/plain; charset=utf-8 if the file is utf-8 encoded.
  • The line endings for the headers must be CR/LF. Only with LF the web server could return HTTP/1.1 505 HTTP Version Not Supported, because the standard HTTP protocol uses CR/LF for line endings (https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html). You can use the unix2dos command to do this conversion.
  • Add a Connection: close header to terminate the request and return from the call. Otherwise, the command will be awaiting for user interaction, if the server normally responds with a Connection: keep-alive header.
-1

I'm surprised no one mentioned cURL. It is made exactly for what you want to do. And it is available on practically any platform (including Windows).

So for your example all you would do is:

curl -H 'Content-Type: text/plain' -d 'Hello World' www.example.com

Which captured from Wireshark will net you:

POST / HTTP/1.1
User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
Host: www.example.com
Accept: */*
Content-Type: text/plain
Content-Length: 11

Hello world

You can easily modify the headers further if you wish to do all kinds of stuff (i.e. change the user-agent, etc).

Edit: Didn't notice the "from a file" requirement. You can do that too, either plain ascii or binary files. You just specify the filename with an @ symbol

-d @/tmp/HelloWorldFile
  • curl is almost what I want, but requires me to specify the headers on the command line, which is very impractical. I need something which allows me to specify the complete text of the request, including the method and the headers, from a file, and ncat does eaxctly what I want. Thank you anyway :-) – Andrea Jul 21 '11 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.