58

I'd like to send the HTTP HEAD request using wget. Is it possible?

64

It's not wget, but you can do that quite easily by using curl.

curl -I http://www.superuser.com/

Produces this output:

HTTP/1.1 301 Moved Permanently                        
Content-Length: 144                       
Content-Type: text/html; charset=UTF-8     
Location: http://superuser.com/
Date: Sat, 09 Oct 2010 19:11:50 GMT
4
  • This is exactly what I want. Oct 11 '10 at 15:23
  • 2
    -I is equivalent to --head.
    – Nicolas
    Dec 10 '15 at 20:48
  • 2
    If you need self-signed certificate based https, you can also add -k or --insecure
    – Mike Aski
    Feb 6 '19 at 17:31
  • Given that the question explicitly mentioned wget i can't understand why an answer using curl can be considered the best, especially that there may be contexts where just one of the two is available.
    – Treviño
    Feb 24 at 22:39
40

Try:

wget -S --spider www.example.com

You can also pass -O /dev/null to prevent wget from writing HTTP response to a file.

5
  • 4
    -S will show headers, but it executes a GET, not a HEAD. In other words, it will fetch the entire URL. Feb 28 '14 at 23:21
  • 14
    wget -S --spider http://localhost log created in apache server is 127.0.0.1 - - [04/Mar/2014:15:36:32 +0100] "HEAD / HTTP/1.1" 200 314 "-" "Wget/1.13.4 (linux-gnu)" Mar 4 '14 at 14:38
  • 1
    from the manual : --spider When invoked with this option, Wget will behave as a Web spider which means that it will not download the pages, just check that they are there.
    – ychaouche
    Apr 6 '20 at 19:37
  • It is still creating an empty directory structure, so I found that -nd (-nH --cut-dirs=100 also works) works to suppress that.
    – haridsv
    Sep 16 '20 at 14:38
  • -nv or --no-verbose for just the facts.
    – Bob Stein
    Jun 22 at 16:27
22

There isn't any need for curl.

With Wget, adding --spider implies that you want to send a HEAD request (as opposed to GET or POST).

This is a great minimalistic way of checking if a URL responds or not. You can for example use this in scripted checks, and the HEAD operation will make sure you do not put any load on neither the network nor the target webserver.

Bonus information: If Wget gets an HTTP error 500 from the server when it performs the HEAD it will then move on to perform a GET against the same URL. I don't know the reasoning for this design. This is the reason why you may see both a HEAD and a GET request being performed against the server. If nothing is wrong then only a HEAD request is performed. You can disable this functionality with the --tries option to limit Wget to only one attempt.

All in all, I recommend this for testing if an URL is responding:

# This works in Bash and derivatives
wget_output=$(wget --spider --tries 1 $URL  2>&1)
wget_exit_code=$?

if [ $wget_exit_code -ne 0 ]; then
    # Something went wrong
    echo "$URL is not responding"
    echo "Output from wget: "
    echo "$wget_output"
else
    echo "Check succeeded: $URL is responding"
fi
4

wget -S gets file:

Content-Length: 2316, Length: 2316 (2.3K) [text/plain], Saving to: `index.html'

wget --spider gets headers:

Spider mode enabled. Check if remote file exists., Length: unspecified [text/plain] Remote file exists.

1

Ubuntu 19.10 Manpage: Wget

   --method=HTTP-Method
       For the purpose of RESTful scripting, Wget allows sending of other HTTP Methods
       without the need to explicitly set them using --header=Header-Line.  Wget will use
       whatever string is passed to it after --method as the HTTP Method to the server.

Used the following in my bash script and can confirm it works as expected!

wget --method=HEAD https://www.website.com/
1
  • At least the Wget version in Alpine Linux v3.12 does not support the --method parameter.
    – Gogowitsch
    Jan 23 at 21:12
-1

Though not wget, many perl installs with lwp module will have a HEAD command installed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.