75

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell.

But when I redirect stdout like this:

curl -v http://somehost/somepage > /dev/null

Some sort of results table appears in the middle of the output to stderr:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

Followed by this near the end:

{ [data not shown]
118   592    0   592    0     0  15714      0 --:--:-- --:--:-- --:--:-- 25739

Which makes the response headers less readable.

I don't see this text when not redirecting.


Another way to see the effects:

Table doesn't appear:

curl -v http://somehost/somepage 2>&1

Table appears:

curl -v http://somehost/somepage 2>&1 | cat

1) How come this shows up only with certain types of redirects?

2) What's the neatest way to suppress it?

Thank you

67

Try this:

curl -vs -o /dev/null http://somehost/somepage 2>&1

That will suppress the progress meter, send stdout to /dev/null and redirect stderr (the -v output) to stdout.

| improve this answer | |
  • 40
    Thanks, -s was the key! – Ian Mackinnon Aug 7 '10 at 17:59
  • 7
    @IanMackinnon Note that with -s but without -v you will not see errors such as failure to connect. For that you should also add -S (or --show-error) as in mhoydis's answer. – Artyom Apr 14 '14 at 9:47
  • but why does the progress bar only appear in the first place when redirecting? I ran into this same issue when piping the output of curl to jq. No progress bar without piping to jq, then when piping to jq I have to go back and add -s. – sixty4bit Jul 29 '19 at 17:54
  • @sixty4bit: That's a design choice the developers made. The program can detect when its STDOUT isn't a tty. When output is not being piped, you don't want progress information to be interspersed with normal output, which you can see and have some idea of progress. When output is redirected or piped, you can't see it so you have no gauge for progress - unless the progress bar is turned on. – Paused until further notice. Jul 29 '19 at 20:18
27
curl --fail --silent --show-error http://www.example.com/ > /dev/null

This will suppress the status dialog, but will otherwise output errors to STDERR.

user@host:~# curl http://www.yahoo.com > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  254k    0  254k    0     0   403k      0 --:--:-- --:--:-- --:--:--  424k

The above outputs the status table when redirecting.

user@host:~# curl --fail --silent --show-error http://www.yahoo.com > /dev/null

The above suppresses the status table when redirecting, but errors will still go to STDERR.

user@host:~# curl --fail --silent --show-error http://www.errorexample.com > /dev/null
curl: (6) Couldn't resolve host 'www.errorexample.com'

The above is an example of an error to STDERR.

user@host:~# curl -v --fail --silent --show-error http://www.errorexample.com > ~/output.txt 2>&1
user@host:~# cat ~/output.txt 
* getaddrinfo(3) failed for www.errorexample.com:80
* Couldn't resolve host 'www.errorexample.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'www.errorexample.com'

Just add 2>&1 to the end to redirect STDERR to STDOUT (in this case, to a file).

| improve this answer | |
6

According to man curl:

-s, --silent : Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute.

Example usage:

curl -s 'http://www.google.com'

or if you want to capture the HTTP-BODY into a variable in bash

BODY=$( curl -s 'http://www.google.com' )
echo $BODY

You can use -s or --silent interchangeably.

| improve this answer | |
5

With reference to question 1 (how cURL knows to only display the table when output is redirected), I didn't realise a program could tell its outputs were being directed, but it seems on POSIX systems there is a function isatty which reports whether or not a file descriptor refers to a terminal.

| improve this answer | |
  • 2
    Here's a Bash snippet: [[ -p /dev/stdout ]] && echo "stdout is to a pipe"; [[ -t 1 ]] && echo "output to terminal"; [[ ! -t 1 && ! -p /dev/stdout ]] && echo "output redirected" – Paused until further notice. Aug 7 '10 at 20:41
2

1) How come this shows up only with certain types of redirects?

from the curl man page

If you want a progress meter for HTTP POST or PUT requests, you need to redirect the response output to a file, using shell redirect (>), -o [file] or similar.

curl must use isatty to determine the redirect and prints the progress meter when redirected to a file or shell pipe.

2) What's the neatest way to suppress it?

from the curl man page

-s, --silent

Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

| improve this answer | |
1

To put real error messages somwhere, you should write strerr into a log file. Something like that:

curl  "http://domain.name/process" --stderr /var/log/curl_err.log > /dev/null
| improve this answer | |
1

Modern versions of curl now have the option --no-progress-meter which disables only the progress meter you are referencing.

| improve this answer | |
  • 1
    According to the cURL manpage, this feature was added in 7.67.0 which was released on November 6 2019. – siikamiika Jul 27 at 9:24
0

Being behind a proxy I use a command like this.

date -s "$(curl --proxy http://PROXY:8080 -s http://google.com --head -s |grep Date|sed 's/Date: //g')"

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.