0

The general form of a curl command in CMD that works is:

curl -o latest.dump -L "https://s3.amazonaws.com/hkpgbackups/app28197640@heroku.com/xxxx.dump?AWSAccessKeyId=AKIAJSCBEZJRDOTGNGZQ&Expires=1411489500&Signature=%2F5zwQNZNN0H2XSR4wSqQ%2FFExBdI%3D"

But the url is dynamically generated so for me to put this into a .bat script. OK, no problem so here's my .bat script that on paper should work.

@echo off

for /f "delims=" %%a in ('heroku pgbackups:url') do @set this_url=%%a
ECHO %this_url%
curl -v -o latest.dump -L %this_url%

But the output bails because at the first '&' as you might expect. The echo of this_url shows the full string with quotes. How can I append, or force, to encapsulate this string with quotes?

I have attempted but it returns the same type of error:

cmd /k ""curl -v -o latest.dump -l" "('heroku pgbackups:url')""

Some may ask why not do this in Unix, or CYGWIN? I did that first. The command works perfectly on my OS X personal machine. So it should work in CYGWIN? Well I get the same type of errors.

I've attempted incorporating a Perl script using the uri_escape library and inserted it. But the URL it passes is not recognized by curl.

Forcing the output within single quotes munges the URL in that it starts correct but the closing single quote is placed within the last few characters of the URL which renders it useless to curl.

The output of heroku pgbackups:url in the CMD console is:

"https://s3.amazonaws.com/hkpgbackups/app28197640@heroku.com/b080.dump?AWSAccess
KeyId=AKIAJSCBEZJRDOTGNGZQ&Expires=1411657655&Signature=MqzqBitAN7MADiLAsdAN4ZQk
RH0%3D"

The output of the .bat file above is:

https://s3.amazonaws.com/hkpgbackups/app28197640@heroku.com/b080.dump?AWSAccessK
eyId=AKIAJSCBEZJRDOTGNGZQ
'Expires' is not recognized as an internal or external command,
operable program or batch file.
'Signature' is not recognized as an internal or external command,
operable program or batch file.
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 205.251.243.66...   % Total    % Received % Xferd  Average Speed   Ti
me    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0co
nnected
* Connected to s3.amazonaws.com (205.251.243.66) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using AES128-SHA
* Server certificate:
*        subject: C=US; ST=Washington; L=Seattle; O=Amazon.com Inc.; CN=s3.amazo
naws.com
*        start date: 2014-04-12 00:00:00 GMT
*        expire date: 2015-04-13 23:59:59 GMT
*        subjectAltName: s3.amazonaws.com matched
*        issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of
use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA
- G3
*        SSL certificate verify result: unable to get local issuer certificate (
20), continuing anyway.
> GET /hkpgbackups/app28197640@heroku.com/b080.dump?AWSAccessKeyId=AKIAJSCBEZJRD
OTGNGZQ HTTP/1.1
> User-Agent: curl/7.21.7 (amd64-pc-win32) libcurl/7.21.7 OpenSSL/0.9.8r zlib/1.
2.5
> Host: s3.amazonaws.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< x-amz-request-id: 4566C5AB208C1248
< x-amz-id-2: zF/zr+fbR/pE7nvF7vvUmOdZQeMzjSBI6SPLKH14LGAI5JAb2xyoLhuuDGBDhqcq
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 25 Sep 2014 14:51:46 GMT
< Server: AmazonS3
<
{ [data not shown]
100   231    0   231    0     0    592      0 --:--:-- --:--:-- --:--:--   704
* Connection #0 to host s3.amazonaws.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]
'Expires' is not recognized as an internal or external command,
operable program or batch file.
'Signature' is not recognized as an internal or external command,
operable program or batch file.

The 403 error indicates that the url got munged, but otherwise it passed through or I would have gotten a 400 error. For some reason in the script, the "&" get redirected because they're interpreted in the script.

If you were me, how would you run this command dynamically pass this URL so it's not interpreted by CMD, bash, etc? thanx, sam

  • Can you show the result of the ECHO command that looks OK, yet the value fails when used with curl? Something doesn't make sense. If you can ECHO the value, and it is enclosed within quotes, then it should work with curl as well. – dbenham Sep 23 '14 at 21:41
  • Yes, something doesn't make sense. The ECHO %this_url% returns "s3.amazonaws.com/hkpgbackups/app28197640@heroku.com/…". – sam452 Sep 24 '14 at 11:25
  • I don't understand how "The output of the curl --verbose option is: ..." relates to your posted code. What is the exact output of your originally posted batch script? – dbenham Sep 25 '14 at 0:49
0

Looking at your originally posted script:

@echo off

for /f "delims=" %%a in ('heroku pgbackups:url') do @set this_url=%%a
ECHO %this_url%
curl -v -o latest.dump -L %this_url%

If the output of ECHO %this_url% is exactly as you posted, with quotes around it, then your existing batch script should work.

If the value of %this_url% does not have quotes, then you should add quotes to the curl command:

curl -v -o latest.dump -L "%this_url%"

You could modify the command to make it work regardless whether %this_url% contains quotes by removing any existing quotes, and explicitly adding your own:

curl -v -o latest.dump -L "%this_url:"=%"

The other command you list looks completely wrong to me:

cmd /k ""curl -v -o latest.dump -l" "('heroku pgbackups:url')""

The outer quotes should not be needed, nor should the quotes around "curl -v -o latest.dump -l". You could put quotes around the exe file only "curl", but that should not be needed unless you include the path to curl and the path includes spaces or poison characters.

I simply do not understand the "('heroku pgbackups:url')" syntax at all.

  • Yes, thank you for walking me through this. The output of the command heroku pgbackups:url yields the extra long url. I've attempted to execute your command to explicitly add my own quotes yields the same result as above: for each "&" in the command the parser is reading each as a separate command and curl fails. It turns out each is part of the Amazon authentication baked in the URL. I had attempted to capture each of those portions into separate variables and concatenating them in the command but for some reason it will only concatenate one variable. – sam452 Sep 29 '14 at 16:34
  • The problem is that for this long URL Windows is adding a carriage return which munges the URL, at least in cygwin. So I can get it to run in cygwin piping the output of the heroku command to dos2unix. As for getting it to work in CMD, .BAT there's probably an answer but it works now in cygwin. – sam452 Sep 29 '14 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.