2

So, in-order to send a POST request to gists (github), you can do something like this as seen in https://gist.github.com/caspyi...

curl --user "user" -X POST --data '{"description":"Created via API","public":"true","files":{"file1.txt":{"content":"Demo"}}' https://api.github.com/gists

But, in the above example, the file name and the content of the file are hard coded which is the part ..file1.txt":{"content":"Demo"}..

I am replacing the above part with my variable $file":{"content":"$content"} but initialize the variable, the json request must be enclosed in double quotes, which I did as

curl --user "user" -X POST --data "{\"description\":\"Created via API\",\"public\":\"true\",\"files\":{\"$file\":{\"content\":\"$content\"}}' https://api.github.com/gists

But this does not work, I get json error.

{
  "message": "Problems parsing JSON",
  "documentation_url": "https://developer.github.com/v3/gists/#create-a-gist"
}

Even, if I replaced all the escaped double-quotes with \' single quotes.

Does anyone know how to include a variable inside this json request? btw: I have used all headers such as

 -H "Content-Type: application/json; charset=UTF-8" 

and many combinations to validate the request but to no avail

UPDATE.

This is what the entire content looks like.

function gist_controller(){
    content=$(cat $1)
    DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
    read -p "enter your password - " pass 
    public="false"
    gist_content=$(cat $1)
    curl --user "samserayo"  -H "Content-Type: application/json; charset=UTF-8" -X POST -d  "{ 'description': 'Created via API', 'public': 'true', 'files':{ ' '$1 ':{ 'content': '$gist_content'}}" https://api.github.com/gists
}
UPDATE 2

The file that causes the script to break (the file I am trying to upload is)

<?php echo 'hello world' ?>
4
  • This is almost certainly better asked on Super User, as I see no real connection to server or network administration. You can flag it and request migration if you want the question moved there.
    – user
    Apr 18, 2016 at 20:15
  • @MichaelKjörling Thanks, flagged it. Though, I think curl request has certainly something to do with server administration.
    – samayo
    Apr 18, 2016 at 20:18
  • Problems with quotes and escaping are the most basic of all programming problems (syntax errors), the question should be migrated to SO.
    – Alexander
    Apr 18, 2016 at 21:11
  • You are missing ; after echo 'hello world'
    – Daniel W.
    Mar 3, 2020 at 15:28

5 Answers 5

1

You don't show what value you have set for $file or $content. What do those both expand out to? It's likely one, or the other, or both, contain characters the JSON parser doesn't like. Most likely, something inside the variable expansion value needs to be further escaped or encoded before submission to the JSON parser through curl.

Have you EXPORTed the environment variables?

Try:

echo "$file $content";

to make sure they are both set as you expect them to be.

2
  • Sorry, updated the content.
    – samayo
    Apr 19, 2016 at 6:55
  • updated to include an example
    – samayo
    Sep 1, 2019 at 10:31
1

You didn't specify what shell you are using, but anything within double-quotes is subject to shell filename expansion. In particular, {} has meaning to most shells. That's the reason that the example you cited used single quotes to wrap the whole thing.

Shell escapes are tricky, and sometimes it seems impossible to get what you want. Honestly, in cases like this I would write a Python script to call curl, where I know the shell isn't "helping" me out.

1
  • Sorry, updated the content. I am using bash shell.
    – samayo
    Apr 19, 2016 at 6:55
1

Since this got bumped for review, here's another answer.

Escape the parts separately (close the first escape, then escape your variable):

curl --user "user" -X POST --data '{"description":"Created via API","public":"true","files":{"'"${file}"'":{"content":"'"${content}"'"}}}' https://api.github.com/gists

or split up for readability:

'{"description":"Created via API","public":"true","files":{"'
"${file}"
'":{"content":"'
"${content}"
'"}}}' 

You may or may not also have to change the internal field separator, so the shell doesn't interpret any whitespace in your variables itself.

MWE: cat /tmp/myfile | ./thisscript.sh "myfilename.txt"

#!/bin/sh

FILENAME="${1}"
#CONTENT="$(cat)"
CONTENT="$(sed -e 's/"/\\"/g')" # escape stuff

OFS="${IFS}"
IFS=''

PERSONAL_ACCESS_TOKEN="e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e"

curl --user "username:${PERSONAL_ACCESS_TOKEN}" -X POST --data '{"description":"Created via API","public":"true","files":{"'"${FILENAME}"'":{"content":"'"${CONTENT}"'"}}}' https://api.github.com/gists
IFS="${OFS}"

For this to work correctly, you will of course still need to escape all the characters that would break JSON, in your file contents, such as " and control characters (DOS carriage-returns), etc.
It is ...unfortunate, that they decided to use JSON datastructures for file content uploads.


If I may suggest a different way of creating Gists, use the curl API call only to create a single file, say README with some boilerplate content (can't be empty).
Then grep the returned json structure for git_push_url and clone the gist git repo. After that you can just git commit, git push stuff to the gist without worrying about binary data or escaping issues.

3
  • thanks, but this didn't work. same error "problem parsing json" I will add a bounty, can't still no answer after 3 years
    – samayo
    Aug 31, 2019 at 15:13
  • Actually, your code made it worse. Before I could upload gists in text file without quotes, not I can't upload anything
    – samayo
    Aug 31, 2019 at 15:16
  • @samayo, sorry about that, I just copied your JSON example and it was missing a closing bracket } at the end. Edited, and added an alternative method.
    – nyov
    Sep 1, 2019 at 6:52
0

You have to put the entire string inside double quotes ". And you need to escape quotes inside the string (not convert them into other types of quotes!).

So instead of:

"{ 'description': 'Created via API', 'public': 'true', 'files':{ ' '$1 ':{ 'content': '$gist_content'}}"

You want:

"{ \"description\": \"Created via API\", \"public\": \"true\", \"files\" : { \"$1\" : { \"content\": \"$gist_content\"}}}"

You should probably test with echo and make sure the string is exactly correct.

3
  • Nope, this is the first thing I did, before going crazy. It does not work like that. I always get an error as I mentioned above "message": "Problems parsing JSON",
    – samayo
    Apr 19, 2016 at 11:56
  • @samayo Paste the exact message you sent. Either it's legal JSON or it isn't. (Make sure each { has a matching }.) Apr 19, 2016 at 16:47
  • added a bounty 100, still none of the answer work
    – samayo
    Aug 31, 2019 at 15:15
0

Here is a solution but it assumes two things.

That the file is a filename provided to the script as an first argument. And the content is the file content as text.

Also please note different POST URL than Yours (see below for explanation).

#!/bin/bash

file=$1
content=$(cat $1)

curl -H "Content-Type: application/json; charset=UTF-8" -X POST -d  "{ 'description': 'Created via API', 'public': 'true', 'files':{ ' '${file}':{ 'content': '${content}'}}" https://postman-echo.com/post

content.txt file contains:

This is content of the content file.

Example run:

./curl.sh content.txt

Example output:

{
  "args": {},
  "data": "{ 'description': 'Created via API', 'public': 'true', 'files':{ ' 'content.txt ':{ 'content': 'This is content of the content file.'}}",
  "files": {},
  "form": {},
  "headers": {
    "x-forwarded-proto": "https",
    "host": "postman-echo.com",
    "content-length": "134",
    "accept": "*/*",
    "content-type": "application/json; charset=UTF-8",
    "user-agent": "curl/7.65.3",
    "x-forwarded-port": "443"
  },
  "json": null,
  "url": "https://postman-echo.com/post"
}

NOTES:

I am using here the https://postman-echo.com/post website which echoes everything that You post to it as JSON.

If the content of the content.txt file is more complicated this solution may break as it still will have to be properly escaped. The content is being put to the "content" field so all escaping rules for json applies here.

To see the response beautifully displayed (as above json) add at the end of the curl | jq . (You may need to install it first) :

https://postman-echo.com/post | jq . 

It is possible to use jq to automatically quote entire content file (if needed for more advanced usage):

$ jq -Rs '.' content.txt
"This is just a text.\n"

$ jq -Rs '.' content.cpp
"#include <iostream>\nusing namespace std;\nint main() \n{\n    cout << \"Hello, World!\";\n    return 0;\n}\n"

You will have to slightly modify the script curl.sh for it to work :

content=$(jq -Rs '.' $1)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.