0

I have an application that connects to BigQuery using multiple creds from .json file. Being that this app will eventually reside on Pivotal Cloud Foundry and will need to be set as User Provided Environment Variable I have used base64 encryption to encode the contents of the .json file into a base64 key.

The idea is I can then use the key as the value of the environment variable so I can pull in all BigQuery API connection info from .json file into one environment variable - using base64 key, instead of having many key=value pair variables to accomplish this.

Note: The base64 key has 3,165 characters - including new line characters.

For reference a short snippet of base64 key is:

nROdXlnWnVmb1k0UnVqTFJlcjgrckZJRlpZdVxuYUFsMklsWk9MUjd3cmRzL0RDZytYdGJGYUp4MzlRaWZHaEp6b2hyblNvbUpyY3ExL2d4aVFYYlplUUtCZ0hja1xua0hqL1M0Wkk0d0dWTzJreEN4QzduYnB1STVQRTRWczRHMjE0RkpldnhPR1hDQ3phOFB5bGJPbm5KUFdZZUJoT1xuV3IwMzduazd5eXpja0p6cE\
p0R0hlNkZLNFJNcHFIRHBmU0pMOE80T2YwWi9UYUZ3ZTM4bE9XbWc1Q2NLUC9XUVxuUjhYdlgzS25UZXRQUWxVeXJwK1BNVHNNdG4yVnBad1hMZDNJTExKYkFvR0JBS1FaODdVbEE1S0kwV3JqVnRic1xuaXJ6UkdDaGhVWE90SXVDT09zWlQyOXNpeFVoQmZka1JzTFpCcnB6VkVWektXeGhBY05lRERRbTJkVzBta09\
1elxuTkNVUlBpbGZYT0pLRkdNQ2lOdkIyZW5OMDVBK2ExdVdOWFFkclQzRG10OG5MS3FFYnp6VTNheXIweWVsc0tKeVxuNE9BVExMK3M5M29oRFV6WEhRZlVtbnNMXG4tLS0tLUVORCBQUklWQVRFIEtFWS0tLS0tXG4iLAogICJjbGllbnRfZW1haWwiOiAidG1wLWFkLWxvY2tvdXRAaW8xLWRhdGFsYWtlLXZpZXdz\

I know this can be set on macOS as system variable (I've seen it and application makes connections and works correctly) but I am not able to do this on Windows 10.

I'm aware of the max length constraint for system variables on Windows but haven't came across a solution to be able to set this using "GOOGLE_JSON_B64" as environment variable name and then the 3,165 key as the environment variable value.

Does anyone have a workaround?

Any insight would be greatly appreciated.

  • base64 is not encryption... the key you just partially shared should not be used again. – Attie Sep 10 '18 at 20:48
0

Microsoft's documentation says that an environment variable on Windows is only limited to 32,767 characters (link). It does not say how to create a very long variable.

The problem here is that the tools that Windows provides all have their limits :

The set command and the setx command truncate values to 1023 characters.

One can set environment variables directly in the registry at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment, but using regedit for that will fail because entered strings via the keyboard are limited to 2047 characters.

As far as I can see, the only remaining option is to write a small program that updates directly the registry. The Windows API function to use is SetEnvironmentVariable, whose documentation says :

The maximum size of a user-defined environment variable is 32,767 characters.

One hopes that this, at least, is correct. You may run here into the maximal line-length that is accepted by the compiler, but in a program you are able to construct such a long string in steps.

  • Why do you need these new line characters? – harrymc Sep 11 '18 at 13:02
  • The new line characters are part of the key. The base64 encoding represents the .json file, which has several lines/ components in it. Meaning, the .json file isn't just one key/value pair, it is several lines of creds, to include service account, service account pword, private key id, private key, client email, client id, and more. So each new line character in the key represents a key/value pair in the .json file. All parts of the json file/ key with new line characters are needed to make said connection to BigQuery. – user3216506 Sep 11 '18 at 15:17
  • When I collapsed the key by removing all white-spaces and re-ran set command I noticed it was able to store more of the key but not all. It seems to of reached a limit of 1,023 characters in total. – user3216506 Sep 11 '18 at 15:19
  • This area is a big mess - I rewrote my answer. – harrymc Sep 11 '18 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.