xpufx Building a JSON string with shell variables, explained.
If you’re trying to build JSON using variables in your shell script you have no doubt seen the stackoverflow example. Here’s what it all means.
The example on stackoverflow (by chepner)
BUCKET_NAME=testbucket
OBJECT_NAME=testworkflow-2.0.1.jar
TARGET_LOCATION=/opt/test/testworkflow-2.0.1.jar
JSON_STRING=$( jq -n \
--arg bn "$BUCKET_NAME" \
--arg on "$OBJECT_NAME" \
--arg tl "$TARGET_LOCATION" \
'{bucketname: $bn, objectname: $on, targetlocation: $tl}' )
explanation
This was super confusing to me for a bit because I didn’t get all the repetition of identifiers for the same thing and duplicating the variable names and such. However, something else is going on here and perhaps this example makes it look more confusing than necessary.
The last line enclosed within curly braces is the template into which we will inject our shell variables. That ‘$bn’ there is NOT a shell variable, but a jq value. jq is not just a json utility, it’s also a “functional” programming language. Unfortunately it also seems to denote things that hold a value with $. With --arg we’re supplying a parameter called ‘bn’ and telling jq to place the value of that parameter where it sees ‘$bn’. The arg name ‘bn’ matches ‘$bn’.
In the example you have two sets of names and values. One is the shell script variables and their values, the other is the jq language identifiers and their values. However, in such a script they actually all refer to the same pairs.
The following will do the same thing.
JSON_STRING=$( jq -n \
--arg BUCKET_NAME "$BUCKET_NAME" \
--arg OBJECT_NAME "$OBJECT_NAME" \
--arg TARGET_LOCATION "$TARGET_LOCATION" \
'{bucketname: $BUCKET_NAME, objectname: $OBJECT_NAME, targertlocation: $TARGET_LOCATION}' )
Note that the template is enclosed in single quotes, meaning there is no risk of conflict because the shell-variable-looking template values won’t be expanded.
The person who posted the example introduced new names to distinguish the template and arg stuff from the shell script stuff but I find it easier to keep track this way.
summary
Unless you plan to write actual jq programs, it helps to see the names in the template that start with $ as just placeholders. That would be like {% BUCKET_NAME %} in some template languages. They could have picked a different symbol or method to distinguish those but there are only so many characters to choose from and jq is a language on its own and templates are not the only place these are used.
You can actually use jq accessors and arbitrary strings as template variables in jq. See the link below for details on their wiki page.
On the same stackoverflow answer, Nicolas Rouqette provides a very good explanation which I only got, after I got what was going on by myself.
Normally, jq processes input; but with -n, there is none. --arg <var> <string> replaces $<var> with the string value <string>. --argjson <var> <json> replaces $<var> with the Json blob <json>. The combination of --arg and --argjson provides powerful mechanisms to construct arbitrary Json structures.