Building a JSON string with shell variables, explained.

If you’re trying to build JSON using variables in your shell script you have no doubt seen the stackoverflow example. Here’s what it all means.

The example on stackoverflow (by chepner)

BUCKET_NAME=testbucket
OBJECT_NAME=testworkflow-2.0.1.jar
TARGET_LOCATION=/opt/test/testworkflow-2.0.1.jar

JSON_STRING=$( jq -n \
                  --arg bn "$BUCKET_NAME" \
                  --arg on "$OBJECT_NAME" \
                  --arg tl "$TARGET_LOCATION" \
                  '{bucketname: $bn, objectname: $on, targetlocation: $tl}' )

explanation

This was super confusing to me for a bit because I didn’t get all the repetition of identifiers for the same thing and duplicating the variable names and such. However, something else is going on here and perhaps this example makes it look more confusing than necessary.

The last line enclosed within curly braces is the template into which we will inject our shell variables. That ‘$bn’ there is NOT a shell variable, but a jq value. jq is not just a json utility, it’s also a “functional” programming language. Unfortunately it also seems to denote things that hold a value with $. With --arg we’re supplying a parameter called ‘bn’ and telling jq to place the value of that parameter where it sees ‘$bn’. The arg name ‘bn’ matches ‘$bn’.

In the example you have two sets of names and values. One is the shell script variables and their values, the other is the jq language identifiers and their values. However, in such a script they actually all refer to the same pairs.

The following will do the same thing.

JSON_STRING=$( jq -n \
                  --arg BUCKET_NAME "$BUCKET_NAME" \
                  --arg OBJECT_NAME "$OBJECT_NAME" \
                  --arg TARGET_LOCATION "$TARGET_LOCATION" \
                  '{bucketname: $BUCKET_NAME, objectname: $OBJECT_NAME, targertlocation: $TARGET_LOCATION}' )

Note that the template is enclosed in single quotes, meaning there is no risk of conflict because the shell-variable-looking template values won’t be expanded.

The person who posted the example introduced new names to distinguish the template and arg stuff from the shell script stuff but I find it easier to keep track this way.

summary

Unless you plan to write actual jq programs, it helps to see the names in the template that start with $ as just placeholders. That would be like {% BUCKET_NAME %} in some template languages. They could have picked a different symbol or method to distinguish those but there are only so many characters to choose from and jq is a language on its own and templates are not the only place these are used.

You can actually use jq accessors and arbitrary strings as template variables in jq. See the link below for details on their wiki page.

On the same stackoverflow answer, Nicolas Rouqette provides a very good explanation which I only got, after I got what was going on by myself.

Normally, jq processes input; but with -n, there is none. --arg <var> <string> replaces $<var> with the string value <string>. --argjson <var> <json> replaces $<var> with the Json blob <json>. The combination of --arg and --argjson provides powerful mechanisms to construct arbitrary Json structures.


495 Words

2021-09-21 00:00 +0000

Push TrueNAS/FreenAS alerts to Alerta

FreeNAS (and TrueNAS) keep track of system alerts and can be configured to send those out via email and a few other services such as Slack and InfluxDB. The selection of these services is pretty limited. Especially limiting is the fact that it does not provide a way to call a custom script or a web service endpoint. But we can capture the alerts from the system and get them out in a roundabout way.

Use FreeNAS middleware client to push alerts from Freenas to Alerta

TrueNAS/FreeNAS does not have a plugin for Alerta, nor does it have a facility to use shell scripts or http/json for alerts.

I wrote a simple script that takes the alerts from the FreeNAS database using the included middleware client midclt and pushes them to the Alerta endpoint using curl, making some assumptions along the way.

Currently the script just invokes FreeNAS provided CLI tool midclt to get the info about ALL alerts without keeping track of their status and pushes them into Alerta. Alerta can take care of duplicates so this is not a big issue.

The freenas-alerta script uses midclt, jq and wget which are all included on a TrueNAS install.

Without going into details, what you have to do is to put freenas-alerta.sh and .secrets in Add FreeNAS/TrueNAS Cron task as:

cd /<PATH TO SOME LOCATION>/bin/cron && /bin/bash ./freenas-alerta.sh

The script is available on GitHub at the url below. You can modify it to do something else rather than pushing the alerts to Alerta.