How to Bash Loop Over JSON with jq

A common task I run into at work is I’ll need to iterate over some JSON in Bash. This is very easy to do in Python, but can be kind of annoying in Bash because sometimes your JSON is spread out over multiple lines, or sometimes your JSON values have whitespace or other special characters that get in your way when using Bash. In this post I’ll show some examples of how I handle it using jq.

I’ll use lists-of-dicts in this post, but the same idea can apply to lists of strings or whatever.

A simple example

Suppose I have some JSON like this:

 1[
 2  {
 3    "name": "Alice",
 4    "color": "Blue"
 5  },
 6  {
 7    "name": "Bob",
 8    "color": "Green"
 9  }
10]

I can loop over it in Bash using jq like this:

1for ITEM in $(cat /tmp/my_data.json | jq -c .[])
2do
3  NAME="$(echo $ITEM | jq -r .name)"
4  COLOR="$(echo $ITEM | jq -r .color)"
5  echo "The color for $NAME is $COLOR"
6done
7
8The color for Alice is Blue
9The color for Bob is Green

The trick seen above which makes this work is the -c (compact) output of jq, which outputs each dict into its own line, which plays nicely with bash loops.

Another example with whitespace and special characters

What if my values have whitespace and special characters? Those won’t play nicely with the above loop and you’ll get a bunch of jq parsing errors if you use the simple example above.

 1[
 2  {
 3    "name": "Alice",
 4    "greeting": "You owe me $5 please!"
 5  },
 6  {
 7    "name": "Bob",
 8    "greeting": "You owe me $5 & also 50% of lunch from yesterday!"
 9  }
10]

When you have messy strings like that, you can base64 encode the JSON as you go into the loop, then base64 decode each item as you use them, which takes care of the messy characters.

1for ITEM in $(cat /tmp/my_data.json | jq -r '.[] | @base64')
2do
3  NAME="$(echo $ITEM | base64 --decode | jq -r .name)"
4  GREETING="$(echo $ITEM | base64 --decode | jq -r .greeting)"
5  echo "Hello $NAME, $GREETING"
6done
7
8Hello Alice, You owe me $5 please!
9Hello Bob, You owe me $5 & also 50% of lunch from yesterday!