Dev tools and workflow · Beginner · 18 min
How to Process JSON on the Command Line with jq
By Padding · Updated 15 Nov 2026
Ubuntu 26.04.1 LTS · jq 1.8 (Ubuntu package, 1.8.1)
You will install jq 1.8, read values out of a JSON file, filter and reshape arrays, total and group data, turn JSON into CSV, pass shell values in safely, change a document, work with files of one JSON object per line, and use jq's exit status in scripts.
On this page
APIs, logs and configuration files are full of JSON, and JSON is hard to read when it arrives as one long line. jq is a small command-line tool that pretty-prints JSON and lets you pick it apart with short filters: take one field, keep only the items that match, reshape the result, add up numbers. It works in a pipe like grep and sort do, so it fits into shell scripts.
You will practise on a small file of shop orders that you create yourself, so every output below is one you can reproduce. Nothing here needs the network once jq is installed. Ubuntu 26.04's package is jq 1.8.1, which is the 1.8 series this tutorial is written for.
Step 1 · Install jq
sudo apt update
sudo apt install -y jq
jq --versionOn macOS, brew install jq installs it. jq --version prints jq-1.8.1 on Ubuntu 26.04.
Step 2 · Create some data to work on
Make a throwaway folder and a file with three orders. Each order has a customer, a status, a list of items and an optional note:
mkdir -p ~/jq-demo
cd ~/jq-demo
cat > orders.json <<'JSON'
{
"store": "Corner Shop",
"orders": [
{"id": 1001, "customer": {"name": "Ada", "email": "ada@example.com"}, "status": "shipped",
"items": [{"sku": "TEA-01", "qty": 2, "price": 4.5}, {"sku": "MUG-02", "qty": 1, "price": 8.0}],
"note": null},
{"id": 1002, "customer": {"name": "Grace", "email": "grace@example.com"}, "status": "pending",
"items": [{"sku": "TEA-01", "qty": 1, "price": 4.5}],
"note": "Leave at the door"},
{"id": 1003, "customer": {"name": "Linus", "email": "linus@example.com"}, "status": "shipped",
"items": [{"sku": "MUG-02", "qty": 3, "price": 8.0}, {"sku": "SPN-03", "qty": 4, "price": 1.25}],
"note": null}
]
}
JSONStep 3 · Pretty-print and pick out values
The simplest filter is ., which means "the whole input". jq prints it indented and with colour in a terminal. This is the quickest way to make an unreadable API response readable:
jq . orders.json | head -5The output starts with {, then "store": "Corner Shop", and the orders array. A filter that starts with a dot and a name picks that field. Dots go deeper, and square brackets index an array, counting from 0:
jq '.store' orders.json
jq '.orders[0].customer.name' orders.json
jq '.orders | length' orders.jsonThese print "Corner Shop", "Ada" and 3. The strings keep their quotes, because jq prints JSON, and a JSON string has quotes. In scripts you usually want the text itself. -r (raw output) removes the quotes:
jq -r '.store' orders.jsonThis prints Corner Shop. The | inside a filter is jq's own pipe: it feeds the left result into the right. Always put the filter in single quotes, so the shell leaves |, $ and the brackets alone.
Check it worked
jq -r '.orders[0].customer.email' orders.jsonThe output is ada@example.com.
Step 4 · Go through every item of an array
.[] turns an array into a stream of its items, and the rest of the filter runs on each:
jq '.orders[] | .id' orders.json
jq -r '.orders[] | .customer.email' orders.jsonThe first prints 1001, 1002 and 1003 on separate lines, the second the three email addresses. A stream is not an array: jq prints each result separately. To collect results back into an array, wrap the expression in [ ]. You will see that in a moment.
Step 5 · Keep only what matches
select(condition) passes an item through when the condition is true and drops it otherwise:
jq '.orders[] | select(.status == "shipped") | .id' orders.json
jq -r '.orders[] | select(.customer.name | test("^[AG]")) | .customer.name' orders.jsonThe first prints 1001 and 1003. The second keeps customers whose name starts with A or G, using test, which matches a regular expression: it prints Ada and Grace. Comparison operators are ==, !=, <, >, <= and >=, and and, or and not combine conditions.
Step 6 · Build new objects
You can build a new shape from the old data by writing an object. {id, status} is shorthand for {id: .id, status: .status}, and any key can take an expression. -c prints each result on one line (compact), which suits streams of objects:
jq -c '.orders[] | {id, name: .customer.name, status}' orders.jsonThis prints:
{"id":1001,"name":"Ada","status":"shipped"}
{"id":1002,"name":"Grace","status":"pending"}
{"id":1003,"name":"Linus","status":"shipped"}Put [ ] around a stream to get one array. This computes each order's total, by multiplying quantity and price for each item and adding them:
jq '[.orders[] | {id, total: ([.items[] | .qty * .price] | add)}]' orders.jsonThe result is an array of three objects, with totals 17, 4.5 and 29. jq shows a number that has no fraction without the trailing .0.
Check it worked
jq '[.orders[] | [.items[] | .qty * .price] | add] | add' orders.jsonThe output is 50.5, the sum of the three order totals.
Step 7 · Count, group and sort
group_by sorts an array by a key and splits it into groups. map(f) applies f to each item and collects the results. Together they count orders by status:
jq '.orders | group_by(.status) | map({status: .[0].status, count: length})' orders.jsonThe result lists pending with a count of 1 and shipped with a count of 2. Inside map, .[0] is the first item of each group, and length is the size of the group.
The same works across items inside orders. Flatten all the items with [.orders[].items[]], group by SKU, and add up the quantities:
jq '[.orders[].items[]] | group_by(.sku) | map({sku: .[0].sku, qty: (map(.qty) | add)})' orders.jsonThe quantities are 4 for MUG-02, 4 for SPN-03 and 3 for TEA-01. Other helpers you will use often are sort_by(.field), unique, reverse, keys, min_by, max_by and join(", ") for strings:
jq '.orders | map(.status) | unique' orders.json
jq -r '.orders | sort_by(.customer.name) | reverse | map(.customer.name) | join(", ")' orders.jsonThe first prints ["pending","shipped"] across several lines, and the second prints Linus, Grace, Ada.
Step 8 · Handle missing values and null
A field that is not there gives null, not an error. The // operator supplies a default when the left side is null or false:
jq '.orders[0].missing' orders.json
jq '.orders[] | .note // "none"' orders.jsonThe first prints null. The second prints "none", "Leave at the door" and "none", because two orders have a null note.
Indexing the wrong type is an error. A name is a string, so asking for .first inside it fails. Add ? to quiet an error and produce nothing for that item:
jq '.orders[0].customer.name.first' orders.json || echo "jq failed with status $?"
jq '.orders[0].customer.name.first?' orders.jsonThe first command prints jq: error (at orders.json:14): Cannot index string with string "first" and then the status, 5. The second prints nothing and succeeds. Use ? sparingly. It also hides real mistakes in a filter.
Step 9 · Turn JSON into CSV
@csv formats an array as one CSV line, quoting strings, and @tsv does tab-separated values. Use -r, or jq would print the result as a quoted JSON string:
jq -r '.orders[] | [.id, .customer.name, .status] | @csv' orders.jsonThe output is:
1001,"Ada","shipped"
1002,"Grace","pending"
1003,"Linus","shipped"Add a header row by printing an array of column names first. A filter can hold more than one expression, separated by commas, and jq prints all their results in order:
jq -r '(["id","name","status"] | @csv), (.orders[] | [.id, .customer.name, .status] | @csv)' orders.jsonString interpolation builds text lines with \( ):
jq -r '.orders[] | "Order \(.id) for \(.customer.name): \(.items | length) line(s)"' orders.jsonThe first line is Order 1001 for Ada: 2 line(s).
Step 10 · Pass shell values into a filter
Never build a filter by pasting a shell variable into it. A value with a quote or a bracket in it will break the filter, and anyone who controls the value can change what the filter does. Use --arg for strings and --argjson for numbers and other JSON values. Inside the filter, they are variables that start with $:
status=pending
jq --arg s "$status" '.orders[] | select(.status == $s) | .customer.name' orders.json
jq --argjson min 2 '[.orders[].items[] | select(.qty >= $min) | .sku]' orders.json
jq -n --arg name "O'Brien" '{name: $name}'The first prints "Grace". The second prints an array with TEA-01, MUG-02 and SPN-03, the lines that have a quantity of 2 or more. The third, with -n (no input), builds an object from a value that holds a quote, and it prints "name": "O'Brien" correctly escaped. Building JSON with jq -n is the safe way to create a request body in a script.
Step 11 · Change a document
jq does not edit files in place. It reads, transforms and prints, and you redirect the result. Update an item with |=, which applies the filter on the right to the value on the left. Delete with del:
jq '.orders[0] | .items[0].price |= . + 1 | .items[0]' orders.json
jq 'del(.orders[].note) | .orders[0] | keys' orders.jsonThe first prints the first item of the first order with its price raised from 4.5 to 5.5. The second removes the note field from every order and prints the remaining keys of the first order: customer, id, items and status.
To save a change, write to a new file and move it into place after you check it. Do not redirect to the file you are reading, because the shell empties it before jq reads anything:
jq '.orders |= map(select(.status == "pending"))' orders.json > pending.json
jq '.orders | length' pending.jsonThe output is 1. orders.json still has all three orders.
Step 12 · Work with one JSON object per line
Logs and exports often come as JSON Lines: one object on each line, with no surrounding array. jq reads those as a stream of inputs. Make a file from the orders, then read it three ways:
jq -c '.orders[]' orders.json > orders.jsonl
jq '.id' orders.jsonl
jq -s 'length' orders.jsonl
jq -n '[inputs | .id]' orders.jsonlThe first command prints 1001, 1002 and 1003, because the filter runs on each line. -s (slurp) reads every input into one array first, so length is 3. -n with inputs does the same job without holding every line in an array, which matters for large files.
To read plain text lines, use -R. Each line becomes a JSON string, which is how you can turn a list of words into JSON:
printf 'a\nb\n' | jq -R .
printf 'a\nb\n' | jq -Rs 'split("\n") | map(select(length > 0))'The second prints ["a","b"] over several lines.
Step 13 · Use jq in scripts
When jq hits an error, such as input that is not valid JSON, it prints a message and exits with status 5. -e (--exit-status) makes the exit status reflect the last output: 0 when it is a value other than false or null, 1 when it is false or null, and 4 when there was no output at all. This lets a shell if test JSON:
if jq -e '.store == "Corner Shop"' orders.json > /dev/null; then
echo "store matches"
fi
jq -e '.orders[] | select(.status == "cancelled")' orders.json > /dev/null || echo "no cancelled orders (status $?)"
echo 'not json' | jq . 2>&1 || echo "status $?"The first prints store matches. The second prints no cancelled orders (status 4), because the filter produced no output. The third prints jq: parse error: Invalid numeric literal at line 1, column 4 and then status 5.
Fetching JSON from an API and filtering it in one line is the usual pattern, shown here against an example address that you replace. -s keeps curl quiet and -f makes it fail on an HTTP error, so jq is never fed an error page:
curl -sf https://api.example.com/items | jq -r '.[] | .name'Troubleshooting
jq: error: syntax error, unexpected ...is almost always a quoting problem. Wrap the whole filter in single quotes, and use double quotes only inside it for JSON strings.jq: error (at <file>:N): Cannot iterate over nullmeans you used.[]on a field that is missing ornull. Check the path with a plain filter first, or use.field[]?.jq: error: ... is not definedfor a function name usually means the name does not exist in the version you run. Checkjq --versionand the manual for that version.- The output has quotes around strings where you wanted plain text. Use
-rfor raw strings. - Big numbers change. jq can pass a large integer through unchanged, but any calculation turns it into a double-precision float, which cannot hold integers above 2^53 exactly.
Step 14 · Clean up
Remove the practice folder:
cd ~
rm -rf ~/jq-demojq stays installed. Remove it with sudo apt remove jq if you do not want it.
Next steps
- Fetch JSON from an API in code: call a web API from Python with httpx.
- Work with JSON in Python itself: read and write files and JSON in Python.
- Try your filters against an API you control: test and debug an API with curl.
Related tutorials
- How to Call a Web API from Python with httpxYou will install httpx 0.28, write a small local API to practise on, then call it from Python by fetching JSON, add query parameters and a reusable client, handle bad statuses, timeouts and network failures, send JSON in a POST request, and make several requests at once with async code.
- How to Read and Write Files and JSON in PythonYou will read and write text files with pathlib, convert Python data to JSON and back, and use both to build a small task list that keeps its data in a JSON file and survives a damaged file.
Need this done for you?
Padding does this work for growing companies, from the first diagnosis to a system that keeps working.
Get new tutorials