Tools: jq Magic: Parse JSON Like a Pro

Tools: jq Magic: Parse JSON Like a Pro

TIL: jq Magic: Parse JSON Like a Pro

installation

basic usage

pretty print JSON

extract a field

array operations

get first element

get last element

get array length

extract field from all array items

real-world examples

1. parse docker images

2. get all pod names in kubernetes

3. extract specific AWS EC2 info

4. parse package.json dependencies

5. get GitHub API data

filtering

filter array items

multiple conditions

check if field exists

transforming data

create new object

rename fields

add field

sorting

grouping

useful one-liners

count items by type

sum values

get unique values

find min/max

advanced — CSV output

advanced — nested data

practical scripts

check all service status

parse AWS cost report

monitor log errors

debug jq expressions

common patterns I use

1. pretty print and save

2. extract and process

3. combine multiple JSON files

4. update JSON file in-place

the gotcha

cheat sheet jq is like sed for JSON. Once you learn it, you'll wonder how you ever lived without it. Output is now colored and formatted. -r = raw output (no quotes) Use jq playground: https://jqplay.org/ Or test step by step: Remember to use -r for raw output when you want to use the result in bash: jq has completely changed how I interact with APIs and JSON data. No more manual parsing or Python scripts for simple tasks. Go forth and parse JSON. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Command

Copy

# Mac -weight: 500;">brew -weight: 500;">install jq # Ubuntu/Debian -weight: 500;">apt-get -weight: 500;">install jq # CentOS/RHEL -weight: 500;">yum -weight: 500;">install jq # Mac -weight: 500;">brew -weight: 500;">install jq # Ubuntu/Debian -weight: 500;">apt-get -weight: 500;">install jq # CentOS/RHEL -weight: 500;">yum -weight: 500;">install jq # Mac -weight: 500;">brew -weight: 500;">install jq # Ubuntu/Debian -weight: 500;">apt-get -weight: 500;">install jq # CentOS/RHEL -weight: 500;">yum -weight: 500;">install jq # Ugly JSON from API -weight: 500;">curl https://api.example.com/data | jq '.' # Ugly JSON from API -weight: 500;">curl https://api.example.com/data | jq '.' # Ugly JSON from API -weight: 500;">curl https://api.example.com/data | jq '.' echo '{"name": "John", "age": 30}' | jq '.name' # "John" # Remove quotes echo '{"name": "John", "age": 30}' | jq -r '.name' # John echo '{"name": "John", "age": 30}' | jq '.name' # "John" # Remove quotes echo '{"name": "John", "age": 30}' | jq -r '.name' # John echo '{"name": "John", "age": 30}' | jq '.name' # "John" # Remove quotes echo '{"name": "John", "age": 30}' | jq -r '.name' # John echo '[1, 2, 3, 4, 5]' | jq '.[0]' # 1 echo '[1, 2, 3, 4, 5]' | jq '.[0]' # 1 echo '[1, 2, 3, 4, 5]' | jq '.[0]' # 1 echo '[1, 2, 3, 4, 5]' | jq '.[-1]' # 5 echo '[1, 2, 3, 4, 5]' | jq '.[-1]' # 5 echo '[1, 2, 3, 4, 5]' | jq '.[-1]' # 5 echo '[1, 2, 3, 4, 5]' | jq 'length' # 5 echo '[1, 2, 3, 4, 5]' | jq 'length' # 5 echo '[1, 2, 3, 4, 5]' | jq 'length' # 5 echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[].name' # "Alice" # "Bob" # Or use map echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq 'map(.name)' # ["Alice", "Bob"] echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[].name' # "Alice" # "Bob" # Or use map echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq 'map(.name)' # ["Alice", "Bob"] echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[].name' # "Alice" # "Bob" # Or use map echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq 'map(.name)' # ["Alice", "Bob"] -weight: 500;">docker images --format='{{json .}}' | jq -r '.Repository + ":" + .Tag + "\t" + .Size' -weight: 500;">docker images --format='{{json .}}' | jq -r '.Repository + ":" + .Tag + "\t" + .Size' -weight: 500;">docker images --format='{{json .}}' | jq -r '.Repository + ":" + .Tag + "\t" + .Size' -weight: 500;">kubectl get pods -o json | jq -r '.items[].metadata.name' -weight: 500;">kubectl get pods -o json | jq -r '.items[].metadata.name' -weight: 500;">kubectl get pods -o json | jq -r '.items[].metadata.name' aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | "\(.InstanceId)\t\(.State.Name)\t\(.PrivateIpAddress)"' aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | "\(.InstanceId)\t\(.State.Name)\t\(.PrivateIpAddress)"' aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | "\(.InstanceId)\t\(.State.Name)\t\(.PrivateIpAddress)"' cat package.json | jq -r '.dependencies | keys[]' cat package.json | jq -r '.dependencies | keys[]' cat package.json | jq -r '.dependencies | keys[]' -weight: 500;">curl -s https://api.github.com/users/torvalds | jq '{name, bio, public_repos, followers}' -weight: 500;">curl -s https://api.github.com/users/torvalds | jq '{name, bio, public_repos, followers}' -weight: 500;">curl -s https://api.github.com/users/torvalds | jq '{name, bio, public_repos, followers}' # Get users older than 25 echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[] | select(.age > 25)' # Get users older than 25 echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[] | select(.age > 25)' # Get users older than 25 echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq '.[] | select(.age > 25)' # AND condition jq '.[] | select(.age > 25 and .name == "Bob")' # OR condition jq '.[] | select(.age > 25 or .name == "Alice")' # AND condition jq '.[] | select(.age > 25 and .name == "Bob")' # OR condition jq '.[] | select(.age > 25 or .name == "Alice")' # AND condition jq '.[] | select(.age > 25 and .name == "Bob")' # OR condition jq '.[] | select(.age > 25 or .name == "Alice")' jq '.[] | select(.email != null)' jq '.[] | select(.email != null)' jq '.[] | select(.email != null)' echo '{"first": "John", "last": "Doe", "age": 30}' | jq '{fullname: (.first + " " + .last), age}' # { # "fullname": "John Doe", # "age": 30 # } echo '{"first": "John", "last": "Doe", "age": 30}' | jq '{fullname: (.first + " " + .last), age}' # { # "fullname": "John Doe", # "age": 30 # } echo '{"first": "John", "last": "Doe", "age": 30}' | jq '{fullname: (.first + " " + .last), age}' # { # "fullname": "John Doe", # "age": 30 # } echo '{"old_name": "value"}' | jq '{new_name: .old_name}' echo '{"old_name": "value"}' | jq '{new_name: .old_name}' echo '{"old_name": "value"}' | jq '{new_name: .old_name}' echo '{"name": "John"}' | jq '. + {age: 30}' # { # "name": "John", # "age": 30 # } echo '{"name": "John"}' | jq '. + {age: 30}' # { # "name": "John", # "age": 30 # } echo '{"name": "John"}' | jq '. + {age: 30}' # { # "name": "John", # "age": 30 # } # Sort array of objects by field echo '[{"name": "Bob", "age": 30}, {"name": "Alice", "age": 25}]' | jq 'sort_by(.age)' # Reverse sort jq 'sort_by(.age) | reverse' # Sort array of objects by field echo '[{"name": "Bob", "age": 30}, {"name": "Alice", "age": 25}]' | jq 'sort_by(.age)' # Reverse sort jq 'sort_by(.age) | reverse' # Sort array of objects by field echo '[{"name": "Bob", "age": 30}, {"name": "Alice", "age": 25}]' | jq 'sort_by(.age)' # Reverse sort jq 'sort_by(.age) | reverse' # Group by field echo '[{"type": "A", "value": 1}, {"type": "B", "value": 2}, {"type": "A", "value": 3}]' | jq 'group_by(.type)' # Group by field echo '[{"type": "A", "value": 1}, {"type": "B", "value": 2}, {"type": "A", "value": 3}]' | jq 'group_by(.type)' # Group by field echo '[{"type": "A", "value": 1}, {"type": "B", "value": 2}, {"type": "A", "value": 3}]' | jq 'group_by(.type)' jq 'group_by(.type) | map({type: .[0].type, count: length})' jq 'group_by(.type) | map({type: .[0].type, count: length})' jq 'group_by(.type) | map({type: .[0].type, count: length})' echo '[{"value": 10}, {"value": 20}, {"value": 30}]' | jq '[.[].value] | add' # 60 echo '[{"value": 10}, {"value": 20}, {"value": 30}]' | jq '[.[].value] | add' # 60 echo '[{"value": 10}, {"value": 20}, {"value": 30}]' | jq '[.[].value] | add' # 60 echo '[1, 2, 2, 3, 3, 3]' | jq 'unique' # [1, 2, 3] echo '[1, 2, 2, 3, 3, 3]' | jq 'unique' # [1, 2, 3] echo '[1, 2, 2, 3, 3, 3]' | jq 'unique' # [1, 2, 3] echo '[10, 5, 20, 15]' | jq 'min' # 5 echo '[10, 5, 20, 15]' | jq 'max' # 20 echo '[10, 5, 20, 15]' | jq 'min' # 5 echo '[10, 5, 20, 15]' | jq 'max' # 20 echo '[10, 5, 20, 15]' | jq 'min' # 5 echo '[10, 5, 20, 15]' | jq 'max' # 20 # Convert JSON to CSV echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq -r '.[] | [.name, .age] | @csv' # "Alice",25 # "Bob",30 # Convert JSON to CSV echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq -r '.[] | [.name, .age] | @csv' # "Alice",25 # "Bob",30 # Convert JSON to CSV echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | jq -r '.[] | [.name, .age] | @csv' # "Alice",25 # "Bob",30 # Deep extraction echo '{"user": {"profile": {"name": "John"}}}' | jq '.user.profile.name' # "John" # Safe navigation (don't error if missing) echo '{"user": {}}' | jq '.user.profile.name // "N/A"' # "N/A" # Deep extraction echo '{"user": {"profile": {"name": "John"}}}' | jq '.user.profile.name' # "John" # Safe navigation (don't error if missing) echo '{"user": {}}' | jq '.user.profile.name // "N/A"' # "N/A" # Deep extraction echo '{"user": {"profile": {"name": "John"}}}' | jq '.user.profile.name' # "John" # Safe navigation (don't error if missing) echo '{"user": {}}' | jq '.user.profile.name // "N/A"' # "N/A" #!/bin/bash -weight: 500;">curl -s http://api/services | jq -r '.[] | if .-weight: 500;">status == "up" then "\(.name): ✓" else "\(.name): ✗ (DOWN)" end' #!/bin/bash -weight: 500;">curl -s http://api/services | jq -r '.[] | if .-weight: 500;">status == "up" then "\(.name): ✓" else "\(.name): ✗ (DOWN)" end' #!/bin/bash -weight: 500;">curl -s http://api/services | jq -r '.[] | if .-weight: 500;">status == "up" then "\(.name): ✓" else "\(.name): ✗ (DOWN)" end' #!/bin/bash aws ce get-cost-and-usage \ --time-period Start=2024-01-01,End=2024-01-31 \ --granularity MONTHLY \ --metrics BlendedCost | \ jq -r '.ResultsByTime[] | .TimePeriod.Start + "\t$" + .Total.BlendedCost.Amount' #!/bin/bash aws ce get-cost-and-usage \ --time-period Start=2024-01-01,End=2024-01-31 \ --granularity MONTHLY \ --metrics BlendedCost | \ jq -r '.ResultsByTime[] | .TimePeriod.Start + "\t$" + .Total.BlendedCost.Amount' #!/bin/bash aws ce get-cost-and-usage \ --time-period Start=2024-01-01,End=2024-01-31 \ --granularity MONTHLY \ --metrics BlendedCost | \ jq -r '.ResultsByTime[] | .TimePeriod.Start + "\t$" + .Total.BlendedCost.Amount' #!/bin/bash -weight: 500;">kubectl logs -f pod-name | jq -r 'select(.level == "error") | "\(.timestamp): \(.message)"' #!/bin/bash -weight: 500;">kubectl logs -f pod-name | jq -r 'select(.level == "error") | "\(.timestamp): \(.message)"' #!/bin/bash -weight: 500;">kubectl logs -f pod-name | jq -r 'select(.level == "error") | "\(.timestamp): \(.message)"' # Start simple echo '{"a": {"b": {"c": 1}}}' | jq '.' # Add one level echo '{"a": {"b": {"c": 1}}}' | jq '.a' # Add another echo '{"a": {"b": {"c": 1}}}' | jq '.a.b' # Final echo '{"a": {"b": {"c": 1}}}' | jq '.a.b.c' # Start simple echo '{"a": {"b": {"c": 1}}}' | jq '.' # Add one level echo '{"a": {"b": {"c": 1}}}' | jq '.a' # Add another echo '{"a": {"b": {"c": 1}}}' | jq '.a.b' # Final echo '{"a": {"b": {"c": 1}}}' | jq '.a.b.c' # Start simple echo '{"a": {"b": {"c": 1}}}' | jq '.' # Add one level echo '{"a": {"b": {"c": 1}}}' | jq '.a' # Add another echo '{"a": {"b": {"c": 1}}}' | jq '.a.b' # Final echo '{"a": {"b": {"c": 1}}}' | jq '.a.b.c' -weight: 500;">curl -s api.example.com/data | jq '.' > formatted.json -weight: 500;">curl -s api.example.com/data | jq '.' > formatted.json -weight: 500;">curl -s api.example.com/data | jq '.' > formatted.json -weight: 500;">curl -s api | jq -r '.items[] | select(.active) | .id' | while read id; do echo "Processing $id" # do something with $id done -weight: 500;">curl -s api | jq -r '.items[] | select(.active) | .id' | while read id; do echo "Processing $id" # do something with $id done -weight: 500;">curl -s api | jq -r '.items[] | select(.active) | .id' | while read id; do echo "Processing $id" # do something with $id done jq -s '.' file1.json file2.json file3.json > combined.json jq -s '.' file1.json file2.json file3.json > combined.json jq -s '.' file1.json file2.json file3.json > combined.json # Add a field jq '.version = "2.0"' package.json > temp.json && mv temp.json package.json # Or use sponge (from moreutils) jq '.version = "2.0"' package.json | sponge package.json # Add a field jq '.version = "2.0"' package.json > temp.json && mv temp.json package.json # Or use sponge (from moreutils) jq '.version = "2.0"' package.json | sponge package.json # Add a field jq '.version = "2.0"' package.json > temp.json && mv temp.json package.json # Or use sponge (from moreutils) jq '.version = "2.0"' package.json | sponge package.json # Wrong (includes quotes) NAME=$(echo '{"name": "John"}' | jq '.name') echo $NAME # "John" # Right (no quotes) NAME=$(echo '{"name": "John"}' | jq -r '.name') echo $NAME # John # Wrong (includes quotes) NAME=$(echo '{"name": "John"}' | jq '.name') echo $NAME # "John" # Right (no quotes) NAME=$(echo '{"name": "John"}' | jq -r '.name') echo $NAME # John # Wrong (includes quotes) NAME=$(echo '{"name": "John"}' | jq '.name') echo $NAME # "John" # Right (no quotes) NAME=$(echo '{"name": "John"}' | jq -r '.name') echo $NAME # John jq '.' # Pretty print jq -r '.field' # Raw output (no quotes) jq '.field' # Get field jq '.[0]' # First array element jq '.[]' # All array elements jq 'length' # Length jq 'keys' # Object keys jq '.[] | select(.x > 5)' # Filter jq 'map(.field)' # Map jq 'sort_by(.field)' # Sort jq 'group_by(.field)' # Group jq 'add' # Sum array jq 'unique' # Unique values jq -s '.' # Slurp (combine files) jq '.' # Pretty print jq -r '.field' # Raw output (no quotes) jq '.field' # Get field jq '.[0]' # First array element jq '.[]' # All array elements jq 'length' # Length jq 'keys' # Object keys jq '.[] | select(.x > 5)' # Filter jq 'map(.field)' # Map jq 'sort_by(.field)' # Sort jq 'group_by(.field)' # Group jq 'add' # Sum array jq 'unique' # Unique values jq -s '.' # Slurp (combine files) jq '.' # Pretty print jq -r '.field' # Raw output (no quotes) jq '.field' # Get field jq '.[0]' # First array element jq '.[]' # All array elements jq 'length' # Length jq 'keys' # Object keys jq '.[] | select(.x > 5)' # Filter jq 'map(.field)' # Map jq 'sort_by(.field)' # Sort jq 'group_by(.field)' # Group jq 'add' # Sum array jq 'unique' # Unique values jq -s '.' # Slurp (combine files)