How to Install jq on Ubuntu – A Comprehensive Guide for Developers and Sysadmins

jq is an indispensable tool for any developer or sysadmin working with JSON data on Linux. This comprehensive, step-by-step guide will teach you how to install, configure, and use jq on Ubuntu to slice, filter, map, and transform structured JSON data with ease.

What is jq?

jq is a lightweight and flexible command-line JSON processor for Linux and Unix-like systems.

Developed by Stephen Dolan, jq allows you to parse, filter, map, and reformat JSON-structured data with just a few keystrokes. It features a simple and intuitive syntax while still supporting many powerful features.

Some key highlights of jq:

• Open source software written in C
• Available for Linux, macOS, Windows, and Unix-like systems
• Actively maintained and updated
• MIT licensed
• Built specifically for JSON processing
• Supports filtering, slicing, mapping, transformations
• Fast and lightweight – quick even for large JSON docs
• Simple but powerful query syntax
• Great for scripting and pipelines
• Actively maintained and updated

jq has quickly become an essential tool for developers and sysadmins working with JSON APIs, data streams, log files, and more. Its flexibility makes it useful in countless situations.

According to the 2022 StackOverflow Developer Survey, jq is the 7th most loved technology by developers. This speaks to its immense utility.

Some common uses for jq:

• Parsing API responses in JSON format
• Transforming JSON data
• Creating reports from JSON log files
• Filtering JSON documents or streams
• Handling JSON output from programs
• Processing streaming JSON data

In the rest of this guide, I‘ll show you how to install jq on Ubuntu Linux and demonstrate some examples of how to use it. Let‘s get started!

Step 1 – Update Package Index

The first step is to update Ubuntu‘s package index. This makes sure you‘ll install the latest available version of jq:

sudo apt update

This fetches package metadata from the configured apt repositories and updates the local package database.

It‘s a good idea to run this before installing any new package on Ubuntu.

Step 2 – Install jq Package

Next, use apt to install the jq package:

sudo apt install jq

This will install jq and any required dependencies.

apt is Ubuntu‘s default package manager. It handles retrieving packages from repositories, managing dependencies, and installing/removing software packages.

Step 3 – Verify the Installation

To confirm that jq installed properly, you can check the version number:

jq --version

On Ubuntu 20.04, this should print out:

jq-1.6

This verifies that jq was installed correctly and is available in your PATH.

Step 4 – Test jq with Sample JSON

Now let‘s create a sample JSON file to test out jq‘s capabilities.

First, make a test JSON file called test.json:

echo ‘{"name": "John", "age": 30, "jobs": [{"job": "Programmer", "years": 5}]}‘ > test.json

This creates a simple JSON document with some nested data structures.

Let‘s start by printing the formatted JSON:

jq ‘.‘ test.json

Output:

{
  "name": "John",
  "age": 30,
  "jobs": [
    {
      "job": "Programmer",
      "years": 5
    }
  ]
}

The . filter pretty-prints JSON output.

Next, let‘s filter specific values from the JSON:

jq ‘.name‘ test.json

Output:

"John"

We can also query nested fields:

jq ‘.jobs[0].job‘ test.json

Output:

"Programmer"

In addition to filtering, we can transform the JSON data. For example, we can rename the "name" field:

jq ‘{firstName: .name}‘ test.json

Output:

{
  "firstName": "John"
}

We can even combine filters to reformat the JSON entirely:

jq ‘{person: {firstName: .name, age: .age}, jobTitle: .jobs[0].job}‘ test.json

Output:

{
  "person": {
    "firstName": "John",
    "age": 30
  },
  "jobTitle": "Programmer"
}

This demonstrates jq‘s flexible query syntax and JSON transformation capabilities.

Step 5 – Use jq in a Script

To show how jq can be used in scripts, let‘s write a simple Bash script that processes a JSON file.

Save this script as process_json.sh:

#!/bin/bash

# Input filename 
INPUT=$1

# Pretty print JSON 
jq ‘.‘ $INPUT

# Filter fields
echo "Name: $(jq ‘.name‘ $INPUT)"
echo "Age: $(jq ‘.age‘ $INPUT)" 

# Transform JSON
jq ‘{firstName: .name, age: .age}‘ $INPUT

To run it:

./process_json.sh test.json

This demonstrates how to use jq‘s output in a shell script. The same approach works with any scripting language like Python, Ruby, Perl, etc.

Step 6 – Process Streaming JSON Data

jq can also process JSON data streams rather than just static files.

For example, we can stream in JSON from a command and filter it with jq:

curl -s https://api.example.com/data.json | jq ‘.results[0]‘

This streams down a JSON response and parses just the first result with jq.

For processing log files in JSON format, you can use the -c argument to output each JSON object one at a time:

jq -c ‘.timestamp‘ log.json

This prints just the timestamp from each JSON log entry.

So jq is handy for handling streaming JSON data.

Step 7 – jq Options and Reference

Here is a quick reference for some useful jq options:

• .: This filter pretty-prints formatted JSON output.

• -c: Compact instead of pretty-printed output, one JSON object per line. Useful for handling streams.

• -n: Use null instead of nothing for missing values. Makes output valid JSON.

• -r: Output raw strings rather than JSON texts.

• -s: Sort JSON arrays before outputting.

• -C: Colorize JSON output for easier reading.

• --version: Print jq version.

• --help: Print help text and usage info.

See the jq manual for the complete reference.

Step 8 – Uninstall jq (Optional)

If you ever need to uninstall jq, just run:

sudo apt purge jq

This will fully remove the jq package and all its dependencies.

Conclusion

jq is an indispensable Swiss army knife for processing JSON data on Linux, especially in scripting applications. Its simple but flexible syntax allows you to easily slice, filter, map, and transform structured JSON documents.

In this guide, you learned how to:

• Install jq on Ubuntu
• Query and filter JSON documents
• Transform and reformat JSON data
• Use jq in shell scripts
• Process streaming JSON with jq
• Access jq help and docs
• Uninstall jq

With this knowledge, you can leverage the power of jq to wrangle JSON data like a pro!

Let me know in the comments if you have any other questions about using this excellent tool. Happy JSON processing!