If you are a software engineer or a programming enthusiast, you might have heard about cURL, a command line utility to transfer data to or from a server. It can simply be thought of as making API calls as well.

We will reserve the basics of cURL for some other day but a few weeks back I came across an amazing command line tool called jq to parse json responses from curl and it felt so neat and beautiful that I decided to share it with you all.

It could be that many of you might know about it but I was working with cURL for quite some time but got to know about this JSON utility just now.

Installation

Here is the link to download the utility on your system https://github.com/stedolan/jq/wiki/Installation

General Syntax

On curl command

curl <url> | jq <jq_string>

On input file

jq <jq_string> filename

Here <jq_string> is the query string/ args for jq command

1. Prettify Response

One of the simplest but pretty features of jq is to beautify the JSON response.

In case we want to skip any filter with jq, we pass the identity filter as jq_string and jq will return unfiltered output, but by default, will prettify the JSON.

Identity Filter = . (dot character)

Normal Response

curl -s http://api.open-notify.org/iss-now.json

Response with jq

curl -s http://api.open-notify.org/iss-now.json | jq '.'

2. Extracting Attributes from JSON Response

Here is the sample API response which we will be using for filtering

$ curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.'

# Output 
{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
  }
}

Accessing single attribute

It is as simple as specifying the attribute in jq_string

Example to get phone, the jq_string will be '.phone'

$ curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.phone'

# Output
"1-770-736-8031 x56442"

Accessing nested attribute

Referencing nested attributes is similar to how ones do it in any programming language i.e by chaining attributes using the dot operator.

jq_string will be '.outerAttribute.innerAttribute'

$ curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.company'

# Output
{
  "name": "Romaguera-Crona",
  "catchPhrase": "Multi-layered client-server neural-net",
  "bs": "harness real-time e-markets"
}

$ curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.company.name'

# Output
"Romaguera-Crona"

Accessing multiple attributes

For accessing multiple attributes, we just need to specify them in comma separate fashion.

$ curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.company.name, .company.catchPhrase'

# Output 
"Romaguera-Crona"
"Multi-layered client-server neural-net"

NOTE :

In case of an attribute name containing spaces or any other special characters, we need to enclose it within quotation marks

Example

$ echo '{ "Full Name" : "Umang Ahuja", "phone-number" : "1234567890" }' | jq '.'

# Output
{
  "Full Name": "Umang Ahuja",
  "phone-number": "1234567890"
}

Here to access the Full Name and phone-number, both need to be enclosed within individual quotes otherwise we will get an error.

$ echo '{ "Full Name" : "Umang Ahuja", "phone-number" : "1234567890" }' | jq '."Full Name", ."phone-number"'

# Output
"Umang Ahuja"
"1234567890"

3. Extracting data from JSON Array

Iterating over array

For iteration, the jq_string used is '.[]' and this will output individual objects in a new line.

Input JSON file input.json

cat input.json

# Output
[
  {
    "id": 1,
    "name": "Leanne Graham"
  },
  {
    "id": 2,
    "name": "Ervin Howell"
  },
  {
    "id": 3,
    "name": "Clementine Bauch"
  },
  {
    "id": 4,
    "name": "Patricia Lebsack"
  },
  {
    "id": 5,
    "name": "Chelsey Dietrich"
  }
]

Output

$ jq '.[]' input.json

# Output
{
  "id": 1,
  "name": "Leanne Graham"
}
{
  "id": 2,
  "name": "Ervin Howell"
}
{
  "id": 3,
  "name": "Clementine Bauch"
}
{
  "id": 4,
  "name": "Patricia Lebsack"
}
{
  "id": 5,
  "name": "Chelsey Dietrich"
}

See how each object is extracted from the list and printed.

Note that no square brackets [ ] are printed since individual objects are printed and not the whole list.

Extracting attributes from objects of JSON Array

Just like we extracted attributes from a single object, we can perform a similar operation on an array of objects too.

First, we need to iterate over objects and then parse individual objects for desired attributes using the pipe character

$ jq ' .[]  | .id, .name ' input.json

# Output
1
"Leanne Graham"
2
"Ervin Howell"
3
"Clementine Bauch"
4
"Patricia Lebsack"
5
"Chelsey Dietrich"

Accessing object by index

It is identical to accessing a normal object in any programming language i.e using square brackets.

Note that indices start from 0.

Also, negative indices are possible with -1 denoting the last object, -2 denoting the second last object and so on.

$ jq ' .[1]' input.json

# Output
{
  "id": 2,
  "name": "Ervin Howell"
}


$ jq ' .[-1]' input.json

# Output
{
  "id": 5,
  "name": "Chelsey Dietrich"
}

Slicing a JSON array

jq_string for slicing '.[startIndex: endIndex]'

where startIndex is inclusive and endIndex is exclusive

If we ignore startIndex, we will get data from 0th index.

If we ignore endIndex, we will get data till the end of the array.

$ jq ' .[1:4]' input.json

# Output
[
  {
    "id": 2,
    "name": "Ervin Howell"
  },
  {
    "id": 3,
    "name": "Clementine Bauch"
  },
  {
    "id": 4,
    "name": "Patricia Lebsack"
  }
]

There are many more things which make jq stand out like functions which can perform almost anything we can think of on the data from transformation to selection and much more. But will leave it for some other day.

In case this article presented something new to you, let me know in the comments below.

If you want to connect with me on LinkedIn, drop a connect at https://www.linkedin.com/in/umangahuja1/