Linux jq For Work With JSON Data Examples: Select, Search, Sort

I use Protovpn and need to choose the fastest free server. All the servers are listed in json file: https://api.protonmail.ch/vpn/logicals Let's try linux jq. First install curl if you have no (apt-get install curl) and download json file: curl https://api.protonmail.ch/vpn/logicals > __tmp_protonvpn

Install jq on Linux

apt-cache show jq
...
Description-en: lightweight and flexible command-line JSON processor
 jq is like sed for JSON data – you can use it to slice
 and filter and map and transform structured data with
 the same ease that sed, awk, grep and friends let you
 play with text.
...

apt-get install jq

 

View the elements of JSON array

First check if you have an array .[] or just a list:

head -c150 __tmp_protonvpn 
{"Code":1000,"LogicalServers":[{"Name":"CA#1","EntryCountry":"CA","ExitCountry":"CA","Domain":"ca-01.protonvpn.net","Tier":1,"Features":0,"Region":nul

This means it is not an array in squared brackets. It is a list from two elements: 'Code' and 'LogicalServers'. We need to use element 'LogicalServers' that is an array itself (in squared brackets).

View the content of array LogicalServers:

jq '.LogicalServers[]' __tmp_protonvpn | head -n 10
{
  "Name": "CA#1",
  "EntryCountry": "CA",
  "ExitCountry": "CA",
  "Domain": "ca-01.protonvpn.net",
  "Tier": 1,
  "Features": 0,
  "Region": null,
  "City": "Toronto",
  "Score": 2.51417729,

File is huge, so use head every time.

Linux jq get only one key

Select only Name

jq '.LogicalServers[].Name' __tmp_protonvpn | head -n 5
"CA#1"
"CA#2"
"CA#3"
"CA#4"
"US-VA#1"

As you can see, dot '.' is using for choose inner elements. Element 'Name' is a string, there why we do not use .Name[], but use only .Name. Brackets are using for arrays.

 

jq get multiple keys from json

After some search... this helped https://stackoverflow.com/questions/34834519/how-do-i-select-multiple-f…

jq '.LogicalServers[] | {Name, Score}' __tmp_protonvpn | head -n 10
{
  "Name": "CA#1",
  "Score": 2.51417729
}
{
  "Name": "CA#2",
  "Score": 2.51417729
}
{
  "Name": "CA#3",

As you can see, vertical bar '|' is using for select elements from a range.

 

jq select values started with

Select only Name and search names started with NL-FREE:

jq '.LogicalServers[]|select(.Name | startswith("NL-FREE")).Name' __tmp_protonvpn
"NL-FREE#2"
"NL-FREE#1"
"NL-FREE#3"
"NL-FREE#4"
"NL-FREE#5"
"NL-FREE#6"
"NL-FREE#7"
"NL-FREE#8"
"NL-FREE#9"
"NL-FREE#10"
"NL-FREE#11"

Special function startswith() search in each Name value string.

Now get scores of NL-FREE servers, this is a speed:

jq '.LogicalServers[]|select(.Name | startswith("NL-FREE")).Score' __tmp_protonvpn
1001.30219968
2.24350678
2.26762048
2.24350678
2.22827098
2.24350678
11.29328938
1001.30219968
11.29328938
2.27979356
2.18001088

But we also want to see server name with score.

 

jq '.LogicalServers[]|select(.Name | startswith("NL-FREE")) | {Score,Name}' __tmp_protonvpn | head -n 8
{
  "Score": 1001.30219968,
  "Name": "NL-FREE#2"
}
{
  "Score": 2.24350678,
  "Name": "NL-FREE#1"
}

Finally we can see the fastest server from the list, but need to choose it automatically. How to compare Scores and print sorted array? Search gave this https://stackoverflow.com/questions/63739892/jq-sort-by-value-of-key

 

jq sort by numeric values

In our JSON field Score has numeric value without quotes. We want to process values of key 'Score' as numbers to sort an array.

jq '.LogicalServers|=sort_by(.Score)|.LogicalServers[]|select(.Name|startswith("NL-FREE"))|{Score,Name}' __tmp_protonvpn  
{
  "Score": 2.18001088,
  "Name": "NL-FREE#11"
}
{
  "Score": 2.22827098,
  "Name": "NL-FREE#5"
}
{
  "Score": 2.24350678,
  "Name": "NL-FREE#1"
}
{
  "Score": 2.24350678,
  "Name": "NL-FREE#4"
}
{
  "Score": 2.24350678,
  "Name": "NL-FREE#6"
}
{
  "Score": 2.26762048,
  "Name": "NL-FREE#3"
}
{
  "Score": 2.27979356,
  "Name": "NL-FREE#10"
}
{
  "Score": 11.29328938,
  "Name": "NL-FREE#7"
}
{
  "Score": 11.29328938,
  "Name": "NL-FREE#9"
}
{
  "Score": 1001.30219968,
  "Name": "NL-FREE#2"
}
{
  "Score": 1001.30219968,
  "Name": "NL-FREE#8"
}
 

Here we see the best server is NL-FREE#8 shown last. We can 'head -n 4' for get the result

jq '.LogicalServers|=sort_by(.Score)|.LogicalServers[]|select(.Name|startswith("NL-FREE"))|{Score,Name}' __tmp_protonvpn  | head -n 4
{
  "Score": 1001.30219968,
  "Name": "NL-FREE#8"
}

The lowest Score is the less loaded server (the best choose).

 

This jq example is using in my bash script for choosing the best server: https://gitgud.io/youni/devuan-config-files/-/blob/master/op

 

Sources

jq manual https://stedolan.github.io/jq/manual/

Section
Category