jq: JSONL ↔︎ JSON conversion

jq_jsonl_conversion.md

jq: JSONL ↔︎ JSON conversion

Prerequisites

• jq — https://jqlang.org/ — "like sed for JSON data"

  There are several options available for installing jq. I prefer to use Homebrew: brew install jq

• JSONL → JSON

  jq -s '.' input.jsonl > output.json

• JSON → JSONL

  jq -c '.[]' input.json > output.jsonl

Note: This document is now included in Cookbook · jqlang/jq Wiki.

@EdGaere, thanks for the example! I thought I could improve (i.e., shorten) the command you wrote. In your example, you called jq twice in the pipeline, but it can be done with one call instead…

gzcat somefile.json.gz | jq -c '.data[] | {input, target}'

I.e., combining the filters: .data + .[] → .data[].

Cleaning the test data

I wanted to test this with your data. `jq` didn't like your hand-edited data's unquoted keys, like `some_other_field`, so I cleaned up the data first…

{
  "meta": {},
  "data": [
    {
      "idx": 1,
      "input": "ABC",
      "target": "123",
      "some_other_field": "zzz"
    },
    {
      "idx": 2,
      "input": "DEF",
      "target": "456",
      "some_other_field": "zzz"
    }
  ]
}

(Maybe jq has some option to ignore errors like unquoted keys.)

Using the test data

Running the shortened command I gave above gives the output…

{"input":"ABC","target":"123"}
{"input":"DEF","target":"456"}

Notice that the -c option for jq compacts the output without whitespace in each record. It's more compact than the hand-edited output of your example.