Easily Convert JSON to JSON lines in Linux jq

John D K -

In this article, we will show how to convert JSON input to JSON lines output. We will also cover multi and single line JSON lines output and saving new JSON file.

To convert JSON to JSON lines in Linux we can use a tool called jq. It's very fast and reliable.

TLDR

  • jq --compact-output . - compact form (single line)
  • jq -c . - compact form (single line)
  • jq -c '.[]' - multi-line JSON lines
  • jq -cs '.' in.json > out.json - read and write file

If you want to convert json to jsonl in python: Python convert normal JSON to JSON separated lines 3 examples

Convert JSON to JSON lines --compact-output

We can use the compact format of jq to convert a multi-line JSON input to a JSON lines output

$ echo '{
  "name": "Tim",
  "age": 20,
  "city": "London"
}' | jq --compact-output .

result:

{"name":"Tim","age":20,"city":"London"}

Convert JSON to JSON lines -c

We can use -c flag to achieve the same result. In this example we work with list of JSON records which will be converted to JSON lines.

echo '[{
  "name": "Tim",
  "age": 20,
  "city": "London"
},
{
  "name": "Jake",
  "age": 15,
  "city": "Paris"
}]' | jq -c .

Result:

[{"name":"Tim","age":20,"city":"London"},{"name":"Jake","age":15,"city":"Paris"}]

JSON file to new JSON lines file

What if we like to save the result of the conversion to a new JSON line file. In this case we can redirect the command output to new file by > out.json:

jq -cs '.' in.json > out.json

The input file in.json:

[{
"name": "Tim",
"age": 20,
"city": "London"
},
{
"name": "Jake",
"age": 15,
"city": "Paris"
}]

will be converted and saved as - out.json:

[[{"name":"Tim","age":20,"city":"London"},{"name":"Jake","age":15,"city":"Paris"}]]

By default the result is single lined. Checked next section to find out how to get multi-lined JSON line file.

Split JSON to multiple JSON lines

In order to split multi-line JSON file to single lined JSON lines file we can use the following syntax:

jq -c '.[]'

Example:

echo '[{
  "name": "Tim",
  "age": 20,
  "city": "London"
},
{
  "name": "Jake",
  "age": 15,
  "city": "Paris"
}]' | jq -c '.[]'

will result into:

{"name":"Tim","age":20,"city":"London"}
{"name":"Jake","age":15,"city":"Paris"}

Convert and chain commands

Finally if we like to convert and replace character/string from the output we can chain commands like: tr, sed etc:

echo '[{
  "name": "Tim",
  "age": 20,
  "city": "London"
},
{
  "name": "Jake",
  "age": 15,
  "city": "Paris"
}]' | jq -c . | sed 's/},/}\n/g'

So instead of getting all the results on a single line:

[{"name":"Tim","age":20,"city":"London"},{"name":"Jake","age":15,"city":"Paris"}]

the JSON lines are split on a separated line each:

[{"name":"Tim","age":20,"city":"London"}
{"name":"Jake","age":15,"city":"Paris"}]

More jq examples: Working with JSON files and JQ

Conclusion

We covered how to convert JSON to JSON lines. We discussed json to jsonl single and multi-line output. Few useful examples will help you out to convert any JSON to JSONL format.