How to open a large JSON file

Your editor is spinning, the online formatter you found gave up, and you just want to see what is in the file. Here is what is actually going wrong and what to do about it.

Why the usual tools stall

Most of them are doing one of three things wrong, and it helps to know which one you have hit, because the fix is different each time.

They render every line

A 200 MB file is a few million lines. Put a DOM node on screen for each one and the browser is finished before you have scrolled anywhere. The fix is a virtual list: only build the forty or so rows that are actually visible, and swap them as you scroll. Any viewer that handles big files does this. Any viewer that does not will lock up somewhere around ten thousand rows.

They parse on the UI thread

JSON.parse is one call that runs to completion. There is no yielding in the middle of it. If it runs on the main thread, the tab is frozen for as long as it takes, and the browser eventually offers to kill the page. Moving the parse into a Web Worker does not make it faster, but it does keep the interface alive, and it means you can show progress and offer a cancel button.

They upload the file

Plenty of "online JSON viewers" post your file to a server. For a 200 MB file that is slow. For a 200 MB file with customer data in it, that is a problem you now have to explain to somebody. Check before you paste. A tool that works locally will keep working with your network unplugged, which is an easy thing to test.

What size are we talking about

There is a hard ceiling in the browser and it is lower than most people expect. Each JavaScript engine caps the length of a single string, and reading a file into a tab means putting it in one:

The exact numbers, and how to check them on your own machine, are on the page about the browser size limit. The short version: under half a gigabyte, a browser is fine. Over it, Chrome will not do it at all and you need a different plan.

The options

Open it in a browser tool

Good for reading and for finding one value in a structure you do not know. You get search, collapsing, and a path you can copy. No install, and if the tool is honest about running locally, no upload.

This is what jsontree.app does. It parses in a worker, shows real byte progress while it reads, and only ever renders the rows on screen. A 450 MB file with 26 million nodes opens in about five seconds.

Use jq

If you know what you are looking for, jq is better than any viewer. It streams, it has no size limit worth worrying about, and it composes with everything else in a shell.

# what keys does the first record have?
jq '.records[0] | keys' big.json

# pull one field out of a big array
jq -r '.records[].sampleId' big.json | head

# turn one huge array into newline delimited JSON
jq -c '.[]' big.json > big.ndjson

The catch is that jq answers questions. It is not much help when you do not yet know the question, which is most of the time when someone hands you an unfamiliar file.

Use jless

jless browses JSON in the terminal the way less browses text. It indexes the bytes instead of building an object graph, so it handles files far past anything a browser will take. If you live in a terminal, install it.

Split it

If the file is one enormous array, it is usually easier to cut it up than to open it whole. Convert to NDJSON and split by lines:

jq -c '.[]' big.json > big.ndjson
split -l 100000 big.ndjson part-

Each piece opens instantly anywhere. NDJSON is also nicer to work with in general, since you can read it line by line without holding the whole thing. More on NDJSON here.

A shortcut worth knowing

If you are hitting the ceiling in Chrome, try Firefox or Safari before you do anything else. Same file, same page, twice the room. It takes ten seconds to find out and it beats every other option on this page when it works.

Try it

Drop a file on jsontree.app and it opens as a tree you can collapse and search. Nothing is uploaded, so it is fine for production data.