Explore a multi-gigabyte log file

This guide takes you from a large log file to the records that explain a problem: you open the log, find the error records, count them by level and by service, and read one of them in full. It works the same on a log of a few kilobytes or of several gigabytes.

Before you start

You need a log with one JSON object per line, a format called NDJSON or JSON Lines. Its name must end in .ndjson, .jsonl, or .jsonlines; rename a .log or .txt log, which would open as plain text. A compressed log such as app.ndjson.gz opens as it is. See Supported formats.

The steps use this excerpt from a file named app.ndjson:

app.ndjson (excerpt)

{"ts":"2026-09-30T08:00:01Z","level":"info","service":"api","msg":"GET /orders 200","request_id":"r-101"}
{"ts":"2026-09-30T08:00:02Z","level":"error","service":"api","msg":"POST /orders 500: database timeout","request_id":"r-102"}
{"ts":"2026-09-30T08:00:02Z","level":"info","service":"worker","msg":"job 77 started","request_id":"r-103"}
{"ts":"2026-09-30T08:00:03Z","level":"warn","service":"worker","msg":"job 77 retry 1","request_id":"r-103"}
{"ts":"2026-09-30T08:00:05Z","level":"error","service":"worker","msg":"job 77 failed: database timeout","request_id":"r-103"}
{"ts":"2026-09-30T08:00:06Z","level":"error","service":"api","msg":"POST /orders 500: database timeout","request_id":"r-104"}
{"ts":"2026-09-30T08:00:07Z","level":"info","service":"api","msg":"GET /health 200","request_id":"r-105"}

The counts in the steps are for this excerpt.

Open the log

Press Ctrl+O and choose app.ndjson. The file opens in the Tree view, with the list of records at the top, shown as [7], and one row per record below it.

On a large file, the first records appear at once, and the status bar shows indexing… N% until the rest is indexed; see How large files open.

The tree lists the first 4,096 records. Search, the Table view, and SQL cover every record; a result past the first 4,096 opens in the NDJSON view at its line.

Find the error records

1. Search by field

   Press Ctrl+F and select Object at the left of the search bar. Type level in Key path, choose equals, type error in Value, and press Enter. The search bar shows 1 of 3, and the tree selects the first error record. Press F3 to go to the next one. See Search for objects by key.

2. Hide the other records

   Select Filter in the search bar. The tree keeps only the three error records. Select Filter again to show every record.

3. List the matches

   Select List in the search bar. The Search results panel lists one row per match; click a row to go to the record.

To follow one request through the log, search for its ID in text mode instead, for example r-103: it finds the three records of job 77, from info to error.

Count records by level

1. Select Table in the view switcher. Each record is a row, with the columns ts, level, service, msg, and request_id.

2. Right-click a cell in the level column and choose Value frequency of level. The panel lists info 3, error 3, and warn 1.

3. Click error in the panel. The table shows only the three error records.

On a large log, value frequency reads the first 100,000 records; select Scan more to read up to 1,000,000. To count every record, use SQL. See Value frequency of a column.

Count errors with SQL

SQL reads every record of the file, whatever its size.

1. Press Ctrl+E. The query bar opens on the SQL tab, and the chip reads doc = records: doc has one row per record.

2. Type this query and press Ctrl+Enter:

   SELECT service, count(*) AS errors
   FROM doc
   WHERE level = 'error'
   GROUP BY service
   ORDER BY errors DESC

   The grid shows api 2 and worker 1.

3. To list the error records with their place in the file, query doc_rows and keep its _row column:

   SELECT _row, ts, service, msg FROM doc_rows WHERE level = 'error'

4. Alt+click a row of the result. The tree selects that record.

To save the result, select Export every row and choose a .csv, .ndjson, or .parquet file name. See SQL queries.

Read a record in full

In the Tree view, select a record and expand it to see its fields. To paste the record into a ticket, right-click it and choose Copy JSON (formatted).

To see the record as it is written in the file, select NDJSON in the view switcher. Click a line, or press Enter on it, to show its record in the tree again. See NDJSON and Text views.

Next steps

• Search a document: text, regex, and exact search, and searching every open tab
• jq playground: reshape the records, for example to list only their messages
• Merge files: read the logs of several days as one document