# csv-to-json throughput benchmark

Reproducible CSV-to-JSON throughput numbers used in
`content/blog/{ja,en,es,pt}/csv-to-json-guide.md`.

## What it measures

A 5-column, 1,000,000-row CSV (≈ 52 MB) is converted to JSON / NDJSON
through six approaches and the wall-clock time is recorded for each.

The six approaches:

1. Python stdlib — `csv.DictReader` + `json.dump`
2. pandas — `read_csv` + `to_json(orient="records", lines=True)`
3. Miller — `mlr --c2j cat`
4. Go — `encoding/csv` + `encoding/json` (streaming, `ReuseRecord`)
5. PapaParse streaming — Node, `papaparse` 5.5.2, header mode
6. FormatArc in the browser (not run here; see notes below)

## How to reproduce

```bash
cd scripts/benchmarks/csv-to-json-throughput

# Generate the sample (1M rows × 5 cols)
python3 gen_sample.py 1000000

# Python stdlib + pandas
python3 -m venv venv
./venv/bin/pip install pandas pyarrow
./venv/bin/python bench_python.py
./venv/bin/python bench_pandas.py

# Miller (requires `brew install miller` or equivalent)
mlr --c2j cat sample.csv > out_miller.json

# Go
go run bench_go.go

# PapaParse (Node)
npm install
node bench_papaparse.mjs
```

The FormatArc-in-browser approach is best measured manually: open
`https://formatarc.com/<locale>/csv-to-json/`, paste in `sample.csv`,
and time it. PapaParse is the underlying engine in both Node and
browser builds, but the browser version is bounded by main-thread budget
and the page's responsiveness rather than raw CPU.

## Measured results (2026-06-23, Apple M5 Pro / 64 GB / macOS 26.5)

| Approach | Version | Rows/sec | Wall (s) |
| --- | --- | ---: | ---: |
| Python `csv.DictReader` + `json.dump` | Python 3.14.6 | 232,137 | 4.31 |
| pandas `read_csv` + `to_json` | pandas 3.0.3 | 1,112,836 | 0.90 |
| Miller `mlr --c2j` (warm median of 5) | Miller 6.19.0 | 1,960,784 | 0.51 |
| Go `encoding/csv` + `encoding/json` | Go 1.26.4 | 2,058,201 | 0.49 |
| PapaParse streaming (Node) | papaparse 5.5.2 / Node 26.3.1 | 1,625,566 | 0.62 |

## Caveats

- These numbers are for **one specific machine, one specific row shape,
  one measurement set** (single warm run for Python / pandas / Go /
  PapaParse; warm median of 5 for Miller, where cold-vs-warm difference
  was material). Re-running on a 2024 Intel laptop or a 2021 M1 Air will
  give different (usually lower) numbers.
- pandas 3.x is materially faster than pandas 2.x; Miller 6.x is faster
  than Miller 5.x. Older versions of the same tool can be 2-5× slower.
- CSVs with wider rows, more quoted fields, or non-ASCII text typically
  produce lower throughput than this synthetic ASCII sample.
- Browser performance is hard to compare apples-to-apples with native
  CLIs: the browser shares CPU with the page render thread, V8 startup
  cost dominates small inputs, and large inputs hit memory pressure on
  the page rather than disk swap.

## History

- 2026-06-23 — Created. Replaces a prior "approximate throughput on a
  2024 M3 MacBook" table in the article that had no reproduction script
  and reported numbers 1.5×–6× lower than what the actual tools achieve
  today.