Skip to content

Performance, honestly

The production shape is a sampled decode: a handful of site gridpoints out of a multi-megapoint field. For that shape this decoder does something no whole-image codec can — decodeJ2kRegion entropy-decodes only the codeblocks the requested points touch and runs window-bounded inverse lifts, returning values bit-identical to the full decode (the contract is in Two-ring correctness).

Measured single-threaded (minimum of 5, Node 24, Apple M5 Max; 4 uniformly scattered points per field):

fixturesamplesbitsfull decoderegion, 4 pointsratiocodeblocks touched
gdps-tmp-2m2,882,40012266 ms22 ms11.9×49/791
hrdps-continental-tmp-2m3,276,60016692 ms44 ms15.6×49/911
raqdps-pm25-sfc436,67120101 ms20 ms5.0×28/136
rdps-cape-sfc-jasper813,27524295 ms4.4 ms67×25/12,711

The cost scales sublinearly in points because nearby points share windows and every point’s coarse-level ancestry converges: on HRDPS-continental (same machine, single thread) 1 point touches 16 codeblocks (14 ms), 4 points 49 (44 ms), 16 points 145 (121 ms), 64 points 366 (297 ms), 256 points 674 (538 ms). The full packet-structure parse is unavoidable — packet lengths are only discoverable sequentially — but it is under half a millisecond on every regular fixture.

Through @azohra/meteo.grib’s worker pool this is the sampled path end-to-end: on the same machine, a 4-point sampled decode of HRDPS-continental through sampleFieldValuesAsync and a 2-worker pool sustains 25.5 ms per field against 394 ms per field for full decodes — 15.5× per core, so a 3,500-field HRDPS lane projects to ~90 s of sampled decode at pool 2 where full decodes would need ~23 minutes. The gate that holds the mechanism (≥6× per core, and bit-exactness against the full decode) is grib/test/production-codec-throughput.test.ts.

Decoding whole images single-threaded, this decoder is slower than the WASM OpenJPEG build and faster than the asm.js one. Measured by the cross-codec bench (grib/tools/bench-j2k-single.ts — it needs both codecs, and only that package depends on both; minimum of 5, Node 24, Apple Silicon):

fixturesamplesbits@azohra/meteo.j2koracleratio
gdps-tmp-2m2,882,40012278 ms98 ms2.84×
hrdps-continental-tmp-2m3,276,60016723 ms282 ms2.57×
raqdps-pm25-sfc436,67120105 ms140 ms0.75×
twelve-fixture total2110 ms883 ms2.39×

On the 20-bit field this decoder already wins outright — the WASM build clamps samples wider than 16 bits, so its former stand-in there was the far slower asm.js artifact. And none of the WASM numbers apply to the production shape at all: the WASM codec decodes whole images only, so a sampled decode under it pays the full-decode column every time.

Profiling puts ~95% of a full decode in Tier-1 (the MQ/EBCOT bit loops; the DWT is ~4%). That is exactly why region decode wins — Tier-1 runs per codeblock, so skipping codeblocks skips the cost — and exactly the work src/parallel.ts decomposes: the largest field is 911 independent codeblock tasks, each a pure function over its own byte slice, so one full decode can also fan across a worker pool within the field — a dimension neither WASM nor native OpenJPEG can use at all.

The pool wiring is deliberately not in this package (no Node APIs here, ever); it lives in @azohra/meteo.grib/j2k-node. Its decodeSampled rides decodeJ2kRegion directly; for full decodes, strategy: "codeblock" fans one field’s codeblocks across the whole pool, cutting its latency several-fold (736 → 133 ms measured on the largest field through an 8-worker pool) while saturated throughput ties per-field fan-out exactly. See JPEG 2000 and the pool for the pool’s API, sizing, and heap behaviour.

Two benches ship with the packages: single-thread full-decode timing over the corpus (tools/bench.ts), and the region bench behind J2K_REGION_BENCH=1 in test/region.test.ts.