How Anyi works

What Anyi’s own code does

• The grid (packages/data-grid, plain JavaScript): a virtualised table that draws only the rows and columns in view from a pool of row elements moved by one transform, fetches cells in blocks under a fixed cell budget (100,000 by default), and owns its scrollbars, keyboard model, spreadsheet-style selection and copy, column resizing, and header drags that keep column groups whole.
• The pivot engine: groups rows into a tree of totals, in memory for small sources or from partial aggregates that DuckDB computes for large ones; three layouts (Tree, Columns, Level) over the same groups; sorting that reorders groups only within their parent.
• Query translation: one description of a filter or sort becomes both SQL for DuckDB and a JavaScript predicate for in-memory data, with tests that run both against the same rows (missing text, big integers, a datetime’s whole day).
• The app (apps/anyi, React): the Grid and Column Inspectors, the field tree, the filter builder, source details with type conversion as a file opens, saved views, and CSV export.
• The interface library (packages/keaiui): the buttons, menus, pickers, checkboxes and popovers shared with the rest of Tanji Systems.

What the libraries do

• DuckDB-wasm runs SQL in the browser: it reads Parquet and parses CSV, JSON and Arrow files, and computes the filters, sorts and grouped totals for large sources.
• hyparquet reads Parquet file footers (row counts, statistics) and lets a local Parquet file of any size be browsed in place.
• Apache Arrow carries query results as columns.
• React and valtio render the app around the grid and hold its state; dnd-kit handles dragging in lists; Vite and Tailwind build it.

Decisions and tradeoffs

Everything in the browser

Data never leaves the device, and there is nothing to run or pay for. The cost is the browser’s memory: an imported CSV must fit DuckDB-wasm’s, and a filtered or sorted result keeps at most 6,000,000 rows (Anyi says when it cut one short).

Pivots from partial aggregates

For a large source, DuckDB groups the rows once, at the finest level the pivot needs, and returns sums, counts, minima and maxima per group; the pivot engine adds those up for every level above, so opening or closing a group, or changing the layout, asks nothing of the database. Medians do not add up that way, so DuckDB computes them for every level at once with GROUPING SETS.

A grid in plain JavaScript

Scrolling redraws many times a second, so the grid core avoids a framework’s render cycle: each frame only writes to the DOM, never reading layout in the same pass, and a repository check fails the build on code that would force the browser to lay out mid-loop. React is used where it fits, for the panels around it.

Remote Parquet, read once

Public datasets are Parquet files. Opening one reads only its footers, by HTTP range requests; a file is downloaded the first time its rows are needed and kept in the browser’s Cache Storage, so the next visit does not download it again. Anyi’s own small datasets are served from its own site.

Benchmark

Measured with apps/anyi/frontend/scripts/bench.mjs: Chromium opens a dataset in a fresh profile (cold: nothing cached, including DuckDB’s WebAssembly), then reloads it (cached). First rows runs from navigation to the first rows drawn; grouping is the time Anyi reports for DuckDB’s grouped query. Medians of three runs, on 11 November 2026.

Machine: Apple M2, 8 cores, 24 GB, macOS 26; Chromium 149 (Playwright, headless), against a production build served locally (vite preview), so the flights file came over this machine’s internet connection and the others from localhost. The cold flights time includes downloading the file; the cold Bikeshare time is mostly loading DuckDB itself.

Rerun it: npm run build -w @tanji-systems/anyi-frontend, serve the build (npx vite preview --port 3027 in apps/anyi/frontend), then node scripts/bench.mjs http://localhost:3027/ 3 flights_2015_grouped bikeshare_by_hour orders_rows_cols. It prints the numbers with the machine and browser. These are timings on one machine, not a comparison with other tools.