Ingesta — An Import Wizard, and the Workspace That Makes Its Mistakes Fixable
Getting a spreadsheet into a database is the easy half. The interesting half is the place in between, where the data is still yours to change.
The problem
Parsing a workbook, inferring a schema and inserting the rows is easy once and impossible to trust. Dates arrive as text, numbers carry thousands separators, the same entity is keyed three different ways — and the usual response is to skip the middle entirely and fix everything in the database with hand-written SQL against a table that has already been committed to. That is unreviewable, hard to take back, and nobody writes down what they changed.
The solution
Split the problem in two. A one-shot seven-stage wizard parses workbooks and creates tables; the browser holds the file list and run history while the server opens a short-lived connection per call. Past the parser everything speaks one `Grid` shape with semantic types that are mapped to a real engine only when the DDL is built, so no layer above the dialect knows whether the destination is PostgreSQL, MySQL, SQL Server or SQLite.
Outcomes
- One `Grid` shape carries the data everywhere past the parser, with semantic column types that reach an engine only at DDL-build time.
- Reshaping steps are plain serialisable values rather than closures, so a step can be stored, shown, re-run and diffed.
- Effects are counted while an operation runs — rows in and out, cells changed — so the history list reports measurements rather than estimates.
+3 more in the write-up
Inside the write-up
- The gap nobody builds
- One shape, one set of types
- Operations are values, not actions
- Rollback is replay, not undo
- The boring part is the guardrails
- Checked by scripts, not vibes