173 lines
7.6 KiB
Markdown
173 lines
7.6 KiB
Markdown
# WrNexus database system — complete design & plan
|
|
|
|
Goal: a batteries-included data layer — **schema, migrations, a sqlc-like typed
|
|
query generator, pluggable adapters (Postgres / MySQL / SQLite / MongoDB / …),
|
|
and tools to run queries and map results back into (validated) schemas.**
|
|
|
|
Reuses what already exists: the `@wrnexus/validation` `v` builder (for field
|
|
types + coercion + validation), the config/CLI/asset plumbing, and the
|
|
per-request `Context` (for `ctx.db`).
|
|
|
|
---
|
|
|
|
## 1. Packages
|
|
|
|
- `@wrnexus/db` — core: model/schema layer, the `Driver` interface, connection
|
|
management, the query runtime, result→schema mapping, migration runner.
|
|
- `@wrnexus/db-postgres`, `@wrnexus/db-mysql`, `@wrnexus/db-sqlite`,
|
|
`@wrnexus/db-mongo` — thin adapter packages (each wraps a proven driver:
|
|
`postgres`/`pg`, `mysql2`, `bun:sqlite`, `mongodb`). Installed on demand; the
|
|
core stays dependency-free.
|
|
- CLI verbs in `@wrnexus/cli`: `wrnexus db migrate|rollback|status|new|generate|studio`.
|
|
|
|
## 2. Schema / models (`app/db/schema.ts`)
|
|
|
|
Models are defined once, in TS, layered on the validation builder so the SAME
|
|
definition drives DDL, query result typing, and input validation:
|
|
|
|
```ts
|
|
import { table, v } from "@wrnexus/db";
|
|
export const users = table("users", {
|
|
id: v.id(), // pk, auto (serial / uuid per adapter)
|
|
email: v.string().email().unique(),
|
|
name: v.string().min(1),
|
|
age: v.number().int().optional(),
|
|
createdAt: v.timestamp().default("now"),
|
|
});
|
|
```
|
|
|
|
`table()` returns a Model with `.columns`, `.parse(row)` (coerce+validate a DB
|
|
row into a typed object), `.describe()` (for migrations), and adapter-agnostic
|
|
metadata (indexes, uniques, relations via `v.ref(users.id)`).
|
|
|
|
## 3. Adapters (`Driver` interface)
|
|
|
|
The core talks to every database through one small interface, so queries and
|
|
migrations are portable where possible:
|
|
|
|
```ts
|
|
interface Driver {
|
|
connect(url: string): Promise<Connection>;
|
|
dialect: "postgres" | "mysql" | "sqlite" | "mongo";
|
|
query(sql: string, params: unknown[]): Promise<Row[]>; // SQL adapters
|
|
exec(sql: string, params: unknown[]): Promise<{ rowCount: number }>;
|
|
transaction<T>(fn: (tx: Connection) => Promise<T>): Promise<T>;
|
|
// Mongo adapter implements a parallel collection API (find/insert/update/...)
|
|
// behind the same Model layer instead of SQL.
|
|
}
|
|
```
|
|
|
|
SQL adapters share one SQL builder/renderer with dialect quirks (`$1` vs `?`,
|
|
`RETURNING`, upserts). Mongo maps Model operations to collection calls. Connection
|
|
pooling + lifecycle handled by the core; `ctx.db` is a per-request handle,
|
|
`getDb()` an app-wide singleton.
|
|
|
|
## 4. Migrations (`app/db/migrations/*`)
|
|
|
|
Versioned, timestamped files with `up`/`down`. Two authoring modes:
|
|
|
|
- **SQL** — `0001_init.sql` (a `-- +up` / `-- +down` split), run as-is.
|
|
- **From models** — `wrnexus db new --from-models` diffs `schema.ts` against the
|
|
last applied state and emits the SQL for you to review.
|
|
|
|
`wrnexus db migrate` applies pending migrations inside a transaction and records
|
|
them in a `_wire_migrations` table; `rollback` runs `down`; `status` lists state.
|
|
Dialect-aware SQL generation for create/alter/index.
|
|
|
|
## 5. sqlc-like typed queries (`app/db/queries/*.sql` → generated TS)
|
|
|
|
Write annotated SQL; `wrnexus db generate` produces typed functions whose
|
|
parameters and results are checked, and whose rows are mapped through the model
|
|
schemas:
|
|
|
|
```sql
|
|
-- name: GetUserByEmail :one
|
|
SELECT * FROM users WHERE email = :email;
|
|
|
|
-- name: ListActiveUsers :many
|
|
SELECT id, name FROM users WHERE age >= :minAge ORDER BY name;
|
|
|
|
-- name: CreateUser :exec
|
|
INSERT INTO users (email, name) VALUES (:email, :name);
|
|
```
|
|
|
|
→ `app/db/queries.gen.ts`:
|
|
|
|
```ts
|
|
export const GetUserByEmail = (db, args: { email: string }): Promise<User | null> => …
|
|
export const ListActiveUsers = (db, args: { minAge: number }): Promise<Pick<User,"id"|"name">[]> => …
|
|
```
|
|
|
|
The generator parses the SQL (params from `:name`, result columns), infers types
|
|
from the referenced models, renders dialect-specific SQL, and wraps execution +
|
|
`model.parse` result mapping. `:one|:many|:exec` set the return shape. For Mongo,
|
|
the equivalent is a typed collection API (`db.users.find(...)`), since raw SQL
|
|
doesn't apply.
|
|
|
|
## 6. Running queries + result → schema
|
|
|
|
Every result is coerced/validated through the model's `.parse` (same engine as
|
|
form/API validation), so the data you get back matches your schema or throws a
|
|
clear error. Helpers: `db.one/all/exec`, transactions via `db.tx(async t => …)`,
|
|
and the generated functions above.
|
|
|
|
## 7. Config & security
|
|
|
|
- `wrnexus.config.ts` `db: { driver: "postgres", url: process.env.DATABASE_URL }`
|
|
(multiple named connections supported). Never hard-code secrets.
|
|
- All queries are **parameterized** (no string interpolation of input); the
|
|
generator forbids raw interpolation. Migrations run only from disk files.
|
|
- `ctx.db` in pages/API/middleware; pooled; closed on shutdown.
|
|
|
|
## 8. `wrnexus db studio` (optional, later)
|
|
|
|
A local Artifact/web UI to browse tables + run read-only queries in dev.
|
|
|
|
---
|
|
|
|
## Phased implementation plan (each phase: built, tested, documented)
|
|
|
|
1. **Core + SQLite adapter** — `@wrnexus/db`, `table()`/`v` model layer, `Driver`
|
|
interface, `bun:sqlite` adapter, `db.one/all/exec/tx`, result→schema mapping.
|
|
(SQLite first = zero external service to test against.)
|
|
2. **Migrations** ✅ done — runner + `_wire_migrations`, `wrnexus db migrate/rollback/status/new`,
|
|
`.sql` (`-- +up`/`-- +down`) + `--from-models` DDL generation. (Model-diffing beyond the
|
|
initial CREATE TABLE is a later enhancement; needs topological ordering by FK refs.)
|
|
3. **sqlc-like generator** ✅ done — `.sql` parsing (`-- name: X :one/:many/:exec`),
|
|
param/result type inference from models (comparisons, INSERT column lists, SELECT
|
|
list, aggregates → number), `wrnexus db generate` + auto-regen in `wrnexus build`.
|
|
(Postgres `$N` placeholders + `::casts` handled when the pg adapter lands.)
|
|
4. **Postgres + MySQL adapters** ✅ done — built on Bun's native `Bun.SQL` (zero
|
|
external driver), `@wrnexus/db/postgres` + `/mysql`. Driver logic validated via
|
|
Bun.sql's SQLite backend; dialect DDL (`SERIAL`/`BOOLEAN`/`JSONB`) + generator
|
|
`$N` placeholders verified. Live connect/query needs a running server to test.
|
|
5. **MongoDB adapter** ✅ built (unverified) — `@wrnexus/db/mongo`: a document
|
|
collection API (`find/findOne/insert/update/delete/count`) keyed by models, reads
|
|
coerced through `model.parse`, `_id`→`id`. Lazily imports `mongodb` (optional dep).
|
|
Not wired into the SQL singleton/migrations/generator (Mongo isn't SQL). Needs
|
|
`mongodb` + a server to test.
|
|
6. **Polish** — ✅ FK-topological ordering in `--from-models` (referenced tables
|
|
first). Deferred: `db studio` (web UI), richer relations/joins helpers.
|
|
|
|
### Transaction model (refactor)
|
|
|
|
`Driver` now exposes `transaction(fn)` (single reserved connection) instead of
|
|
`begin/commit/rollback`, so pooled drivers (Bun.sql) keep a transaction on one
|
|
connection. `Db.tx` opens it; nested `tx` reuses the current one.
|
|
|
|
### Adapter resolution
|
|
|
|
`@wrnexus/db/connect` `connectFromConfig({driver,url}, appRoot?)` maps a driver name
|
|
to its adapter (sqlite/postgres/mysql) and resolves `file:` URLs against the app
|
|
root. Used by `wrnexus db`, the dev server, and prod.
|
|
|
|
## Open decisions (need your call before phase 1)
|
|
|
|
- **Query authoring:** sqlc-style annotated `.sql` + generator (recommended, matches
|
|
your ask) vs a fluent TS query builder vs support both.
|
|
- **Schema source of truth:** TS models (`schema.ts`, recommended — one definition
|
|
for DDL + validation + result typing) vs raw SQL DDL (`schema.sql`, closer to
|
|
sqlc) vs both.
|
|
- **First adapter to build:** SQLite (recommended — testable with no server) vs
|
|
Postgres vs MySQL vs Mongo.
|