first commit
This commit is contained in:
@@ -0,0 +1,172 @@
|
||||
# WrNexus database system — complete design & plan
|
||||
|
||||
Goal: a batteries-included data layer — **schema, migrations, a sqlc-like typed
|
||||
query generator, pluggable adapters (Postgres / MySQL / SQLite / MongoDB / …),
|
||||
and tools to run queries and map results back into (validated) schemas.**
|
||||
|
||||
Reuses what already exists: the `@wrnexus/validation` `v` builder (for field
|
||||
types + coercion + validation), the config/CLI/asset plumbing, and the
|
||||
per-request `Context` (for `ctx.db`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Packages
|
||||
|
||||
- `@wrnexus/db` — core: model/schema layer, the `Driver` interface, connection
|
||||
management, the query runtime, result→schema mapping, migration runner.
|
||||
- `@wrnexus/db-postgres`, `@wrnexus/db-mysql`, `@wrnexus/db-sqlite`,
|
||||
`@wrnexus/db-mongo` — thin adapter packages (each wraps a proven driver:
|
||||
`postgres`/`pg`, `mysql2`, `bun:sqlite`, `mongodb`). Installed on demand; the
|
||||
core stays dependency-free.
|
||||
- CLI verbs in `@wrnexus/cli`: `wrnexus db migrate|rollback|status|new|generate|studio`.
|
||||
|
||||
## 2. Schema / models (`app/db/schema.ts`)
|
||||
|
||||
Models are defined once, in TS, layered on the validation builder so the SAME
|
||||
definition drives DDL, query result typing, and input validation:
|
||||
|
||||
```ts
|
||||
import { table, v } from "@wrnexus/db";
|
||||
export const users = table("users", {
|
||||
id: v.id(), // pk, auto (serial / uuid per adapter)
|
||||
email: v.string().email().unique(),
|
||||
name: v.string().min(1),
|
||||
age: v.number().int().optional(),
|
||||
createdAt: v.timestamp().default("now"),
|
||||
});
|
||||
```
|
||||
|
||||
`table()` returns a Model with `.columns`, `.parse(row)` (coerce+validate a DB
|
||||
row into a typed object), `.describe()` (for migrations), and adapter-agnostic
|
||||
metadata (indexes, uniques, relations via `v.ref(users.id)`).
|
||||
|
||||
## 3. Adapters (`Driver` interface)
|
||||
|
||||
The core talks to every database through one small interface, so queries and
|
||||
migrations are portable where possible:
|
||||
|
||||
```ts
|
||||
interface Driver {
|
||||
connect(url: string): Promise<Connection>;
|
||||
dialect: "postgres" | "mysql" | "sqlite" | "mongo";
|
||||
query(sql: string, params: unknown[]): Promise<Row[]>; // SQL adapters
|
||||
exec(sql: string, params: unknown[]): Promise<{ rowCount: number }>;
|
||||
transaction<T>(fn: (tx: Connection) => Promise<T>): Promise<T>;
|
||||
// Mongo adapter implements a parallel collection API (find/insert/update/...)
|
||||
// behind the same Model layer instead of SQL.
|
||||
}
|
||||
```
|
||||
|
||||
SQL adapters share one SQL builder/renderer with dialect quirks (`$1` vs `?`,
|
||||
`RETURNING`, upserts). Mongo maps Model operations to collection calls. Connection
|
||||
pooling + lifecycle handled by the core; `ctx.db` is a per-request handle,
|
||||
`getDb()` an app-wide singleton.
|
||||
|
||||
## 4. Migrations (`app/db/migrations/*`)
|
||||
|
||||
Versioned, timestamped files with `up`/`down`. Two authoring modes:
|
||||
|
||||
- **SQL** — `0001_init.sql` (a `-- +up` / `-- +down` split), run as-is.
|
||||
- **From models** — `wrnexus db new --from-models` diffs `schema.ts` against the
|
||||
last applied state and emits the SQL for you to review.
|
||||
|
||||
`wrnexus db migrate` applies pending migrations inside a transaction and records
|
||||
them in a `_wire_migrations` table; `rollback` runs `down`; `status` lists state.
|
||||
Dialect-aware SQL generation for create/alter/index.
|
||||
|
||||
## 5. sqlc-like typed queries (`app/db/queries/*.sql` → generated TS)
|
||||
|
||||
Write annotated SQL; `wrnexus db generate` produces typed functions whose
|
||||
parameters and results are checked, and whose rows are mapped through the model
|
||||
schemas:
|
||||
|
||||
```sql
|
||||
-- name: GetUserByEmail :one
|
||||
SELECT * FROM users WHERE email = :email;
|
||||
|
||||
-- name: ListActiveUsers :many
|
||||
SELECT id, name FROM users WHERE age >= :minAge ORDER BY name;
|
||||
|
||||
-- name: CreateUser :exec
|
||||
INSERT INTO users (email, name) VALUES (:email, :name);
|
||||
```
|
||||
|
||||
→ `app/db/queries.gen.ts`:
|
||||
|
||||
```ts
|
||||
export const GetUserByEmail = (db, args: { email: string }): Promise<User | null> => …
|
||||
export const ListActiveUsers = (db, args: { minAge: number }): Promise<Pick<User,"id"|"name">[]> => …
|
||||
```
|
||||
|
||||
The generator parses the SQL (params from `:name`, result columns), infers types
|
||||
from the referenced models, renders dialect-specific SQL, and wraps execution +
|
||||
`model.parse` result mapping. `:one|:many|:exec` set the return shape. For Mongo,
|
||||
the equivalent is a typed collection API (`db.users.find(...)`), since raw SQL
|
||||
doesn't apply.
|
||||
|
||||
## 6. Running queries + result → schema
|
||||
|
||||
Every result is coerced/validated through the model's `.parse` (same engine as
|
||||
form/API validation), so the data you get back matches your schema or throws a
|
||||
clear error. Helpers: `db.one/all/exec`, transactions via `db.tx(async t => …)`,
|
||||
and the generated functions above.
|
||||
|
||||
## 7. Config & security
|
||||
|
||||
- `wrnexus.config.ts` `db: { driver: "postgres", url: process.env.DATABASE_URL }`
|
||||
(multiple named connections supported). Never hard-code secrets.
|
||||
- All queries are **parameterized** (no string interpolation of input); the
|
||||
generator forbids raw interpolation. Migrations run only from disk files.
|
||||
- `ctx.db` in pages/API/middleware; pooled; closed on shutdown.
|
||||
|
||||
## 8. `wrnexus db studio` (optional, later)
|
||||
|
||||
A local Artifact/web UI to browse tables + run read-only queries in dev.
|
||||
|
||||
---
|
||||
|
||||
## Phased implementation plan (each phase: built, tested, documented)
|
||||
|
||||
1. **Core + SQLite adapter** — `@wrnexus/db`, `table()`/`v` model layer, `Driver`
|
||||
interface, `bun:sqlite` adapter, `db.one/all/exec/tx`, result→schema mapping.
|
||||
(SQLite first = zero external service to test against.)
|
||||
2. **Migrations** ✅ done — runner + `_wire_migrations`, `wrnexus db migrate/rollback/status/new`,
|
||||
`.sql` (`-- +up`/`-- +down`) + `--from-models` DDL generation. (Model-diffing beyond the
|
||||
initial CREATE TABLE is a later enhancement; needs topological ordering by FK refs.)
|
||||
3. **sqlc-like generator** ✅ done — `.sql` parsing (`-- name: X :one/:many/:exec`),
|
||||
param/result type inference from models (comparisons, INSERT column lists, SELECT
|
||||
list, aggregates → number), `wrnexus db generate` + auto-regen in `wrnexus build`.
|
||||
(Postgres `$N` placeholders + `::casts` handled when the pg adapter lands.)
|
||||
4. **Postgres + MySQL adapters** ✅ done — built on Bun's native `Bun.SQL` (zero
|
||||
external driver), `@wrnexus/db/postgres` + `/mysql`. Driver logic validated via
|
||||
Bun.sql's SQLite backend; dialect DDL (`SERIAL`/`BOOLEAN`/`JSONB`) + generator
|
||||
`$N` placeholders verified. Live connect/query needs a running server to test.
|
||||
5. **MongoDB adapter** ✅ built (unverified) — `@wrnexus/db/mongo`: a document
|
||||
collection API (`find/findOne/insert/update/delete/count`) keyed by models, reads
|
||||
coerced through `model.parse`, `_id`→`id`. Lazily imports `mongodb` (optional dep).
|
||||
Not wired into the SQL singleton/migrations/generator (Mongo isn't SQL). Needs
|
||||
`mongodb` + a server to test.
|
||||
6. **Polish** — ✅ FK-topological ordering in `--from-models` (referenced tables
|
||||
first). Deferred: `db studio` (web UI), richer relations/joins helpers.
|
||||
|
||||
### Transaction model (refactor)
|
||||
|
||||
`Driver` now exposes `transaction(fn)` (single reserved connection) instead of
|
||||
`begin/commit/rollback`, so pooled drivers (Bun.sql) keep a transaction on one
|
||||
connection. `Db.tx` opens it; nested `tx` reuses the current one.
|
||||
|
||||
### Adapter resolution
|
||||
|
||||
`@wrnexus/db/connect` `connectFromConfig({driver,url}, appRoot?)` maps a driver name
|
||||
to its adapter (sqlite/postgres/mysql) and resolves `file:` URLs against the app
|
||||
root. Used by `wrnexus db`, the dev server, and prod.
|
||||
|
||||
## Open decisions (need your call before phase 1)
|
||||
|
||||
- **Query authoring:** sqlc-style annotated `.sql` + generator (recommended, matches
|
||||
your ask) vs a fluent TS query builder vs support both.
|
||||
- **Schema source of truth:** TS models (`schema.ts`, recommended — one definition
|
||||
for DDL + validation + result typing) vs raw SQL DDL (`schema.sql`, closer to
|
||||
sqlc) vs both.
|
||||
- **First adapter to build:** SQLite (recommended — testable with no server) vs
|
||||
Postgres vs MySQL vs Mongo.
|
||||
Reference in New Issue
Block a user