Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/design-decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,61 @@ the existing per-commit GC bounds in-memory chain growth.

---

### 12h. REPL holds `Vec<Connection>` rather than a single `Database` (Phase 11.11a)

**Decision.** The `sqlrite` REPL binary keeps its state in a small
`ReplState` struct holding a `Vec<Connection>`, a parallel
`Vec<String>` of stable display names (`A`, `B`, …), and a
`usize` index pointing at the active handle. SQL dispatch goes
through [`Connection::execute_with_render`](../src/connection.rs)
(new in this slice); meta-commands either operate on the
underlying `Database` via `ReplState::lock_active()` or mutate
the connection list itself (`.spawn`, `.use`, `.conns`).

**Why migrate from `&mut Database`.** `.spawn` only makes sense
across multiple `Connection`s sharing the same
`Arc<Mutex<Database>>` — that's the entire point of
`Connection::connect()`. The previous REPL owned a single
`Database` by value, which made siblings unrepresentable. The
migration is mostly mechanical: the SQL dispatch routes through
the active connection, and every existing meta-command keeps
operating on the database directly by grabbing the mutex guard.

**Why `Vec<Connection>` and not a `HashMap<String, Connection>`.**
Handle creation order is the only ordering that matters for
demos. A `Vec` keeps `.conns` output deterministic without
sorting; the parallel `names` vector keeps the name → index
lookup O(handles), which is fine for the realistic upper bound
(a handful of siblings in a demo session). Skipping the map also
avoids a `String` key per access on the hot SQL path.

**Why `.open` collapses every sibling back to a single handle.**
Replacing the underlying `Database` via the mutex guard works in
place — every sibling sees the new content. But sibling handles
typically hold per-connection MVCC transaction state
(`Connection::concurrent_tx`) keyed by the *previous* clock /
`MvStore`; after a `.open` swap, that state is meaningless and
attempting to commit would walk the wrong active-tx registry.
Dropping siblings on `.open` is cleaner than retroactively
invalidating their tx state, and matches the user's mental model
("`.open` is a fresh start").

**Why `execute_with_render` instead of pre-parsing in the REPL.**
The REPL needs both the rendered SELECT table and the BEGIN
CONCURRENT routing. The old `process_command_with_render` gives
the former but bypasses the per-connection MVCC dispatch
(`BEGIN CONCURRENT` / `COMMIT` / `ROLLBACK` interception, the
snapshot-read swap). The new method mirrors `Connection::execute`
but threads the `CommandOutput` struct through every branch —
the BEGIN/COMMIT/ROLLBACK arms produce
`CommandOutput { status, rendered: None }`; the
process-command path produces the full output. One method, one
dispatch tree, every REPL line goes through it.

**Plan-doc reference.** [`concurrent-writes-plan.md`](concurrent-writes-plan.md) §10.8 (REPL `.spawn` meta-command and demos).

---

## Query execution

### 13. `NULL`-as-false in `WHERE` clauses
Expand Down
16 changes: 14 additions & 2 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -685,9 +685,21 @@ MVCC commits now leave a typed log-record frame in the WAL on top of the existin

Each secondary-index entry becomes its own `RowVersion`. Turso explicitly punted on this; SQLRite's v0 will reject `CREATE INDEX` while `journal_mode = mvcc`.

### Phase 11.11 — REPL `.spawn` + bench workload *(planned)*
### Phase 11.11a — REPL `.spawn` for interactive `BEGIN CONCURRENT` demos

REPL `.spawn` meta-command for interactive `BEGIN CONCURRENT` demos. New "N concurrent writers" benchmark workload pitting SQLRite-MVCC against SQLite + DuckDB on disjoint-row write throughput. Plus Go SDK multi-handle work (cross-pool sibling shape).
Lift the REPL from a single `Database` to a `Vec<Connection>` so users can mint sibling handles in-session and step through cross-handle MVCC scenarios. The prompt now shows the active handle (`sqlrite[A]> ` / `sqlrite[B]> `) so it's always obvious which connection is about to execute the next line.

- **`.spawn`** mints a sibling off the active handle (via `Connection::connect`) and switches to it. Each handle gets a stable letter name (`A`, `B`, `C`, …, `Z`, then `AA`, `AB`).
- **`.use <NAME>`** switches the active handle (case-insensitive); errors with the list of valid names if the target is unknown.
- **`.conns`** lists every handle, marks the active one with `*`, and tags any handle that holds an open `BEGIN CONCURRENT` so demos can show the conflict-detection state at a glance.
- **`.open`** collapses every sibling back to a single handle named `A` so the new database doesn't strand siblings pointing at the old one.
- New [`Connection::execute_with_render`](../src/connection.rs) returns a `CommandOutput` instead of a bare status string, so the REPL's SQL dispatch routes through `Connection` (catching `BEGIN CONCURRENT` / `COMMIT` / `ROLLBACK` and the in-flight tx swap) while still printing the prettytable for `SELECT`. The old non-render `execute` stays for callers that don't need it.

The downstream "N concurrent writers" benchmark workload (originally bundled into 11.11) is its own follow-up: it touches the `benchmarks/` harness, links SQLite + DuckDB drivers, and is much heavier than this slice.

### Phase 11.11b — `N concurrent writers` benchmark workload *(planned)*

New benchmark in [`benchmarks/`](../benchmarks/) that pits SQLRite-MVCC against SQLite + DuckDB on a disjoint-row "N writers, mostly disjoint rows" scenario. Slots into the existing SQLR-16 harness as a Group D differentiator workload. Also includes Go SDK multi-handle work (cross-pool sibling shape) — see the 11.8 note for why that's a separate slice.

### Phase 11.12 — Docs *(planned, plan-doc "Phase 10.9")*

Expand Down
8 changes: 5 additions & 3 deletions docs/smoke-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,11 @@ Enter .exit to quit.
Enter .help for usage hints.
Connected to a transient in-memory database.
Use '.open FILENAME' to reopen on a persistent database.
sqlrite>
sqlrite[A]>
```

(The `[A]` suffix is the active connection handle — Phase 11.11a added `.spawn` / `.use` / `.conns` for multi-handle demos.)

(The version line in the banner tracks the current build — `cargo run` always shows the live value, so don't be surprised if it's later than what's printed here.)

Also verify the help text is detailed:
Expand All @@ -58,7 +60,7 @@ Should print the project description, the meta-command table, and a summary of s
sqlrite> .help
```

Expect the 5-command list (`.help`, `.open`, `.save`, `.tables`, `.exit`) and a note that `.read` / `.ast` aren't implemented.
Expect the 9-command list (`.help`, `.open`, `.save`, `.tables`, `.ask`, `.spawn`, `.use`, `.conns`, `.exit`) and a note that `.read` / `.ast` aren't implemented.

### 1.3 Create a table

Expand Down Expand Up @@ -585,7 +587,7 @@ When you want a fast before/after comparison for a change, run this condensed ch
- [ ] `cargo run --bin sqlrite-mcp -- --help` prints the MCP server CLI without crashing — quick check that the stdio_redirect dance still works
- [ ] `cargo run -- --help` prints the full description + meta-command table + SQL surface (not just `-h` / `-V`)
- [ ] `cargo run -- somefile.sqlrite` on a non-existent path creates the file and enters the REPL with auto-save on
- [ ] REPL launches, `.help` shows 5 commands
- [ ] REPL launches, `.help` shows 9 commands
- [ ] `.tables` in a populated DB prints one name per line
- [ ] CREATE TABLE + INSERT + SELECT `*` work in memory
- [ ] `SELECT ... WHERE col = literal` on a UNIQUE column returns the right row (index probe path)
Expand Down
32 changes: 31 additions & 1 deletion docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ Meta commands start with a dot and don't need a trailing semicolon.
| `.open FILENAME` | Open (or create) a `.sqlrite` file. From this point on, every committing SQL statement auto-saves. |
| `.save FILENAME` | Force-flush the current DB to `FILENAME`. Rarely needed — auto-save makes this redundant when it's the active file. Useful for "save as" to a different path. |
| `.tables` | List tables in the current database, sorted alphabetically |
| `.ask QUESTION` | Natural-language → SQL via the configured LLM. Requires `SQLRITE_LLM_API_KEY`. |
| `.spawn` | Mint a sibling connection sharing the same backing database. Switches to it. (Phase 11.11a) |
| `.use NAME` | Switch the active handle (case-insensitive). (Phase 11.11a) |
| `.conns` | List every active handle; marks the current one with `*` and flags handles in an open `BEGIN CONCURRENT`. (Phase 11.11a) |
| `.read` / `.ast` | Not yet implemented |

### `.open` semantics
Expand All @@ -31,7 +35,33 @@ Meta commands start with a dot and don't need a trailing semicolon.
- If `FILENAME` doesn't exist: create an empty database at that path (auto-save enabled immediately).
- If `FILENAME` exists but is not a valid SQLRite database: reject with a `bad magic bytes` error — the REPL stays in its previous state.

Only one database is active at a time. A subsequent `.open` replaces the in-memory state.
Only one database is active at a time. A subsequent `.open` replaces the in-memory state and **collapses every sibling handle minted via `.spawn` back to a single one** (named `A`) — siblings pointing at the previous database would be stranded otherwise.

### Multi-handle mode (Phase 11.11a)

The REPL holds a vector of `Connection`s; the prompt always shows which one is active: `sqlrite[A]> `, `sqlrite[B]> `, etc.

- `.spawn` mints a new sibling off the active handle and switches to it. Each new handle gets the next letter in sequence (`A`, `B`, `C`, …, `Z`, `AA`, `AB`).
- `.use NAME` switches the active handle. The next SQL line runs on that connection.
- `.conns` shows the current roster, with `*` next to the active handle and `(BEGIN CONCURRENT)` next to any handle holding an open concurrent transaction.

A worked demo (assumes `PRAGMA journal_mode = mvcc;`):

```text
sqlrite[A]> CREATE TABLE t (id INTEGER PRIMARY KEY, v INTEGER);
sqlrite[A]> INSERT INTO t (id, v) VALUES (1, 0);
sqlrite[A]> .spawn
Spawned sibling handle 'B' and switched to it. 2 handles open.
sqlrite[B]> .use A
sqlrite[A]> BEGIN CONCURRENT;
sqlrite[A]> UPDATE t SET v = 100 WHERE id = 1;
sqlrite[A]> .use B
sqlrite[B]> BEGIN CONCURRENT;
sqlrite[B]> UPDATE t SET v = 200 WHERE id = 1;
sqlrite[B]> COMMIT;
sqlrite[B]> .use A
sqlrite[A]> COMMIT; -- Busy: write-write conflict on t/1
```

## Supported SQL

Expand Down
59 changes: 57 additions & 2 deletions src/connection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -297,13 +297,42 @@ impl Connection {
}
}

/// Phase 11.11a — same as [`Connection::execute`], but returns
/// the full [`CommandOutput`] (status + optional pre-rendered
/// prettytable for `SELECT`). The REPL needs this to print the
/// table the engine produced *and* the status line in one
/// pass, while still routing `BEGIN CONCURRENT` / `COMMIT` /
/// `ROLLBACK` through the per-connection MVCC state.
///
/// `BEGIN` / `COMMIT` / `ROLLBACK` carry no rendered output —
/// they return `CommandOutput { status, rendered: None }`.
pub fn execute_with_render(&mut self, sql: &str) -> Result<crate::sql::CommandOutput> {
let intent = concurrent_tx_intent(sql);
let has_tx = self.concurrent_tx_is_open();
let status = match intent {
ConcurrentTxIntent::Begin => self.begin_concurrent()?,
ConcurrentTxIntent::Commit if has_tx => self.commit_concurrent()?,
ConcurrentTxIntent::Rollback if has_tx => self.rollback_concurrent()?,
ConcurrentTxIntent::None
| ConcurrentTxIntent::Commit
| ConcurrentTxIntent::Rollback => return self.execute_dispatch_with_render(sql),
};
Ok(crate::sql::CommandOutput {
status,
rendered: None,
})
}

/// Phase 11.5 — cheap probe used by [`Connection::execute`]
/// (and [`Statement::query`]) to decide whether to route
/// through the concurrent-tx dispatch. Acquires the
/// `concurrent_tx` mutex briefly; never blocks for a
/// meaningful amount of time because the only other lockers
/// are this connection's own writers.
fn concurrent_tx_is_open(&self) -> bool {
///
/// Public so the REPL can render per-handle tx state in
/// `.conns` output (Phase 11.11a).
pub fn concurrent_tx_is_open(&self) -> bool {
self.lock_concurrent_tx().is_some()
}

Expand Down Expand Up @@ -407,6 +436,20 @@ impl Connection {
}
}

/// Phase 11.11a — render-aware twin of
/// [`Connection::execute_dispatch`]. Same branching, but the
/// non-concurrent path calls `process_command_with_render` and
/// the concurrent path goes through
/// [`Connection::execute_in_concurrent_tx_with_render`].
fn execute_dispatch_with_render(&mut self, sql: &str) -> Result<crate::sql::CommandOutput> {
if self.concurrent_tx_is_open() {
self.execute_in_concurrent_tx_with_render(sql)
} else {
let mut db = self.lock();
crate::sql::process_command_with_render(sql, &mut db)
}
}

/// Phase 11.4 — opens a `BEGIN CONCURRENT` transaction on this
/// connection. Allocates a new `TxHandle` (which advances the
/// MVCC clock by one), deep-clones the live tables into the
Expand Down Expand Up @@ -630,6 +673,18 @@ impl Connection {
/// new tables to the live database without a separate merge
/// pass).
fn execute_in_concurrent_tx(&mut self, sql: &str) -> Result<String> {
self.execute_in_concurrent_tx_with_render(sql)
.map(|o| o.status)
}

/// Render-aware twin of [`Connection::execute_in_concurrent_tx`].
/// Same swap-based dispatch; the only difference is the inner
/// call goes through `process_command_with_render` so the
/// caller gets the rendered SELECT table (Phase 11.11a).
fn execute_in_concurrent_tx_with_render(
&mut self,
sql: &str,
) -> Result<crate::sql::CommandOutput> {
let intent = legacy_tx_intent(sql);
if matches!(intent, LegacyTxIntent::Begin) {
return Err(SQLRiteError::General(
Expand Down Expand Up @@ -672,7 +727,7 @@ impl Connection {
tables: HashMap::new(),
});

let result = crate::sql::process_command(sql, &mut db);
let result = crate::sql::process_command_with_render(sql, &mut db);

// Unwind in reverse: take the dummy txn off (don't restore
// anything from it), swap the tables back.
Expand Down
Loading
Loading