SIGN IN SIGN UP

feat(streams): rollover block log and atomic stream → session cutover (#2216)

* bench(streams): experimental SQLite block strategies for stream→message cutover

Compares immutable packed rows, reusable generational slots and mutable
rollover blocks on billed rows, write time, page growth, replay, and the
cursor a restarted isolate recovers, with an atomic transactionSync
cutover to the message row. Numbers logged, invariants asserted.

* feat(streams): rollover block log and atomic stream → message cutover

- The Streams chunk log is mutable rollover blocks: an append grows the open
  block row (UPDATE) until 256 KB, then opens the next. Same one billed row
  per append; a stream of thousands of chunks is a handful of rows to
  delete. Schema v2 folds existing cf_agents_stream_chunks rows into blocks.
- writer.close({ commit, discard }) / error(reason, { ... }) settle, run the
  caller's synchronous writes and delete the stream's rows in one SQLite
  transaction. Session.__DO_NOT_USE_WILL_BREAK__sync().upsert() is the
  matching synchronous message write.
- ResumableStream.complete(id, { persist }) exposes the cutover;
  discardCompleted() drops a completed stream's rows once its message is
  persisted; start() reclaims completed streams a crash left behind.
- AIChatAgent and Think discard right after persisting (agent-tool child
  turns excepted: the parent tails their stored chunks), so the retention
  sweep no longer finds completed streams.

* test(streams): e2e crash matrix for the block log and cutover

ctx.abort() at each point that matters — before an append commits, right
after, during rollover (before/after commit), after persist before discard,
and inside/after the atomic cutover — then inspect the fresh instance. Each
restart finds either the exact committed prefix or the finished message.

* feat(chat): persist inside the cutover; remove the stream-buffer sweep

- ResumableStream.finish() leaves a finished stream live until cutover();
  cutover() settles it, runs the message write and deletes its rows in one
  transaction; finalizePending() settles when nothing is persisted;
  reclaim() on every start() replaces the alarm-driven sweep.
- AIChatAgent persists the turn's messages inside the cutover
  (persistMessages { _cutover }); Think does the same through
  _persistAssistantMessageWithCutover. The session change feed and
  auto-compaction run once the transaction commits.
- cleanupStreamBuffers / STREAM_CLEANUP_DELAY_SECONDS removed; the host
  _cleanupStreamBuffers callbacks stay as no-ops for persisted alarms.

* fix(ci): drop cleanupStreamBuffers from experimental agents; test typing

* fix(streams): lazy per-stream v1 fold, idempotent cutover, events after commit

- Fold cf_agents_stream_chunks rows into blocks one stream at a time on
  first touch (paged reads, whole-block inserts) instead of the whole log
  at startup; drop the table once empty.
- A settle with commit/discard is a no-op on a stream already terminal or
  deleted: commit does not run, nothing is discarded. ResumableStream's
  cutover falls back to a plain persist in that case.
- Terminal, deleted events and reader wakeups fire after the cutover
  transaction returns, never for a rolled-back commit.

* fix(ai-chat): cutover rides the instance, not a persistMessages argument

A subclass that overrides persistMessages and calls super with only the
messages dropped the internal _cutover option, so the message write and
the stream settlement fell back to two commits. The pending cutover now
lives on the instance keyed by the turn's message id: the first persist
that carries that message runs the cutover, a persist of other messages
takes the plain path, and the turn's end clears it. persistMessages'
public signature is unchanged.
M
Matt committed
dd09d44913e60aa519aac51e53b13eac2111c732
Parent: 2847a28
Committed by GitHub <noreply@github.com> on 9/8/2026, 4:37:04 PM