perf(context,search): per-project session index and observation lookup index (#1418)

* perf(context): add per-project recent-sessions index

mem::context listed the entire mem:sessions scope on every session
start to find one project's 10 most recent sessions, then discarded
everything else. The iii file store rewrites and clones a whole scope
on every list, so this cost grows with total sessions across every
project, forever, on the highest-frequency path in the product.

Add a small per-project index (scope mem:idx:project-sessions, one key
per project) holding up to 50 recent session ids and startedAt
timestamps, capped and sorted newest-first. Maintain it wherever a new
session is written: event::session::started, api::session::start, the
implicit session create in mem::observe, and mem::import; remove
entries wherever a session is deleted: mem::evict, mem::forget, and
the bulk replace path in mem::import.

mem::context now reads the index and kv.get's only those sessions
instead of listing the whole scope. If a project has no index yet, it
falls back to the previous full-list behavior once and writes the
index from that result so subsequent calls stay bounded.

* perf(smart-search): add sharded obsId -> sessionId reverse index

findObservation's fallback path (used by memory_smart_search's
expandIds when no sessionIdHint is available, which is the common
case from the documented MCP call path) listed every session and
scanned each one's observation scope in batches of 5 until it found a
match. Worst case that is O(sessions) kv.get round-trips per id, times
up to 20 ids per call.

Add a reverse index from obsId to sessionId, sharded across 64 fixed
scopes (mem:idx:obs:<n>, bucket = hash(obsId) mod 64) so no single
scope grows with the corpus and the file store's whole-scope rewrite
on flush stays cheap. Maintain it wherever an observation is written
(mem::observe's per-observation hot path, mem::import) and remove it
wherever an observation is deleted (mem::evict, mem::forget,
mem::auto-forget, the bulk replace path in mem::import).

findObservation now tries the reverse index before falling back to the
session scan, and backfills the index when the fallback path finds a
match so the next lookup for that id is O(1).

* chore: credit co-authors

The memories-first observation lookup follows #1091, and #1230 traced
the expandIds path this index serves.

Co-authored-by: Erik Bogado <erikbogado@gmail.com>
Co-authored-by: cmondragon023 <cmondragon023@outlook.com>

* fix(session-index): close index consistency gaps from review

Three gaps in the new per-project session index and its context.ts
fallback could silently hide sessions from mem::context:

- addSessionToProjectIndex built a fresh index containing only the
  session being added when no index existed yet, discarding every
  other session already stored for that project. Cold start now
  seeds the index from the project's stored sessions before merging
  in the new entry.
- context.ts's fallback (full scan + write-back when the index is
  missing) ran outside any lock, so it could race a concurrent
  addSessionToProjectIndex call and overwrite its update. Both paths
  now go through ensureProjectSessionIndex, which rechecks under the
  same per-project lock before writing.
- The 50-entry cap picked the most recent sessions across all agents
  sharing a project, so an agent with less session volume than
  others could be pushed out of the index entirely and lose all
  context. The cap now reserves up to 10 slots per agent before
  filling the remainder by recency, and removeSessionFromProjectIndex
  replenishes from stored sessions when a removal drops the index
  below the cap instead of letting it shrink permanently.

ProjectSessionIndexEntry gained an optional agentId used only for
the fairness reservation; existing entries without it keep behaving
exactly as before (single implicit bucket).

Left the evict.ts observation reverse-index entries alone: that path
never deletes the observations themselves (mem::summarize creates a
summary for the recovered session first), matching the same
long-lived reverse-index behavior a normal event::session::stopped
leaves in place. Removing the mapping there would break expandIds
for observations mem::smart-search's own query path can still
legitimately return as hits.

* perf(session-index): refill only a full index after a removal

An index holding fewer than 50 entries already lists every session in
its project, so removing one needs no refill. Refilling only when the
index was full keeps deletions in small projects from listing the
whole sessions scope each time.

* fix(session-index): self-heal untracked writers and drop O(N^2) refill

Replay, migrate, and snapshot-restore wrote sessions directly to KV
without touching the per-project session index, so those sessions
were permanently invisible to mem::context. A failed sessions listing
during the index's cold-seed path was silently swallowed and replaced
with an empty array, so a transient state::list failure could wipe a
project's visible history. Session removal rescanned the full
sessions scope every time the index crossed the 50-entry cap, turning
bulk eviction or a replace-strategy import into an O(N^2) sweep over
the sessions scope.

Fix: replay, migrate, and snapshot-restore now call
addSessionToProjectIndex after writing each session row. The cold-seed
list no longer swallows a failed state::list, so a failure leaves the
index key absent instead of persisting an empty one. Session removal
now only drops the matching entry and never re-lists. A new
rebuildAllProjectSessionIndexes function lists every session once,
groups by project, and rewrites each project's index key; it runs
once at boot behind a KV.config generation marker and is registered as
mem::diagnostic::session-index-rebuild for use after bulk operations.

Tests exercise mem::forget, mem::evict, mem::import (replace),
mem::replay::import-jsonl, and mem::snapshot-restore through their
real registered functions, including an assertion that removing 100
sessions from one project performs at most one sessions listing.

* fix(session-index): unindex evicted observations and keep concurrent index changes across a rebuild

Evicting a stale session deleted the session record but left its
observations in the obsId to session reverse index, so expandIds could
still return an observation from the evicted session. The eviction now
removes those entries using the observations it already loaded.

The boot rebuild runs in the background and lists sessions once before
writing each project index. A session started or removed between that
listing and the write was overwritten by the older snapshot, and the
generation marker then stopped any later repair. Adds and removals made
while a rebuild is running are now recorded and applied to the rebuilt
index under the project lock.

* fix(session-index): drop the project index when an add cannot be written

A failed index write left the existing index in place without the new
session, and since the index was present no later read or boot rebuilt
it, so the session never reached mem::context. The add now deletes that
project index on a failed write, so the next read rebuilds it from the
session scan, and records the change first so a running rebuild keeps it.

---------

Co-authored-by: Erik Bogado <erikbogado@gmail.com>
Co-authored-by: cmondragon023 <cmondragon023@outlook.com>