DeepSeek Harness in Depth

Storage in DeepSeek Harness is layered: a backend owns a medium, the domain layer adds schema and change events, and session persistence is a separate, event-sourced log. The hub does no I/O itself. The official design notes live in docs/persistence-catalog.md (the generated event catalog) and docs/subsystems/persistence.md.

Package	Role	Registered as / ctx key
`packages/storage/storage`	KV hub + named backend registry	`ctx.storage`
`packages/storage/storage-domain`	schema-validated, change-emitting KV domains	`ctx.storageDomain`, mounts `domain` form
`packages/storage/storage-json`	one human-readable file per unit	backend `json`
`packages/storage/storage-sqlite`	one DB file, document-per-row tables	backend `sqlite`
`packages/session/session-persistence`	durable session-log Service Definition	`ctx.sessionPersistence`
`packages/session/session-persistence-jsonl`	append JSONL per-session artifact	implements it
`packages/session/session-persistence-sqlite`	SQLite-backed session log	implements it
`packages/session/session-projection`	derived per-session projection table	projection keys
`packages/session/session-projection-cache`	durable projection checkpoints	`session_projcache` domain
`packages/session/session-stats`	whole-log conversation figures	`sessionStats` key
`packages/session/session-checkpoint-policy`	semantic durability barriers	plugin `session-checkpoint-policy`
`packages/session-query/session-query-sqlite`	SQLite query/cross-ref index	plugin
`packages/util/home-paths`	where all user data lives	`ctx.dshHomePath`

The KV hub

ctx.storage (packages/storage/storage/src/index.ts) is a Service holding a BackendRegistry plus a mounted forms table. A backend is a medium owner exposing optional facets (kv today):

interface StorageBackend {
  readonly kv?: KvFacet
  close(): Promise<void>
}
interface KvFacet {
  open(descriptor: KvUnitDescriptor): Promise<KvUnit>
}

A KvUnit is opened by descriptor ({ name, version, tables, hasGlobal }), validates names against UNIT_NAME_RE = /^[a-z][a-z0-9_]*$/ (safe as both a filename and a SQL identifier), and offers loadAll(), putRecord, deleteRecord, setGlobal. Values are opaque JSON at this layer — no schema, no events. Write ordering is explicitly the caller's job; the unit only guarantees each single call is atomic and durable once it resolves.

The hub mounts data forms under StorageForms keys (declaration-merged). storage-domain merges domain, so ctx.storage.domain resolves it.

The domain layer

packages/storage/storage-domain is the single implementation of typed KV domains (ctx.storageDomain), and is what application code actually touches. Its Config decides routing: backend is the default, routes overrides per domain name — a route naming an unregistered backend fails loud at open.

export interface Config { backend: string; routes?: Record<string, string> }

DomainFacility.open(spec) validates the name is free, routes to a backend, requires its kv facet, opens the unit, then zod-validates every stored record against the spec (invalid-record with table+key on mismatch). A Domain<S> gives typed table handles KvTable<K,V> (get/entries/keys/size/put/delete/update) plus a DomainGlobal. Reads are synchronous from an authoritative in-memory map; every write queues on a single per-domain write chain, awaits backend durability first, then mutates memory, then emits domain/changed ({ domain, table, key, operation: 'put'|'deleted', value? }). A rejected backend write leaves memory untouched.

The spec split: plugin Config is schemastery, record schemas inside specs are zod (src/spec.ts documents the rationale). Example durable specs: message_feedback (packages/feedback/message-feedback/src/spec.ts) and session_projcache (packages/session/session-projection-cache/src/spec.ts).

Backends: JSON vs SQLite

storage-json

packages/storage/storage-json registers as backend json. One human-readable file per unit under a configured root — legibility is its reason to exist. Each write republishes the whole file atomically (src/atomic.ts):

json

{
  "unit": { "name": "workspace", "version": 0 },
  "global": null,
  "tables": { "sessions": { "<id>": { /* record */ } } }
}

root has no default on purpose (src/index.ts): a process.cwd() fallback would scatter unit files wherever the process starts. The shipped shell sets it to dshHomePath('storages').

storage-sqlite

packages/storage/storage-sqlite registers as backend sqlite. One database file hosts every routed unit as document-per-row (key TEXT PRIMARY KEY, value TEXT … STRICT), one physical u_<unit>_<table> table per unit table. Config = { path, journalMode? } where journalMode is 'wal' (default) | 'delete' | 'truncate' | 'persist' — rollback-journal modes exist for filesystems where WAL re-memory files fail. The physical layout is pinned at STORAGE_SQLITE_SCHEMA_VERSION = 1 via PRAGMA user_version; a stale-stamped DB rejects rather than migrating (this pre-release format has no migrations). Fresh files are created owner-only (0o600), and defaulting to :memory: is supported for tests.

When which: JSON when you want an operator- or reader-visible, diffable single file per unit (workspace, projections). SQLite when you want one DB, per-row streaming, or readFrom-by-seq seeking. The domain route table decides; both backends implement one negotiated contract, so a domain can move between media by editing routes without changing its spec.

Session persistence (event-sourced)

The session log is not a KV record — it is an append-only event log. ctx.sessionPersistence (packages/session/session-persistence/src/index.ts) is a Service Definition with the contract:

create(meta) — register a SessionHeader (may defer physical write to first append, so abandoned sessions leave nothing behind);
append(id, events) — a contiguous batch whose first seq must equal the stored next-seq; rejects non-JSON-serializable data;
load(id) — returns a balanced log ending on turn/end, closing an interrupted final turn with synthetic error closers, discarding only a torn final record;
list() / listSnapshots() — cheap header/revision listing without a full parse;
readFrom(id, fromSeq) — the watermark primitive for read models (SQLite seeks; JSONL parses forward);
prepare(id) — rehydrate an unpublished SessionPreparation for resume.

The format is pinned at SESSION_FORMAT_VERSION = 0 (pre-release, no compatibility implied). Every event conforms to the envelope in the persistence catalog:

export type SessionEvent<T extends SessionEventType = SessionEventType> = {
  type: K; seq: number; time: number
  data: SessionEventMap[K]
  ignorable?: true
} & (K extends SurfaceEventType ? { sourceEventSeqs?: number[]; surfaceOp?: SurfaceOp } : {})

Only user/message, assistant/message, tool/result are SurfaceEventType — they can carry surfaceOp ('append' or { op:'replace'; start; end }) and cite their source seqs. A reader meeting an unrecognized type without ignorable must refuse rather than silently drop the event.

JSONL backend

packages/session/session-persistence-jsonl writes one artifact per session. Path layout (src/format.ts):

<root>/<projectKey(cwd) or _no-cwd>/<sessionId>/session(.jsonl | .jsonl.zstd)

The first record is the immutable header tagged { type: 'session', version, id, createdAt, cwd?, parentSession?, seedLength?, origin?, delegationDepth, agentPreset? }; every following line is one JSON-lines event. JsonlCompression = 'zstd' | 'none' selects .jsonl.zstd (packed Zstandard frames via the zstd-*.ts decoders) or plaintext. packChunks folds delta-chunk runs into text-chunks storage rows.

SQLite backend

packages/session/session-persistence-sqlite mirrors the same contract in one SQLite file (packed rows, seekable by seq for readFrom). The cross-session SQLite query/cross-ref index lives in packages/session-query/session-query-sqlite (the "Session Query & Log Export" surface); it shares the open/configure sequence with storage-sqlite.

Projections

ctx-mountable read models are derived from the durable log and folded into a shared SessionProjectionMap (packages/session/session-projection/src/types.ts), a merge-extensible interface … {} inside which domain packages declare their keys by declaration merging. Values are wire-JSON whole values; how a value renders is the UI slot system's business.

session-stats (packages/session/session-stats) contributes the sessionStats key: { turns, steps, llmMs, toolMs, ttftMs, ttftSteps, decodeMs, decodeTokens } — whole-log counts independent of how much history a client has paged in, field names mirroring the client window fold.
session-projection-cache declares the session_projcache domain (default storage-json, so it lands at <root>/session_projcache.json beside workspace.json). Each row is a full projection checkpoint { key → { ver, seq, val } }; a ver mismatch vs the live unit's state version discards the row at read time, and the bound identity (createdAt, cwd, …) fences a row to one session lifecycle, so a reused id or swapped persistence root cannot seed an unrelated log. The shipped shell config: writeEveryEvents: 200, writeIntervalMs: 5000.

Checkpoints

packages/session/session-checkpoint-policy installs semantic durability barriers on model requests, top-level tool dispatch, and completed agent steps. It wraps the llm/stream chain so that a model request is delayed until "the complete logged request prefix is durable" (ctx.sessions.flush(session) before yielding the first chunk); a checkpoint rejection prevents adapter dispatch. It is fail-closed at model and tool side-effect boundaries — the downstream adapter or tool body is only invoked after the barrier.

Where everything lives on disk

packages/util/home-paths defines the single-root Harness home: default ~/.dsh, overridable with DSH_HOME, resolved by resolveDshHome(), exposed to Loader !!js expressions as ctx.dshHomePath(...). Typical layout at this revision:

Path under the home	Content
`$DSH_HOME/storages/`	JSON storage-domain unit files (e.g. `workspace.json`, `session_projcache.json`) via `storage-json root: dshHomePath('storages')`
`$DSH_HOME/attachments/v1/`	content-addressed local attachment objects (see identity-feedback page)
`$DSH_HOME/.anonymous-user-id`	the per-home anonymous user id
`$DSH_HOME/profiles/<name>/`	profile dirs: `package.json`, `cordis.patch.yml`, `pnpm-workspace.yaml`
`$DSH_HOME/profiles/node_modules/`	maintained flat symlink fallback for out-of-tree plugins

The KV hub ​

The domain layer ​

Backends: JSON vs SQLite ​

storage-json ​

storage-sqlite ​

Session persistence (event-sourced) ​

JSONL backend ​

SQLite backend ​

Projections ​

Checkpoints ​

Where everything lives on disk ​

Further reading ​