DeepSeek Harness in Depth

The LLM layer is the provider-neutral brain of the harness: a single vocabulary for conversation, model calls, and streaming that every other subsystem — the agent loop, the session log, the metadata projections — speaks, while provider-specific wire formats live only behind a thin adapter boundary. This page dissects packages/llm/*, the ctx.llm service, and how a chat turn actually reaches a model.

Packages and roles

Package	Role
`@deepseek-ai/dsh-llm`	The seam: `LlmRuntime`, `LlmAdapter`, messages, `GenerateOptions`, `StreamChunk`, usage, errors, retry policy
`@deepseek-ai/dsh-llm-deepseek`	DeepSeek chat-completions adapter (direct `fetch` + SSE)
`@deepseek-ai/dsh-llm-pi-ai`	A library-backed voice/chat adapter (pi-ai SDK), not covered here
`@deepseek-ai/dsh-llm-retry`	Provider-routed request retry policy executor
`@deepseek-ai/dsh-token-meter`	Replay-aware token measurement service (`ctx.tokenMeter`)
The seam keeps its runtime dependencies thin and deliberate: `dsh-llm`'s runtime `dependencies` are `@deepseek-ai/dsh-util-crypto`, `@deepseek-ai/schemastery`, `zod`, `@deepseek-ai/dsh-timeout`, `@deepseek-ai/dsh-brand`, `@deepseek-ai/dsh-util-values`, and `@deepseek-ai/dsh-typert-protocol` — Cordis itself is a peer dependency the harness supplies, keeping the adapter boundary reusable outside the harness.

The ChatCompletion vocabulary

There is no class named ChatCompletion. The seam's request/response types are GenerateOptions (request), Message (conversation unit), ContentBlock (streamed/assembled content), and StreamChunk (raw adapter protocol). All live in packages/llm/llm/src/types.ts and packages/llm/llm/src/message.ts.

Messages are immutable, merge-extensible content

A Message is a role-tagged bundle of ContentBlock[]. Blocks are keyed into ContentBlockMap (text, reasoning, image, tool-call, tool-result), and the map is merge-extensible: plugins add new block types. Every message is constructed through createMessage/createUserMessage/createAssistantMessage/createToolResultMessage, which detach (structuredClone) and deep-freeze a fresh identity, so the durable history, the model request, and the client UI all share the one immutable object:

// packages/llm/llm/src/message.ts (condensed)
export interface Message {
  readonly id: MessageId
  readonly role: 'system' | 'user' | 'assistant'
  readonly content: ContentBlock[]
  readonly source: MessageSource   // user | plugin | model | tool
}
export interface TokenUsage {
  inputTokens: number          // UNcached input
  outputTokens: number
  cacheReadTokens?: number     // cached input, disjoint from inputTokens
  cacheWriteTokens?: number
  reasoningTokens?: number
}

Note the disjoint-usage convention: inputTokens never includes cache reads; cached input is reported separately. That rule is what makes the adapter's usage mapping (and the token meter's baseline) well-defined.

The adapter contract: one `stream` method

A provider plugs in by subclassing LlmAdapter (abstract in packages/llm/llm/src/index.ts). Only stream(options: GenerateOptions): AsyncIterable<StreamChunk> is required. Optional hooks describe the provider for configuration surfaces:

Hook	Purpose
`providerInfo(provider)`	Human-readable provider name for selectors
`providerRetryPolicy(provider)`	Provider-owned resolved retry policy
`listModels(provider)`	Advisory model catalog (never request validation)
`resolveModel(provider, model, signal)`	Exact-route metadata: context window, `defaultMaxTokens`, reasoning efforts
`stream(options)`	Required — emit `StreamChunk`s honoring `options.signal`

StreamChunk is the fine-grained protocol: block-start, text-delta, reasoning-delta, tool-call-delta, block-end, usage, and a terminal finish (with reason: stop | tool-calls | max-tokens | aborted | error, the last two carrying the LlmFailure). Tool arguments stay raw JSON strings end to end.

`LlmRuntime` and provider registration

LlmRuntime (ctx.llm) is a Cordis Service owning an adapter registry keyed by provider route, plus a configurable-provider directory and a model-discovery registry.

registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle
registerConfigurableProviders(entries): DirectoryRegistrationHandle
registerModelDiscovery(settingsNs, discover): () => void

Route registration is all-or-nothing: a duplicate route throws llm DUPLICATE_ADAPTER and nothing is committed. The returned handle supports atomic replace() (used by HMR and by live settings changes to re-route without a registration gap). Adapter selection is options.provider, resolved at call time; there is no single "default" adapter the harness dispatches to — a provider route simply must be registered to be callable.

The configurable-provider directory decouples which providers can be configured (LlmConfigurableProvider: provider, displayName, settingsNs, settingsPath, declared?) from which are currently live, so the Models page can offer dormant providers and interrogate their endpoints (discoverModels) without a stored route or network cost when the adapter already knows the models.

The streaming call path

LlmRuntime.stream(options) builds a generator pipeline around the adapter:

Select registration for options.provider (NO_ADAPTER if none) and resolve exact-model metadata (resolveModel), materializing adapter defaults (maxTokens, reasoningEffort) and validating supported efforts.
Route through the llm/stream waterfall (ctx.waterfall(this, 'llm/stream', options, () => adapterStream(...))). Listeners may yield their own chunks to short-circuit (retry/replay/routing), or call next() to reach the adapter.
adapterStream is the fixed adapter boundary: it constructs the iterator and iterates, converting any adapter dispatch/iteration failure into one terminal error/aborted finish chunk. Middleware and consumer failures stay thrown.
Builder consumers — the agent loop — feed StreamChunks into BlockAssembler to reassemble full ContentBlocks and the definitive usage + finish.

The loop (packages/core/agent-loop/src/agent.ts) either uses a PreparedLlmCall (from llm.prepareCall, binding one adapter registration across capability resolution and dispatch) or ctx.llm.stream(request) directly, and records every chunk into the session log.

What precisely is an "event"?

The task of "events (llm/request, llm/response?)" has a concrete answer — and it is not those names. There is no llm/request or llm/response cordis event. Instead:

Channel	Kind	Meaning
`llm/stream`	cordis waterfall	intercept every streaming model call (retry, replay, routing)
`llm/adapters-updated`	cordis emit	adapter registry or configurable-provider directory changed
`agent/request-error`	cordis waterfall	loop observed a failed step; retry policy hooks here
`request/header`	session event	durable request envelope (provider, model, config, system, tools)
`assistant/chunk`	session event	one raw `StreamChunk` (including `usage`)
`assistant/message`	session event	finalized assistant message with `usage` and `sourceEventSeqs`
`llm/retry` / `llm/retry-started`	session event	durable record of scheduled/started retries

So "request/response" materialize as session-log events, which is what makes the session reconstructable and what the token meter and client projections replay. The only live cordis engine is the llm/stream waterfall.

Retry and backoff

packages/llm/llm/src/retry-policy.ts defines the provider-owned policy; packages/llm/llm-retry/src/index.ts executes it. Each adapter exposes one resolved policy per route, captured at registration.

mode: 'normal' → bounded retries for configured retryableCodes (defaults: EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT), maxRetries: 5 (DEFAULT_MAX_RETRIES, packages/llm/llm/src/retry-policy.ts:13).
mode: 'always' → unbounded retry of every failure until success/cancellation/disposal.
Backoff is bounded exponential with symmetric jitter: initialDelayMs 500, maxDelayMs 10_000, jitterRatio 0.1. A provider retry-after (providerRetryAfterMs on a failure) is honored unless it exceeds the policy cap.

The executor listens on agent/request-error (a waterfall). Scheduling is durable before delay: it appends llm/retry to the session log before the cancellable wait, then llm/retry-started when the wait actually fires — so a crash between them is observationally consistent on replay. RetryId chains retries for one provider-policy across attempts.

Model IDs and routing

Model routing uses a two-tuple: { provider, model }. The model string is the wire model id (e.g. deepseek-v4-flash), and the provider string selects the adapter. GenerateOptions.model is passed verbatim to the provider (the harness model name is the wire name for the DeepSeek adapter). An optional reasoningEffort ('off' | 'low' | 'high' | 'max' for DeepSeek) selects adapter-owned effort levels; unsupported requests throw UNSUPPORTED_REASONING_EFFORT before any provider I/O.

Data flow diagram

text

      Agent loop (agent-loop)                    Web client / projections
      ┌──────────────────────────┐               ┌────────────────────────────┐
      │  buildRequest()          │               │  token-meter / projections  │
      │  request/header (log)    │               │  replay session events       │
      │  llm.prepareCall() │     │               └─────────────▲───────────────┘
      └──────────┬───────────────┘                             │ replay
                 │ ctx.llm.stream(req)                          │
                 ▼                                              │
   ┌──────────────────────────────────────┐                     │
   │  LlmRuntime.stream()                  │                    │
   │  ① select registration (NO_ADAPTER)   │   appends           │
   │  ② resolveModel → defaults + efforts  │   assistant/chunk   │
   │  ③ llm/stream  ──waterfall──> adapter │   usage finish      │
   └──────────────┬───────────────────────┘   assistant/message  │
                  ▼                                              │
   ┌──────────────────────────────────────┐                     │
   │  LlmAdapter.stream(options)           │   BlockAssembler     │
   │  (e.g. DeepSeekAdapter: fetch + SSE)  │◄── StreamChunks ─────┘
   └──────────────────────────────────────┘
                  │   llm/retry ↔ agent/request-error (retry policy)
                  ▼
        provider endpoint (api.deepseek.com / other)

Component responsibilities

Concern	Where
Request/response/message vocabulary	`packages/llm/llm/src/types.ts`, `message.ts`
Registry, waterfall, dispatch	`packages/llm/llm/src/index.ts` (`LlmRuntime`)
Chunk reassembly	`packages/llm/llm/src/assembler.ts` (`BlockAssembler`)
Typed failures (`LlmError`, `LlmFailure`)	`packages/llm/llm/src/error.ts`, `adapter-failure.ts`
Retry policy definition + defaults	`packages/llm/llm/src/retry-policy.ts`
Retry policy execution + durable events	`packages/llm/llm-retry/src/index.ts`
Token measurement service	`packages/llm/token-meter/src/index.ts`

Packages and roles ​

The ChatCompletion vocabulary ​

Messages are immutable, merge-extensible content ​

The adapter contract: one stream method ​

LlmRuntime and provider registration ​

The streaming call path ​

What precisely is an "event"? ​

Retry and backoff ​

Model IDs and routing ​

Data flow diagram ​

Component responsibilities ​

Further reading ​