Compaction

Long sessions accumulate tool output, file reads, and reasoning traces until the prompt no longer fits. Rather than crash on a context_length_exceeded response, the engine summarises older history into a single synthetic user message and replays only the recent turns. Compaction is engine-driven; the provider only exposes its contextWindow(model) so the loop knows when to fire.

#Why compaction exists

Every provider request carries the full message history. As a session runs, that history grows monotonically — new user messages, assistant turns, and tool results keep accumulating. Sooner or later the prompt approaches the model’s context window and the next call either wastes budget on stale state or fails outright with a ContextLengthError.

Compaction is the SDK’s answer: when usage gets close to the window, the engine asks a CompactionStrategy to rewrite the provider-facing view of the conversation. The default strategy keeps the last few turns intact and replaces everything older with a short summary produced by the same provider. The session’s durable history is never modified — only the in-memory providerView that gets serialised on each turn. On resume, the full transcript is replayed and (if needed) compacted again from scratch.

#When compaction triggers

There are two entry points. The proactive path runs before every turn; the reactive path runs only when the provider has already rejected the prompt.

#Proactive — token budget check

At the top of every turn (after the user message is appended but before provider.stream is called), the loop calls maybeCompact(session, agent, signal). The check is purely numeric:

• If the session has no lastUsage yet (first turn), nothing happens.
• Otherwise the loop projects the next request’s prompt size as lastUsage.input_tokens + reserve, where reserve is AgentOptions.maxOutputTokens if set, else 32768. The reserve approximates how much of the window the model will need for its own output.
• If the projected total is at or above 80% of provider.contextWindow(model), the configured strategy runs and rewrites providerView in place.
• A CompactionEvent is then yielded to the consumer carrying messages_before, messages_after, tokens_before, a placeholder tokens_after: 0 (no local tokenizer ships with the SDK — the real post-compaction count appears in the next UsageEvent), and the strategy’s id.

If the strategy returns the array unchanged (same length, same head reference), the event is suppressed so consumers don’t see a no-op compaction.

#Reactive — ContextLengthError recovery

The 80% projection is conservative but not infallible: the model could expand its reasoning, a tool could return an unusually large result, or the proactive check could be wrong about maxOutputTokens. When the provider call still fails with a ContextLengthError, the loop catches it inside streamTurnWithContextRetry and runs a one-shot recovery:

1. Set a per-turn flag (compactionRetryUsedThisTurn) so this only fires once.
2. Call runForcedCompaction — the same strategy as before, but unconditional. No threshold check.
3. Emit a CompactionEvent for the consumer.
4. Rebuild the request and retry provider.stream exactly once. The retry runs on the session’s default model — any one-turn model override (from a skill overlay) is considered spent.

After a compaction event, the loop also re-injects the skill listing (if any) and any previously invoked skill bodies into providerView so the model retains skill context after the older history is replaced by the summary. This is in-memory only — the durable store keeps the originals untouched and re-injection happens fresh on each compaction.

#Default strategy

When no compaction option is passed to Agent, the SDK uses defaultCompaction, exposed via strategy.id === "default-keep-recent-10". The heuristic is small and entirely deterministic.

1. Walk the message array backwards counting assistant messages. Stop at the 10th. Everything from that index to the end is the recent slice and stays verbatim.
2. Everything before that index is the older slice. If it is empty (the conversation has fewer than 10 assistant turns), return the input unchanged — compaction becomes a no-op and the event is suppressed.
3. Otherwise call summarizeWithProvider(provider, model, older, signal). This issues a non-streamed provider.stream call with a fixed SUMMARY_PROMPT (capture decisions, file edits with paths, tool calls and outcomes, open questions, current task state), max_output_tokens: 4096, no tools, no cache control, and consumes the stream into a single text blob.
4. Prepend one synthetic user message — { role: "user", content: [{ type: "text", text: "<summary of earlier conversation>\n\n…" }] } — to the recent slice and return the new array.

Three things survive the rewrite untouched. The system prompt isn’t a message — it lives on ProviderRequest.system and is rebuilt from Agent state on every turn, so it’s immune by construction. Tool definitions are similarly request-level and unaffected. And the last 10 assistant turns plus their interleaved user/tool-result messages are passed through verbatim.

#Custom strategies

Pass compaction on the Agent constructor to override the default. A strategy is a plain object with two members and one method.

CompactionContext is the only argument:

interface CompactionContext {
  messages: Message[];
  provider: BaseProvider;
  model: ModelId;
  signal: AbortSignal;
}

interface CompactionStrategy {
  readonly id: string;
  compact(ctx: CompactionContext): Promise<Message[]>;
}

Two things to note about the interface. First, there is no shouldCompact predicate — the 80% threshold is hard-coded in maybeCompact and the only way to change the trigger point is to make compact itself a no-op below your preferred ratio (return the input unchanged; the event will be suppressed). Second, the strategy does not get a token count directly — only the message array. If you need the budget, call ctx.provider.contextWindow(ctx.model) and reason about message length.

A minimal custom strategy that keeps fewer recent turns than the default looks like this — both lastNTurnBoundaries and summarizeWithProvider are exported so you can reuse the building blocks the default strategy uses.

import { Agent } from "@skawld/agent-sdk";
import type {
  CompactionStrategy,
  CompactionContext,
  Message,
} from "@skawld/agent-sdk";
import { lastNTurnBoundaries, summarizeWithProvider } from "@skawld/agent-sdk";

// Keep the last 4 assistant boundaries instead of the default 10.
const tighter: CompactionStrategy = {
  id: "keep-recent-4",
  async compact(ctx: CompactionContext): Promise<Message[]> {
    const recent = lastNTurnBoundaries(ctx.messages, 4);
    if (recent === ctx.messages) return ctx.messages; // nothing to drop
    const older = ctx.messages.slice(0, ctx.messages.length - recent.length);
    const summary = await summarizeWithProvider(ctx.provider, ctx.model, older, ctx.signal);
    return [
      { role: "user", content: [{ type: "text", text: `Prior conversation summary:\n${summary}` }] },
      ...recent,
    ];
  },
};

const agent = new Agent({ provider, model, compaction: tighter });

#Worked examples

#Never compact (short-lived agents)

For ephemeral agents that only run a handful of turns, summarising the prefix wastes tokens and adds latency. Return the input unchanged and the loop suppresses the event.

import { Agent } from "@skawld/agent-sdk";
import type { CompactionStrategy } from "@skawld/agent-sdk";

const neverCompact: CompactionStrategy = {
  id: "never",
  async compact({ messages }) {
    return messages;
  },
};

const agent = new Agent({ provider, model, compaction: neverCompact });

#Summarise via a cheaper model

The default summariser uses the same provider/model as the main loop, which is expensive when you’re paying Opus rates to summarise old tool output. Build a strategy that routes the summarisation call to a second provider instance pointed at a cheap model, while still keeping the recent turns intact via lastNTurnBoundaries.

import {
  lastNTurnBoundaries,
  summarizeWithProvider,
} from "@skawld/agent-sdk";
import type {
  CompactionStrategy,
  Message,
  BaseProvider,
  ModelId,
} from "@skawld/agent-sdk";

export function cheapSummary(
  summaryProvider: BaseProvider,
  summaryModel: ModelId,
): CompactionStrategy {
  return {
    id: "cheap-summary-keep-8",
    async compact({ messages, signal }) {
      const recent = lastNTurnBoundaries(messages, 8);
      const older = messages.slice(0, messages.length - recent.length);
      if (older.length === 0) return messages;

      const text = await summarizeWithProvider(
        summaryProvider,
        summaryModel,
        older,
        signal,
      );

      const summary: Message = {
        role: "user",
        content: [
          { type: "text", text: `<summary of earlier conversation>\n\n${text}` },
        ],
      };
      return [summary, ...recent];
    },
  };
}

lastNTurnBoundaries and summarizeWithProvider are both exported from the package, so a custom strategy can reuse the default building blocks and just swap the pieces that matter. Threading signal through keeps cancellation working — a long summarisation call cancels cleanly when the parent run aborts.

#Failure modes

Compaction is best-effort: it does not iterate, and it does not check the resulting size. A handful of edge cases are worth knowing.

• Strategy returns unchanged messages. If compact returns an array with the same length and the same head reference as the input, the engine treats it as a no-op: the event is suppressed and the next provider call sees the same prompt. Proactive compaction will keep attempting on each turn as usage stays above 80%.
• Recent slice alone exceeds the window. When the last 10 turns plus the new user message and the system prompt already overflow, compacting the older slice doesn’t help. The provider returns context_length_exceeded; the loop catches it once via runForcedCompaction (which re-runs the same strategy and produces the same output); the retry fails with the same error; the run throws ContextLengthError out of session.run.
• Summariser itself throws. If summarizeWithProvider (or your custom equivalent) throws — network error, auth error, model returning no text — the error propagates out of compact, out of maybeCompact, and out of session.run. Compaction has no internal retry; the surrounding turn fails.
• Tokenizer drift. The 80% check uses the provider’s self-reported input_tokens from the previous turn plus an output reserve. If the provider underreports (or the reserve is too small relative to actual output), the proactive check can miss and the reactive path takes over.

All of these surface as ContextLengthError or a wrapped provider error. See Errors for the typed catch shape and retryable semantics.

#Provider context windows

Compaction’s threshold is a percentage of provider.contextWindow(model) — every BaseProvider subclass returns a number for the model passed in. The built-ins ship with small lookup tables, and anything unknown falls back to a provider default.

For non-OpenAI models served on OpenAI-compatible endpoints (Ollama, Groq, vLLM, DeepSeek, OpenRouter), the 128k fallback will be wrong for most models. Pass contextWindowOverride on OpenAIChatCompletionsProvider to tell compaction the real window — see Providers for the option table and per-provider knobs.

Custom providers implement contextWindow(model): number directly. The value is read on every turn, so it can be dynamic if the backend reports different limits per tenant or per key.