> Part of the walkerOS documentation. Project overview and full index: <https://www.walkeros.io/llms.txt>

# Understanding walkerOS Destinations

## Overview[​](#overview "Direct link to Overview")

Destinations receive processed events from the collector and deliver them to third-party tools (analytics, marketing, data warehouses).

**Core principle:** Destinations transform and deliver. They don't capture or process—that's sources and collector.

## Destination Interface[​](#destination-interface "Direct link to Destination Interface")

See [packages/core/src/types/destination.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/destination.ts) for canonical interface.

| Method                      | Purpose                    | Required     |
| --------------------------- | -------------------------- | ------------ |
| `init(context)`             | Load scripts, authenticate | Optional     |
| `push(event, context)`      | Transform and send event   | **Required** |
| `pushBatch(batch, context)` | Batch processing           | Optional     |
| `destroy(context)`          | Cleanup on shutdown        | Optional     |
| `config`                    | Settings, mapping, consent | **Required** |

`destroy?: DestroyFn` — Optional cleanup method. Called during `command('shutdown')`. Use to close DB connections, flush buffers, or release SDK clients. Receives `{ id, config, env, logger }`.

## The env Pattern[​](#the-env-pattern "Direct link to The env Pattern")

Destinations use dependency injection via `env` for external APIs. This enables testing without mocking.

```
// Destination defines its env type

export interface Env extends DestinationWeb.Env {

  window: {

    gtag: Gtag.Gtag;

    dataLayer: unknown[];

  };

}



// Destination uses env, not globals

async function push(event, context) {

  const { env } = context;

  env.window.gtag('event', mappedName, mappedData);

}
```

### Testing with env[​](#testing-with-env "Direct link to Testing with env")

**REQUIRED SKILL:** See [testing-strategy](/preview/pr-749/skills/walkeros-testing-strategy.md) for full testing patterns.

```
import { mockEnv } from '@walkeros/core';

import { examples } from '../dev';



const calls: Array<{ path: string[]; args: unknown[] }> = [];

const testEnv = mockEnv(examples.env.push, (path, args) => {

  calls.push({ path, args });

});



await destination.push(event, { ...context, env: testEnv });



expect(calls).toContainEqual({

  path: ['window', 'gtag'],

  args: ['event', 'purchase', expect.any(Object)],

});
```

### The env as observability seam[​](#the-env-as-observability-seam "Direct link to The env as observability seam")

The env pattern is also the interception seam for trace-level vendor-call capture. At `trace` level the collector may attach an `observe` recorder under the `observe` key of the env, and `@walkeros/web-core`'s `getEnv` wraps the callables the destination declared as observable, recording each call. A destination that reaches its vendor through `getEnv(env)` is automatically observable; one that reaches a global directly, bypassing `env`, is not. This is one more reason to route every external call through `env`.

Rules that keep the seam intact:

* **`init` keeps an injected env.** A client built in `init` (SDK client, auth client, writer) is taken from `env` when present and created only otherwise (`env?.BigQuery ?? BigQuery`). Every key the mock env declares must be read as `env.<key>` in `src`.
* **`simulation` names the request call** (`sendServer`, `call:PubSub.topic.publishMessage`, `call:JSONWriter.appendRows`), not a constructor. The one recorder is `observeEnv` in `@walkeros/core`; it navigates constructors, factories and Promises, and records only the leaf.
* **Trace-mode limit.** CLI and MCP simulate inject the mock env before `init`, so a client built in `init` is recorded. Runtime trace mode wraps the env per push only, so calls through a client built in `init` are not recorded there.
* **Client IP and user agent.** Server conversion-API destinations (Meta, TikTok, Snapchat, Pinterest, Bing, Reddit, Criteo, X, Data Manager, Piwik PRO) expose `ip` and `userAgent` settings (`Mapping.Value | false`), defaulting to `['ingest.ip', 'event.user.ip']` and `['ingest.userAgent', 'event.user.userAgent']`, resolved with `getMappingValue(createMappingRoot(ingest, event), setting, { collector })`. `false` disables, an explicit mapped vendor field wins, each package carries its own few lines (no shared helper).

## Destination Config[​](#destination-config "Direct link to Destination Config")

```
config: {

  settings: { /* destination-specific */ },

  mapping: { /* event transformation rules */ },

  data: { /* global data mapping */ },

  consent: { /* required consent states */ },

  policy: { /* processing rules */ },

  queue: boolean,  // queue events

  dryRun: boolean, // test mode

  queueMax: number,  // consent-queued events cap (default 1000)

  dlqMax: number,    // dead-letter queue cap (default 100)

}
```

## Buffer bounds[​](#buffer-bounds "Direct link to Buffer bounds")

Each destination keeps two internal buffers: `queuePush` (consent-denied events) and `dlq` (failed pushes). Both are size-bounded with FIFO drop-oldest eviction. Defaults: `queueMax: 1000`, `dlqMax: 100`. Set either on a destination's config to override per destination. Operators read drop counts from `collector.status.dropped[stepId('destination', id)]?.queue` (consent-denied evictions) and `?.dlq` (DLQ evictions). Build the key with `stepId()` from `@walkeros/core`. Point-in-time sizes stay on `collector.status.destinations[id].queuePushSize` / `dlqSize`.

`queuePush` drains back into the destination once consent allows delivery. The DLQ is never replayed: an event that reaches it is gone once it is evicted, which is what makes the delivery rules below worth getting right.

## Delivery timeout[​](#delivery-timeout "Direct link to Delivery timeout")

`config.timeout` bounds one delivery. A `push` or `pushBatch` that has not settled within it is abandoned, its events are routed to the DLQ and `failed` is incremented. The default is 10000 ms, applied whenever `timeout` is `0` or unset, so one slow destination never wedges the collector push.

**A destination's own client timeout has to fit inside it, retry delays included.** Many SDK defaults are longer (30 s in several HTTP clients), and a longer one produces the worst outcome available: the collector stops waiting at 10 s, dead-letters the batch and counts it lost, then the request lands anyway a few seconds later. Those events are both counted lost and actually written. Derive the client's per-attempt timeout from `config.timeout` minus what the retry delays can take, split across the attempts, so the last attempt finishes before the race does.

The same bound applies at shutdown, where the collector races each pending batch flush against a fixed 5000 ms step timeout before destroying the destination. A destination whose retries need longer than that loses them on shutdown.

## Batch scheduling[​](#batch-scheduling "Direct link to Batch scheduling")

Set `config.batch` on a destination (with `pushBatch` implemented) to batch **all** of its events into one shared buffer. No `'* *'` wildcard mapping rule is needed. The configuration shape is `batch?: number | { wait?, size?, age? }` at both the destination-config layer and the mapping-rule layer.

**`pushBatch` is the gate, and a missing one fails silently.** The collector engages batching only when `destination.pushBatch` exists. Set `config.batch` on a destination that lacks it and every event is sent one at a time, with no warning and no error: the config is accepted and inert. When adding batching to a destination, implementing `pushBatch` is the change; `config.batch` alone buys nothing.

* `wait` (ms): debounce window. The timer resets on every push. Legacy form `batch: 1000` is shorthand for `{ wait: 1000 }`.
* `size`: hard count cap. Default `1000`. Flushes immediately when reached.
* `age` (ms): hard age cap since the first entry of the current window. Default `30000`. Prevents debounce starvation under sustained load.

A mapping rule's own `batch` splits that entity-action into its own buffer and overrides `config.batch` per field (`rule ?? config ?? default`). To batch only specific events, omit `config.batch` and set `batch` on those rules. Pending batches flush on shutdown.

### Per-event metadata[​](#per-event-metadata "Direct link to Per-event metadata")

`Batch.entries[]` carries per-event `{ event, ingest?, respond?, rule?, data? }`. Destinations that need per-event request IDs or HTTP responses (BigQuery, mParticle, HubSpot) should read `entries` instead of assuming all events in a batch share one `ingest`. `batch.events` and `batch.data` are derived views kept for backward compatibility.

### Failure handling[​](#failure-handling "Direct link to Failure handling")

`pushBatch` has three return outcomes:

* **Resolves `void`** - the whole batch succeeded; every entry counts as delivered.
* **Throws or rejects** - the whole batch failed. Every entry is routed to the destination's `dlq` and `status.destinations[id].failed` is incremented by the batch size.
* **Resolves a `BatchOutcome`** - partial failure. `{ failed: [{ index, error? }] }` names the entries that did not make it, by index into `batch.entries`. Only those are DLQ'd; the rest count as delivered. Use this when the vendor reports per-row results (BigQuery's `rowErrors` is the reference case), so a later DLQ retry does not rewrite rows that already landed.

**A `BatchOutcome` is for sinks that report per row, and for nothing else.** A sink that succeeds or fails as a whole (one SQL `INSERT`, one file write, one publish) must never report an all-failed outcome in place of throwing. Only a throw records a transport failure on the breaker, and only a flush with at least one delivered entry records a success, so an outcome where nothing succeeded records neither: a total outage passes the breaker unseen and it never opens. For a whole-batch sink there are two answers and no third one, resolve `void` or throw.

Batching, the delivery timeout, the DLQ and the breaker are the collector's; per-call retry is the vendor SDK's (BigQuery, Kafka, HubSpot and the AWS SDK each have their own backoff semantics). Where a destination relies on that SDK retry, it keeps the attempts and their delays inside `config.timeout`, and makes a repeat safe through an idempotency key or a deduplication token computed once and reused across attempts. Without that, a retry of a delivery the vendor had already accepted duplicates data. Counters (`count`, `out`) are bumped only after a successful flush.

**Three rules every destination follows.** First, no retry loops, backoff or DLQ code of its own: each request is sent once through the vendor SDK, whose built-in retry stays at its defaults, because a standard retry belongs in the collector, not in each package. Second, `init` is offline: it validates the config and builds clients, but makes no network call, loads no credentials and creates no remote resources; authentication and lookups happen on the first delivery, and provisioning belongs to `setup`. Third, every failure, thrown or reported in a `BatchOutcome`, is an `Error` that names the cause (and, for a configuration error, the fix) and carries `code` (the vendor error name or a package code), `status` (the HTTP status when the vendor answered) and `retryable`. Take `retryable` from the vendor SDK's own public classifier where one exists, else from documented codes or statuses, and keep the SDK error reachable as `cause`. `@walkeros/server-destination-aws` is the reference implementation.

Operators also see `status.destinations[id].inFlightBatch`: the number of events buffered but not yet delivered.

### Out-of-band errors from an EventEmitter SDK[​](#out-of-band-errors-from-an-eventemitter-sdk "Direct link to Out-of-band errors from an EventEmitter SDK")

`tryCatchAsync` only catches errors on the awaited `push`/`pushBatch` path. If a destination owns an SDK object that is an EventEmitter (a BigQuery `StreamConnection`, a Redis client, a Kafka producer), that object can emit `'error'` on a detached tick with no awaiter. An EventEmitter that emits `'error'` with zero listeners throws synchronously and crashes the process.

A destination that owns such an object MUST attach an `'error'` listener and route the error through `context.reportError`, which is available on every step context:

* `context.reportError(err)` (no event): a connection-level error between pushes. Logs (redacted) and bumps `status.connectionErrors[stepId]`. Does not count as a failed event.
* `context.reportError(err, event)`: a specific event was lost. Routes it to the DLQ and bumps `failed`, exactly like an in-band push failure.

`reportError` is guarded and must never be called in a way that throws back into the emitter tick. The Pub/Sub pull source (`sources/gcp/src/pubsub/pull`) is the reference for attaching an `'error'` listener. The runner's process guards are a backstop, but the listener is what gives clean DLQ routing, attribution, and redaction. Optionally pair this with the `breaker` config so a persistently broken transport stops retrying every event.

## Require vs Consent[​](#require-vs-consent "Direct link to Require vs Consent")

Two separate mechanisms control when destinations receive events:

| Mechanism | Purpose              | Scope             | Effect                                                                                 |
| --------- | -------------------- | ----------------- | -------------------------------------------------------------------------------------- |
| `require` | Delay initialization | Whole destination | Destination stays in `pending` until all required events fire (e.g., `walker consent`) |
| `consent` | Filter events        | Per-event         | Events without matching consent are silently skipped or queued                         |

**Require** gates the destination *lifecycle*. A destination with `require: ["consent"]` does not exist in the collector until consent state is present: a `"walker consent"` event, or starting consent (`startFlow`'s `consent`, which `walkeros push --simulate` sets with `--consent`). Until then, events are queued internally.

**Consent** gates *individual event delivery*. A destination with `consent: { marketing: true }` only receives events where the collector's consent state (or the event's own `consent` field) includes `{ marketing: true }`.

**State refresh on flush:** When queued events are flushed to a destination, they receive the *current* collector state (`consent`, `user`, `globals`) — not the stale state from when they were originally captured. Any state-mutation command (`walker consent`, `walker user`, `walker globals`, etc.) triggers a flush attempt. The consent gate still applies: events without required consent simply return to the queue.

Both can be combined:

```
{

  "config": {

    "require": ["consent"],

    "consent": { "marketing": true }

  }

}
```

This means: don't initialize until consent fires, then only accept events with marketing consent. The state that satisfies `require` reaches `on()` before the destination's queued events are pushed.

**Simulation impact:** `require` causes "destination not found" errors in `flow_simulate` because the destination stays pending. Remove `require` temporarily for simulation testing.

## Policy[​](#policy "Direct link to Policy")

Policy modifies the event BEFORE mapping rules run. Defined at config level (applies to all events) or rule level (applies to specific events):

```
{

  "config": {

    "policy": {

      "user_data.email": {

        "key": "user.email",

        "consent": { "marketing": true }

      }

    }

  }

}
```

Policy supports consent-gated field injection — fields are only added when the required consent is present in the event.

## Destination Paths[​](#destination-paths "Direct link to Destination Paths")

| Type   | Path                            | Examples                             |
| ------ | ------------------------------- | ------------------------------------ |
| Web    | `packages/web/destinations/`    | gtag, meta, api, piwikpro, plausible |
| Server | `packages/server/destinations/` | aws, gcp, meta                       |

## Template Destination[​](#template-destination "Direct link to Template Destination")

Use as starting point: `packages/web/destinations/plausible/`

## Transformer Wiring[​](#transformer-wiring "Direct link to Transformer Wiring")

Destinations can wire to their own transformer chain via the `before` property. It runs for this destination only, after the collector chain (`collector.next`, shared by every destination); a `stop` in it skips only this destination:

```
destinations: {

  gtag: {

    code: destinationGtag,

    before: 'redact'  // First transformer to run before this destination

  }

}
```

The transformer chain runs after collector enrichment, before the destination receives events. Each destination can have its own chain. See [understanding-transformers](/preview/pr-749/skills/walkeros-understanding-transformers.md) for chain details.

## Consent: Two-Layer Pattern[​](#consent-two-layer-pattern "Direct link to Consent: Two-Layer Pattern")

Destinations that integrate vendor SDKs typically need two consent layers:

**Layer 1: `config.consent`** — gates walkerOS event delivery. If consent is not granted, events don't reach the destination. This is the primary barrier.

**Layer 2: `on('consent')`** — controls vendor SDK internals. Even when walkerOS stops sending events, the vendor SDK may still run its own behaviors (DOM capture, polling, fetching configs). Use `on('consent')` to pause/resume these.

```
on(type, context) {

  if (type !== 'consent') return;

  const consent = context.data;

  // Derive from config.consent keys — don't hardcode consent names

  const granted = Object.keys(config.consent || {}).every(k => consent[k]);

  vendorSdk.setOptOut(!granted);

}
```

Both layers are needed for complete consent compliance. `config.consent` prevents data flow. `on('consent')` prevents vendor SDK side effects.

`on('consent')` runs before the first push on every activation path (`require`, `walker destination` at runtime, state passed to `walker run`, state set before `run`) and is delivered exactly once per state change. It may be async: the collector awaits it, bounded by `config.timeout`. A handler that does not settle in that time, or rejects, holds that destination: its events stay queued until a later delivery of the state settles, and are discarded at the next `walker run`. Lifecycle handlers such as `on('run')` are awaited too, bounded by the same timeout, and never hold events. An async state handler must settle: until it settles or reaches `config.timeout`, that destination receives no events. `elb` commands and `startFlow` resolve only after the destinations' `on()` handlers for that command have settled or timed out.

## Response Delegation (env.respond)[​](#response-delegation-envrespond "Direct link to Response Delegation (env.respond)")

Destinations can customize HTTP responses by calling `context.env.respond?.({ body, status?, headers? })`. This is useful for destinations that need to signal success/failure back to the HTTP caller. First call wins (idempotent). The respond function is optional — only present when the source provides one.

## Setup (optional)[​](#setup-optional "Direct link to Setup (optional)")

Destinations can implement an optional `setup()` lifecycle to provision external resources, for example a BigQuery dataset and table, a Pub/Sub topic, or a warehouse schema. Setup is **never** invoked by the runtime, push, init, or deploy. It runs only when an operator explicitly types `walkeros setup destination.<name>`.

The signature is `(ctx: LifecycleContext<Config<T>, Env<T>>) => Promise<unknown>`, where `LifecycleContext` carries `{ id, config, env, logger }`. Idempotency is the package's responsibility: the framework adds no opinion. Use `resolveSetup(ctx.config.setup, DEFAULTS)` from `@walkeros/core` to normalize the `boolean | object` shape into a concrete options object.

See [walkeros-create-destination](/preview/pr-749/skills/walkeros-create-destination.md), [walkeros-understanding-sources](/preview/pr-749/skills/walkeros-understanding-sources.md), [walkeros-understanding-stores](/preview/pr-749/skills/walkeros-understanding-stores.md), and the `walkeros setup` CLI documentation for the authoring template and operator workflow.

## Related Skills[​](#related-skills "Direct link to Related Skills")

* [walkeros-understanding-flow](/preview/pr-749/skills/walkeros-understanding-flow.md) - How destinations fit in architecture
* [walkeros-understanding-mapping](/preview/pr-749/skills/walkeros-understanding-mapping.md) - Configure transformations
* [walkeros-create-destination](/preview/pr-749/skills/walkeros-create-destination.md) - Create new destination

**Source Files:**

* [packages/core/src/types/destination.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/destination.ts) - Interface

**Package READMEs:**

* [packages/web/destinations/gtag/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/destinations/gtag/README.md) - gtag example
* [packages/web/destinations/plausible/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/destinations/plausible/README.md) - Plausible (template)

**Documentation:**

* [Website: Destinations](/preview/pr-749/docs/destinations.md) - Overview
* [Website: Create Your Own](/preview/pr-749/docs/destinations/create-your-own.md) - Guide