Skip to main content
Ask your AI

Observe

You can verify a flow at three levels, each one step closer to production:

  1. Simulate: run an event through the built flow offline, with destinations mocked, see bundled mode.
  2. Run locally: run the flow where your app runs and watch each event in your console, see integrated mode.
  3. Observe: attach a running flow to an Observe session and watch its live records arrive in a feed, from any browser or server deployment.

This page covers the third rung: connecting a flow to an observer.

What you commit vs what arrives later​

Your code and config carry only public connect values: the observer url and the project binding. Neither is a secret, both are safe to commit.

The per-session credential arrives out-of-band: as a URL parameter in the browser, or as environment variables on the server. No credential is ever stored in a committed artifact: flow.json, a built bundle and your application code carry none.

Web, integrated mode​

Add the public connect pair to startFlow:

import { startFlow } from '@walkeros/collector';
import { pageObserveSlot } from '@walkeros/web-core';
import { sourceBrowser } from '@walkeros/web-source-browser';

const { elb } = await startFlow({
  sources: {
    browser: { code: sourceBrowser, config: { settings: { pageview: true } } },
  },
  observe: {
    url: 'https://observer.example.com', // public, safe to commit
    binding: 'pb_x', // public project binding, safe to commit
    slot: pageObserveSlot(), // keeps the session credential in this browser
  },
});

slot is required in a browser: a web observe without it logs one error line naming pageObserveSlot and Observe stays off. The observe block under collector in a flow file cannot carry a slot, so use config.observe (bundled mode below) or a hand-written startFlow as shown.

Then open your dev URL with a session credential appended:

http://localhost:3000/?elbObserve=obsw_pb_x.ses_9.sec_1

On load, the runtime checks that the credential's binding matches the one in your config (a credential for another binding is ignored and logged at WARN, which the default ERROR log level hides), persists the credential in localStorage, removes the parameter from the address bar, and starts posting flow records to the observer. Later page loads reuse the stored credential, so one activation lasts until the Observe session ends.

Without a credential, the same page does zero observation work: nothing is parsed, stored, or sent. When the Observe session ends, the observer rejects the next post and the runtime detaches and clears the stored credential on its own.

Web, bundled mode​

The same connect pair goes into flow.json under config.observe:

{
  "version": 4,
  "flows": {
    "default": {
      "config": {
        "platform": "web",
        "observe": {
          "url": "https://observer.example.com",
          "binding": "pb_x"
        },
        "bundle": {
          "packages": {
            "@walkeros/collector": { "version": "^4" },
            "@walkeros/web-source-browser": { "version": "^4" }
          }
        }
      },
      "sources": {
        "browser": {
          "package": "@walkeros/web-source-browser",
          "config": { "settings": { "pageview": true } }
        }
      },
      "collector": { "run": true }
    }
  }
}

Build once:

walkeros bundle flow.json -o ./dist/walker.js

The CLI adds the browser slot to the bundle, so a bundle needs an up-to-date CLI. The bundle carries only the public pair. url and binding must both be set: a partial pair warns at build time and wires nothing. Activation works exactly as in integrated mode, via the ?elbObserve= parameter on any page that loads the bundle.

Server​

Server deployments read their connect config from the environment:

import { startFlow } from '@walkeros/collector';
import { observeFromEnv } from '@walkeros/core';

const { elb } = await startFlow({
  // undefined unless all three variables are set; undefined means zero observation work
  observe: observeFromEnv(process.env),
  // ...your sources and destinations
});

Set the environment on the deployment, never in code or config files:

WALKEROS_OBSERVER_URL=https://observer.example.com
WALKEROS_DEPLOYMENT_ID=ses_1a2b3c
WALKEROS_INGEST_TOKEN=... # secret, inject via deployment env only
WALKEROS_OBSERVE_LEVEL=standard # optional: off | standard | trace

observeFromEnv returns a config only when the full trio (URL, DEPLOYMENT_ID, INGEST_TOKEN) is present; otherwise it returns undefined and the deployment does no observation work. A set but unrecognized WALKEROS_OBSERVE_LEVEL (a typo like offf) also makes it return undefined, so nothing is observed. The ingest token is the one secret in this setup, which is why it lives in the deployment environment and nowhere else.

A bundled server flow needs no code for this: runneros start reads the same variables and observes once all three are set. It handles an unrecognized WALKEROS_OBSERVE_LEVEL differently: it logs Ignoring invalid WALKEROS_OBSERVE_LEVEL and observes at standard.

Connect values vs telemetry knobs​

The observe block carries two kinds of fields, and they answer different questions:

FieldsQuestionWho sets them
url, bindingWhere to attach, and which project may attachYou, committed once
level, sampleHow verbose the records are, and what fraction of events they coverYou, in the web bundle's config; on a server, level in WALKEROS_OBSERVE_LEVEL and sample only in code, on the observe object of startFlow

The web connect pair is url and binding, nothing else. level (off | standard | trace) and sample (a fraction from 0 to 1) decide what gets captured. A web arm captures at the level and sample baked into the bundle it loads, from config.observe or the observe object of startFlow; unset, they are standard and 1. On the server, WALKEROS_OBSERVE_LEVEL is the same knob in environment form; the config.observe block of a server flow has no effect on runneros start. An Observe session in the walkerOS app sets the level of the container arm it starts.

Bring your own observer​

Observation also works without the hosted app: any function can receive the same flow records via observers:

const { elb } = await startFlow({
  observers: [(state) => console.log('flow state', state.stepId, state.phase)],
});

Observers are advisory by contract: they run inside the collector's emit loop, and a throwing observer never breaks event processing. Use this to stream records to your own endpoint, a log pipeline, or a test assertion.

Observability and runtime tracing​

level decides what a runtime emits:

LevelWhat the runtime emits
offNo records. Under runneros start, a trace window can still raise it to trace (see below).
standard (default)Structural FlowState records, without inbound or outbound event payloads.
traceFull payloads on every hop, inbound and outbound.

Activating trace at runtime (no redeploy)​

runneros start can turn trace on and off for a running server flow without a rebuild or redeploy. The mechanism is a poll, not the heartbeat:

  1. When the three observer variables are set and the level is not already trace, the runtime calls GET <WALKEROS_OBSERVER_URL>/trace/<WALKEROS_DEPLOYMENT_ID> with the ingest token, at start and then about every 15 seconds.
  2. The observer answers { "traceUntil": "<ISO timestamp>" | null }.
  3. The runtime feeds the value into resolveTelemetryOptions({ traceUntil }) in @walkeros/core.

The resolver applies the value per emit:

  • A traceUntil that parses to a future ISO timestamp forces level: trace with full inbound and outbound payloads and sample: 1, overriding the baseline level (WALKEROS_OBSERVE_LEVEL).
  • A null or past value reverts to the baseline level.
  • The window self-expires once the timestamp passes, so trace stops on its own with no further action.

Only a 200 with a parseable body changes the active trace state. A network error, a non-200, or an unparseable response leaves the current window untouched, so a transient observer outage never silently disables (or enables) tracing.

Web bundles and a startFlow with observeFromEnv do not poll; they keep their configured level. The walkerOS app sets no trace window, so the poll matters only for an observer you run yourself that serves this endpoint.

Journey correlation across flows​

Every FlowState record carries correlation fields so a single observer can stitch one request across web and server flows. These fields are present at every level, including standard.

FieldMeaning
traceIdW3C 32-hex trace id of the originating run. Resolved as event.source.trace, falling back to an inbound header's trace, then the run-scoped collector trace. Never synthesized per hop.
eventIdW3C span id of the walker event at this hop (event.id).
parentEventIdThe upstream runtime's event.id, present when this flow was entered across a $flow crossing (from an inbound traceparent).
sourceIdThe originating source id.
seqPer-poster monotonic counter, stamped on transport (see Loss-visible transport).

Correlation crosses the wire via the W3C traceparent header:

  • Emit. The web api destination adds traceparent: 00-<traceId>-<eventId>-01 to each non-batched send when the event carries both ids. A traceparent you configure yourself always wins (matched case-insensitively). Batched sends carry no traceparent, since one batch may aggregate distinct upstream traces.
  • Adopt. Any server source with a request header bag (for example an Express or fetch handler) reads a valid inbound traceparent and threads its trace and parent span into the ingest context, so the receiving flow's records share the caller's traceId and link back via parentEventId. An absent or malformed header is ignored.

Journey status​

The records of one event assemble into a single journey, identified by that event's id. A traceId is run-scoped rather than per event, so it is carried as a shared handle that links every journey of one page load or container run. Each journey carries a status derived from the hops observed so far:

StatusMeaning
pendingThe journey is still settling: its last record arrived within the settle window (15 seconds by default), so more may follow.
completeEvery expected hop reached a terminal state (out, error, or skip) or was handed to a batch.
partialThe journey has settled, but an expected hop never produced a terminal record.

"Expected" comes from the flow's step topology when one is known (only steps downstream of a processed hop are expected); without a topology, a journey is complete once every observed hop is terminal. A separate lossy flag is orthogonal to status: a journey is lossy when a seq gap (see Loss-visible transport) overlaps its window, so a journey can be complete and lossy, or partial and lossy, at once.

Vendor-call capture​

At trace level a destination can record the outgoing vendor calls it makes, so an observer sees the exact call that left for the vendor as well as the mapped event. A destination opts in by declaring its observable callables as dot-paths (the gtag destination declares call:window.gtag); the runtime then wraps those callables and records each { fn, args, ts } onto the record's outbound payload, with arguments sanitized to JSON-safe markers.

Capture works whenever the destination reaches its callable through the env. Server and simulate runs carry the callable in the env directly. On live web a destination resolves a real global (for example window.gtag) through @walkeros/web-core's getEnv; the runtime wraps the declared callables at that resolution point, so calls are recorded without the destination or the page changing. The batched delivery path is not captured: batched sends flush against the unwrapped env, so a batching destination shows its per-event input and output but not its vendor calls. Recorded calls are payload-class: they appear only when the outbound payload does (trace, or an explicit includeOut).

Loss-visible transport​

The batched poster stamps each record with a monotonic seq before sending. A dropped or rejected POST is never re-sequenced, so a gap in seq at the observer shows lost telemetry. The poster also splits a batch by real UTF-8 byte size (maxBodyBytes, default 60000) rather than letting an oversized body fail with a 413.

Next steps​

💡 Need implementation support?
elbwalker offers hands-on support: setup review, measurement planning, destination mapping, and live troubleshooting. Book a 2-hour session (€399)