Skip to main content
Ask your AI

Logger

walkerOS has one logger for the collector, sources, and destinations. It has four log levels and scoped loggers.

Configuration​

Configure logging when initializing the collector:

import { startFlow } from '@walkeros/collector';

const { collector, elb } = await startFlow({
  logger: {
    level: 'DEBUG', // 'ERROR' | 'WARN' | 'INFO' | 'DEBUG'
  },
});

Log levels​

LevelValueDescription
ERROR0Only errors (default)
WARN1Errors and warnings
INFO2Errors, warnings, and informational
DEBUG3All messages including debug information

Choosing a level on public endpoints​

A collector endpoint is exposed to the open internet, so log level decides whether ambient noise becomes an alert. The levels follow one contract:

  • ERROR: the intent is not met. An event was not delivered, a setting has no effect, a producer was told something false, an invalid event was rejected.
  • WARN: walkerOS acted and the intent is still met.
  • INFO: lifecycle and self-healing.
  • DEBUG: the trace of each event and deliberate decisions. A skipped event writes a DEBUG line. A consent wait writes no line: the push result lists the destination under queued with the reason consent. Neither is a failure.
SituationLevel
Ambient internet noise: unparseable bodies, oversized payloadsdebug, plus the source's rejected counter
Ambient internet noise: unknown pathsanswered with 404, no line
An event the pipeline rejects as invaliderror with the reason, one line per source and minute, counted as rejected
A settled ok: false arriving after an HTTP ack was already sentdebug; the collector's failure line reports the cause at error
Handler faults and rejected push promiseserror

The collector already logs thrown pipeline failures at error, so sources do not duplicate that. Apart from the invalid-event line, nothing a scanner can trigger should reach error.

In the complete example​

The server collector of flow-complete.json, the tested example flow, takes its log level from a flow variable:

/flows/server/collector/logger
{
  "level": "$var.logLevel"
}
The log level comes from the flow variable, so LOG_LEVEL changes it per environment. Guide chapter operate

Using the logger​

The logger is automatically available in destination and source contexts:

In destinations​

import { StepError } from '@walkeros/core';

const myDestination = {
  type: 'my-destination',
  config: {},

  async push(event, { logger }) {
    logger.debug('Processing event', { name: event.name });

    // Throw instead of logging a failure: the collector turns the throw into
    // a failed push and writes the one ERROR line.
    const response = await sendToService(event);
    if (!response.ok)
      throw new StepError(`HTTP ${response.status}`, {
        code: 'transport',
        status: response.status,
      });
  },
};

A destination does not log its own failures. It throws, and the collector writes one line per cause (see Failure lines). Use the logger for debug traces and for notes that are not failures.

In sources​

const mySource = async ({ config, env }) => {
  const { logger } = env;

  logger.info('Source initialized');

  return {
    type: 'my-source',
    config,
    push: async (event) => {
      logger.debug('Received event', { name: event.name });
      await env.push(event);
    },
  };
};

Logger API​

Methods​

MethodDescription
error(message, context?)Log an error message
warn(message, context?)Log a warning (degraded state, config issues)
info(message, context?)Log an informational message
debug(message, context?)Log a debug message
throw(message, context?)Throw an exception (returns never); it writes no log line
json(data)Output structured JSON data
scope(name)Create a scoped logger with a prefixed scope

Error handling​

The logger accepts Error objects directly and automatically extracts relevant information:

try {
  await riskyOperation();
} catch (error) {
  // message, name and stack are extracted automatically, and of the cause
  // only its name and code
  logger.error(error);

  // Or with additional context
  logger.error('Operation failed', { error, operation: 'sync' });
}

Throwing errors​

logger.throw() throws an exception in a single call and returns never. It does not log: the collector reports the failed step once, so a throw never produces a second line.

function validateConfig(config) {
  if (!config.apiKey) {
    // Throws - returns 'never' type
    logger.throw('API key is required');
  }

  // TypeScript knows apiKey exists here due to 'never' return type
  return config.apiKey;
}

In a destination, a transformer or a store, prefer throwing a StepError from @walkeros/core with a code, so the failure carries its cause (see Create your own destination).

Failure lines​

The collector writes every failure of a step, once per cause. A destination, transformer, store or source never writes a failure line of its own. A line is keyed by the step, the cause code and the HTTP status, and the collector writes one line per key and minute. The first line names the counter that the repeats are counted in, for example status.destinations.ga4.failed. Later occurrences are counted, and the next line of that key, or the one written at server shutdown, carries suppressed: <n>. If more than 100 different causes are active, the extra ones are logged on every occurrence rather than dropped.

A line carries the step id, the step type, the cause code, the HTTP status, retryable, the event name, the mapping key, a count and the error message capped at 300 characters. It never carries event data (data, user, consent, globals, context, custom), ingest values, vendor response bodies, setting values or an error's cause.

The handler of the logger configuration receives every line, including these, with the structured fields (step, code, status, retryable, suppressed) in its context, so forwarding failures to a monitoring service needs no parsing. Handlers add up: a flow's own handler and the runtime's both see each line.

Failure codes​

CodeCauseRetryableFix
transportA destination's request failed or threw an unknown erroryes, except a 4xx other than 429Check the endpoint and credentials; the status is on the line
runtimeA destination or a store threw a TypeError, RangeError or ReferenceError with no code on its cause; a transformer's throw is transformer, a rule condition's mappingnoFix the code the message names; a retry fails the same way. A browser fetch rejection that a destination throws unwrapped lands here too: throw it as transport (see Reporting failures)
timeoutA destination or transformer push or init, or a destination's rule condition, did not settle within its config.timeout (10 s by default)yesRaise config.timeout or check the vendor
not-readyThe destination or transformer could not startnoFix the cause its message names
breaker-openconfig.breaker is set and threshold retryable failures in a row opened it; events fail at onceyesFix the outage. After breaker.cooldown (30 s) the next event probes the destination and a success closes the breaker; events failed while it was open stay in the dead-letter queue
blockedA state delivery to the destination (its on handler, for example consent) failed and did not recover within config.timeoutyesFix the destination's on handler (see its consent-not-applied line). A handler that failed is retried by a push after the window, one that timed out gets a new delivery once it settles, and one that never settles keeps the destination blocked; events fail until a delivery succeeds
queue-fullA queue refused the event: the collector has not run yet, or a held destination's queue is fullyesStart the collector sooner or raise queueMax
mappingA mapping rule condition threw, or a mapped value could not be builtnoFix the condition or the value
transformerA transformer threwnoFix the transformer
missing-inputThe event has nothing this step can send (a required field is empty)noMap the missing value or skip the event with ignore: true
configA required setting is missing or invalidnoSet it; walkeros validate finds most of these
vendor-not-loadedThe vendor script did not load or was blockednoAllow the script or load it yourself
vendor-not-readyThe vendor SDK is not ready yetyesUsually resolves on the next event
vendor-timeoutThe vendor script did not become ready in timeyesCheck the network and the script URL
beacon-rejectedThe browser refused a sendBeacononly up to 64 KiBSend smaller batches
consent-not-appliedA destination's on('consent') handler failedn/aFix the handler; the destination's events wait for one window of config.timeout, then fail with blocked
rule-not-appliedA destination's before chain renamed an event onto another rule; that rule and its consent do not applyn/aRename the event in a source chain or collector.next
dlq-overflowA destination's dead-letter queue was full and evicted its oldest entryn/aRaise dlqMax or fix the failing destination; counted in status.dropped
on-overflowA destination's queue of waiting on() deliveries was full and dropped the oldestn/aRaise the destination's own config.queueMax; counted in status.dropped
decodeA store holds a value that cannot be decodednoRepair or delete the stored value; the read counts as a miss for that event
invalidAn event was rejected as invalid input: a missing or malformed name, or an event that cannot be serialized to JSON (an ERROR)noFix the producer

A step's own StepError.code wins over these. A code outside this table with neither a status nor a retryable option is not retryable. Unmet consent is not in this table because it is never a failure.

Log format​

By default lines print as text. On a server flow run by runneros, set format to json to print one JSON object per line, with severity, message, time, scope and context. Errors and warnings go to stderr, info and debug to stdout, which platforms such as Cloud Run read as structured logs. The runner's first lines, printed before it loads the flow (up to Health server listening), stay text.

"collector": {
  "logger": { "level": "INFO", "format": "json" }
}

Under runneros, a flow's own logger level (DEBUG included) and format apply to the collector's lines without --verbose; the runner's own DEBUG lines still need --verbose. A custom handler in the flow receives an originalHandler that does nothing under runneros, because the host prints every line once, masked and in the flow's format. A web flow has no output format, so walkeros validate rejects logger.format there (WEB_LOGGER_FORMAT). Secrets are masked in both formats, and a JSON line is capped at 32 KiB: a longer line keeps at most half the cap of its message (messageTruncated: true) and cuts its context to fit (contextTruncated: true).

Scoped loggers​

Scoped loggers prefix each message with its scope, so a line shows where it came from:

// In a destination, the logger is already scoped to its type: [google-gtag]
push(event, { logger }) {
  logger.info('Processing'); // Output: INFO [google-gtag] Processing

  // Create nested scope
  const ga4Logger = logger.scope('ga4');
  ga4Logger.debug('Sending to GA4'); // Output: DEBUG [google-gtag:ga4] Sending to GA4
}

Custom handler​

To send logs to an external service, configure a custom handler:

import { startFlow } from '@walkeros/collector';

const { collector, elb } = await startFlow({
  logger: {
    level: 'INFO',
    handler: (level, message, context, scope, originalHandler) => {
      // Send to external logging service
      loggingService.log({
        level,
        message,
        context,
        scope: scope.join(':'),
        timestamp: Date.now(),
      });

      // Optionally call the original console handler
      originalHandler(level, message, context, scope);
    },
  },
});

The handler receives:

ParameterTypeDescription
levelLevelLog level enum (ERROR=0, WARN=1, INFO=2, DEBUG=3)
messagestringThe log message
contextLogContextAdditional context object
scopestring[]Array of scope names
originalHandlerDefaultHandlerThe default console handler

Testing​

Use the mock logger utility for testing destinations and sources:

import { createMockLogger } from '@walkeros/core';

describe('MyDestination', () => {
  it('logs events correctly', async () => {
    const mockLogger = createMockLogger();

    await myDestination.push(testEvent, {
      config: {},
      logger: mockLogger,
    });

    expect(mockLogger.debug).toHaveBeenCalledWith('Processing event', {
      name: testEvent.name,
    });
  });
});

The mock logger provides jest mock functions for all logger methods plus:

  • scopedLoggers: Array of scoped loggers created via scope()
💡 Need implementation support?
elbwalker offers hands-on support: setup review, measurement planning, destination mapping, and live troubleshooting. Book a 2-hour session (€399)