# walkerOS Documentation > Privacy-first, composable event data collection (Source → Collector → Destination). ## docs Event data collection as code - capture, transform, and route events with built-in consent management - [What is walkerOS](/docs.md): Event data collection as code - capture, transform, and route events with built-in consent management ### getting-started - [Deploy](/docs/getting-started/deploy.md): Take your flow to production, self-hosted or managed - [Event model](/docs/getting-started/event-model.md): Understanding the walkerOS event data structure - [Bot detection](/docs/getting-started/examples/bot-detection.md): Annotate every server event with a bot and AI-agent score - [Fingerprint to BigQuery](/docs/getting-started/examples/fingerprint-bigquery.md): Hash IP and user agent into a privacy-friendly id, then land events in BigQuery - [One event, four tools](/docs/getting-started/examples/mapping-comparison.md): Fan a single event stream out to GA4, Amplitude, Mixpanel, and PostHog - [First critical checks](/docs/getting-started/first-checks.md): Evaluate walkerOS in 15 minutes, five proofs on your own machine - [Flow](/docs/getting-started/flow.md): Understanding the walkerOS flow configuration file structure - [Contract](/docs/getting-started/flow/contract.md): Named, inheritable event schemas that define what your data should look like - [Step examples](/docs/getting-started/flow/step-examples.md): Embed input/output examples on each flow step for testing, documentation, and simulation - [Validate](/docs/getting-started/flow/validate.md): Enforce a contract at runtime with the validate transformer. - [GA4 ecommerce](/docs/getting-started/ga4-ecommerce.md): Send a page view and a product add event to Google Analytics 4 - [Operating modes](/docs/getting-started/modes.md): Two ways to use walkerOS - integrated and bundled - [Bundled mode](/docs/getting-started/modes/bundled.md): Configure walkerOS with JSON and build standalone bundles with the CLI - [Integrated mode](/docs/getting-started/modes/integrated.md): Build walkerOS directly into your application with full TypeScript control - [Quickstart](/docs/getting-started/quickstart.md): Send your first walkerOS event in five minutes - [Docker](/docs/getting-started/quickstart/docker.md): Get walkerOS running with Docker in 5 minutes - [Next.js Quickstart](/docs/getting-started/quickstart/nextjs.md): Get started with walkerOS in Next.js applications - [React Quickstart](/docs/getting-started/quickstart/react.md): Get started with walkerOS in React applications - [Stores](/docs/getting-started/stores.md): Key-value storage for walkerOS components ### sources Event data sources for walkerOS collector - [Sources](/docs/sources.md): Event data sources for walkerOS collector - [Create Your Own Source](/docs/sources/create-your-own.md): Guide to building custom sources for walkerOS - [Server sources](/docs/sources/server.md): HTTP event collection sources for server environments - [AWS Lambda](/docs/sources/server/aws.md): AWS Lambda HTTP handler for event collection - [Express](/docs/sources/server/express.md): Turn-key HTTP event collection server with Express.js - [Fetch](/docs/sources/server/fetch.md): Web Standard Fetch API source for edge and serverless platforms - [GCP Cloud Functions](/docs/sources/server/gcp.md): Google Cloud Functions HTTP handler source for walkerOS - [Google Pub/Sub](/docs/sources/server/pubsub.md): Server-side ingestion from Google Cloud Pub/Sub via streaming pull subscriber or HTTP push webhook - [AWS SQS](/docs/sources/server/sqs.md): Server-side ingestion from Amazon SQS via streaming long-poll subscriber - [Browser source](/docs/sources/web/browser.md): DOM-based event tracking for web applications - [Browser source commands](/docs/sources/web/browser/commands.md): Browser source API methods and commands - [Tagger](/docs/sources/web/browser/tagger.md): HTML data attribute generator for walkerOS event tracking - [HTML Attributes](/docs/sources/web/browser/tagging/html-attributes.md): How to tag a page using data-elb HTML attributes - [JavaScript](/docs/sources/web/browser/tagging/javascript.md): Capture events programmatically using the elb function - [CMP sources](/docs/sources/web/cmps.md): Automate consent handling with walkerOS CMP source packages - [CookieFirst](/docs/sources/web/cmps/cookiefirst.md): Integrate CookieFirst consent management with walkerOS - [CookiePro / OneTrust](/docs/sources/web/cmps/cookiepro.md): Integrate CookiePro/OneTrust consent management with walkerOS - [Usercentrics](/docs/sources/web/cmps/usercentrics.md): Integrate Usercentrics consent management with walkerOS - [DataLayer source](/docs/sources/web/dataLayer.md): Integration with existing dataLayer implementations - [Session source](/docs/sources/web/session.md): Standalone session detection and management for walkerOS ### collector Unified event collector for web and server environments - [Collector](/docs/collector.md): Unified event collector for web and server environments - [Cache](/docs/collector/cache.md): Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests. - [Collector commands](/docs/collector/commands.md): Collector API methods and commands - [Logger](/docs/collector/logger.md): Centralized logging system for walkerOS - [State](/docs/collector/state.md): State is a built-in config property available on source, transformer, and destination steps. It replaces inline $code stash a value (set) and fetch a value (get). Both directions reuse the same mapping engine, so any value you can express in a mapping you can stash or fetch. ### destinations Send events to analytics tools, marketing platforms, and custom endpoints - [Destinations](/docs/destinations.md): Send events to analytics tools, marketing platforms, and custom endpoints - [API Destination](/docs/destinations/api.md): Send events to any HTTP endpoint - browser or server-side - [Server API Destination](/docs/destinations/api/server.md): Server-side HTTP API destination for Node.js - [Web API Destination](/docs/destinations/api/web.md): Browser-side HTTP API destination with fetch, XHR, and beacon transports - [Code Destination](/docs/destinations/code.md): Built-in destination for executing custom code strings - [Create Your Own Destination](/docs/destinations/create-your-own.md): A guide to building custom destinations for walkerOS. - [Amplitude](/docs/destinations/server/amplitude.md): Server-side event delivery to Amplitude for product analytics with per-event identity - [AWS Firehose](/docs/destinations/server/aws.md): Stream events to Amazon Kinesis Data Firehose for real-time analytics - [Bing UET Conversions API](/docs/destinations/server/bing.md): Server-side event delivery to Microsoft Advertising (Bing) UET Conversions API for enhanced conversion tracking - [Criteo Events API](/docs/destinations/server/criteo.md): Server-side event delivery to Criteo's Events API for retargeting - [Customer.io](/docs/destinations/server/customerio.md): Server-side event delivery to Customer.io for email, SMS, push, and in-app messaging automation - [Google Data Manager API](/docs/destinations/server/datamanager.md): Server-side event ingestion via the Google Data Manager API to Google Ads, Display & Video 360, and Google Analytics 4 - [File](/docs/destinations/server/file.md): Local file sink for walkerOS server flows. Appends events as JSONL, TSV, or CSV for debug logging, audit trails, and cold archive. - [Google BigQuery](/docs/destinations/server/gcp.md): Stream events to Google BigQuery via the Storage Write API for analytics and machine learning workloads - [HubSpot](/docs/destinations/server/hubspot.md): Server-side event delivery to HubSpot CRM (custom events + contact upsert) - [Apache Kafka](/docs/destinations/server/kafka.md): Server-side event streaming to Apache Kafka topics via kafkajs with JSON serialization and compression - [Klaviyo](/docs/destinations/server/klaviyo.md): Server-side event delivery to Klaviyo for email and SMS marketing automation - [LinkedIn Conversions API](/docs/destinations/server/linkedin.md): Server-side conversion event delivery to LinkedIn for enhanced ad attribution - [Meta Conversions API](/docs/destinations/server/meta-capi.md): Server-side Meta advertising events for enhanced privacy and tracking accuracy - [Mixpanel](/docs/destinations/server/mixpanel.md): Server-side event delivery to Mixpanel for product analytics with people profiles and groups - [mParticle](/docs/destinations/server/mparticle.md): Server-side event delivery to mParticle CDP via input feed batches - [Pinterest Conversions API](/docs/destinations/server/pinterest.md): Server-side event delivery to Pinterest Conversions API for enhanced conversion tracking - [PostHog](/docs/destinations/server/posthog.md): Server-side event delivery to PostHog for product analytics and feature flags - [Google Pub/Sub](/docs/destinations/server/pubsub.md): Server-side event publishing to Google Cloud Pub/Sub topics with per-key ordering and dynamic attributes - [Reddit Conversions API](/docs/destinations/server/reddit.md): Server-side event delivery to Reddit Conversions API for enhanced conversion tracking - [Redis Streams](/docs/destinations/server/redis.md): Server-side event streaming to Redis Streams via ioredis with XADD, MAXLEN trimming, and JSON or flat serialization - [RudderStack](/docs/destinations/server/rudderstack.md): Server-side event delivery to RudderStack CDP (open-source Segment alternative) with full Segment Spec support - [Segment](/docs/destinations/server/segment.md): Server-side event delivery to Segment CDP for routing to 400+ downstream destinations - [Slack](/docs/destinations/server/slack.md): Server-side Slack notifications with dual-mode support (Incoming Webhooks + Web API) - [Snapchat Conversions API](/docs/destinations/server/snapchat.md): Server-side event delivery to Snapchat Conversions API for enhanced conversion tracking - [AWS SNS](/docs/destinations/server/sns.md): Server-side event publishing to AWS SNS topics with FIFO ordering, KMS encryption, and idempotent topic provisioning - [SQLite](/docs/destinations/server/sqlite.md): Server-side event persistence to a local SQLite file (better-sqlite3) or remote libSQL / Turso (@libsql/client). Driver picked automatically from the URL. - [TikTok Events API](/docs/destinations/server/tiktok.md): Server-side event delivery to TikTok Events API for enhanced conversion tracking - [X (Twitter) Conversions API](/docs/destinations/server/twitter.md): Server-side conversion event delivery to X (Twitter) for enhanced ad attribution - [Amplitude](/docs/destinations/web/amplitude.md): Product analytics, identity, revenue, and optional session replay, experiments, and guides & surveys - [Microsoft Clarity](/docs/destinations/web/clarity.md): Session replays, heatmaps, and behavioural insights - [d8a](/docs/destinations/web/d8a.md): GA4-compatible, warehouse-native analytics - [FullStory](/docs/destinations/web/fullstory.md): Session replay, custom events, and user/page properties - [Google Gtag](/docs/destinations/web/gtag.md): Unified Google destination supporting GA4, Google Ads, and GTM through a single gtag implementation - [Google Ads](/docs/destinations/web/gtag/ads.md): Google Ads conversion tracking within the unified gtag destination - [Google Analytics 4 (GA4)](/docs/destinations/web/gtag/ga4.md): Google Analytics 4 configuration within the unified gtag destination - [Google Tag Manager (GTM)](/docs/destinations/web/gtag/gtm.md): Google Tag Manager configuration within the unified gtag destination - [Heap](/docs/destinations/web/heap.md): Product analytics with auto-capture and retroactive events - [Hotjar](/docs/destinations/web/hotjar.md): Session recordings, heatmaps, and on-page feedback - [LinkedIn Insight Tag](/docs/destinations/web/linkedin.md): Conversion tracking and retargeting for LinkedIn Campaign Manager - [Matomo](/docs/destinations/web/matomo.md): Self-hosted or cloud privacy-first web analytics (GA alternative) - [Meta Pixel](/docs/destinations/web/meta-pixel.md): Facebook and Instagram advertising pixel integration - [Mixpanel](/docs/destinations/web/mixpanel.md): Product analytics with people profiles and group analytics - [Optimizely](/docs/destinations/web/optimizely.md): Feature experimentation, A/B testing, and conversion tracking - [Piano Analytics](/docs/destinations/web/piano.md): Send events to Piano Analytics - [Pinterest Tag](/docs/destinations/web/pinterest.md): Conversion tracking, enhanced matching, and dynamic retargeting via the Pinterest Tag - [Piwik PRO](/docs/destinations/web/piwikpro.md): Privacy-focused web analytics and visitor data platform - [Plausible Analytics](/docs/destinations/web/plausible.md): Simple, privacy-friendly and lightweight analytics - [PostHog](/docs/destinations/web/posthog.md): Product analytics, session replay, feature flags, surveys - [Segment](/docs/destinations/web/segment.md): Segment CDP, routing events to 400+ downstream destinations - [Snowplow](/docs/destinations/web/snowplow.md): Behavioral data platform destination with ecommerce tracking - [TikTok Pixel](/docs/destinations/web/tiktok.md): Conversion tracking and Advanced Matching for TikTok Ads ### transformers Event processing middleware for walkerOS collector - [Transformers](/docs/transformers.md): Event processing middleware for walkerOS collector - [Bot detection](/docs/transformers/bot.md): Server-side bot and AI-agent annotation - [Create your own transformer](/docs/transformers/create-your-own.md): Guide to building custom transformers for walkerOS - [File](/docs/transformers/file.md): Serve static files from a store backend with MIME detection - [Fingerprint](/docs/transformers/fingerprint.md): Server-side user identification via deterministic hashing - [GA4](/docs/transformers/ga4.md): Decode Google Analytics 4 Measurement Protocol v2 hits into walkerOS events - [Validate](/docs/transformers/validate.md): Enforce JSON Schema contracts on events at runtime ### stores Key-value storage for walkerOS flows - [Stores](/docs/stores.md): Key-value storage for walkerOS flows - [Store-level cache](/docs/stores/cache.md): Cache reads and writes on any walkerOS store with the built-in tier, compose multi-tier chains, deduplicate concurrent reads. - [Filesystem](/docs/stores/server/fs.md): Local filesystem store for walkerOS server flows - [GCS](/docs/stores/server/gcs.md): Google Cloud Storage for walkerOS server flows - [S3](/docs/stores/server/s3.md): S3-compatible object storage for walkerOS server flows - [Sheets](/docs/stores/server/sheets.md): Google Sheets store for walkerOS server flows ### mapping Transform events with a code-free, declarative mapping syntax - [Mapping](/docs/mapping.md): Transform events with a code-free, declarative mapping syntax - [Mapping.Rule](/docs/mapping/rule.md): How a destination rule composes values into a complete transform - [Mapping.Value](/docs/mapping/value.md): The polymorphic primitive you put into any mapping field ### apps Ready-to-use walkerOS bundle for instant web tracking - [Apps](/docs/apps.md): Ready-to-use walkerOS bundle for instant web tracking - [walkerOS CLI](/docs/apps/cli.md): walkerOS command-line tool for building, testing, and running flows - [walkerOS Docker](/docs/apps/docker.md): Deploy walkerOS flows with Docker containers - [MCP servers](/docs/apps/mcp.md): Use walkerOS tools from AI assistants via Model Context Protocol - [walkerOS Runner](/docs/apps/runner.md): Self-hosted walkerOS runner with config polling, hot-swap, and dashboard visibility - [Storybook addon](/docs/apps/storybook.md): Visualize and debug walkerOS events in Storybook - [Telemetry](/docs/apps/telemetry.md): What walkerOS CLI and MCP can collect, how to opt in, and what "opt in" means legally. - [walker.js](/docs/apps/walkerjs.md): Ready-to-use walkerOS bundle for instant web tracking ### guides How-to guides and best practices for walkerOS - [Guides](/docs/guides.md): How-to guides and best practices for walkerOS - [Consent management](/docs/guides/consent.md): Privacy-compliant consent handling and management with walkerOS - [My events aren't arriving](/docs/guides/debugging.md): A how-to for tracing walkerOS events when a destination stays silent - [Deploy walkerOS to Bunny Magic Containers](/docs/guides/deploy-bunny-magic-containers.md): Deploy walkerOS to Bunny Magic Containers with BigQuery destination - [Deploy walkerOS to GCP Cloud Run](/docs/guides/deploy-gcp-cloudrun.md): Deploy walkerOS to Google Cloud Run with BigQuery destination - [Migration guide: from @elbwalker to @walkerOS](/docs/guides/migration.md): How to migrate to the latest walkerOS packages - [Reference Syntax](/docs/guides/reference-syntax.md): Dynamic references inside flow.json, variables, env vars, contracts, stores, secrets, inline code. - [Session](/docs/guides/session.md): Detect and track user sessions with configurable storage and consent options - [User stitching](/docs/guides/user-stitching.md): User identification and cross-device/cross-session data unification with walkerOS ### comparisons Comparisons between walkerOS and other popular tracking and analytics solutions - [Comparisons](/docs/comparisons.md): Comparisons between walkerOS and other popular tracking and analytics solutions - [walkerOS tagging vs. dataLayer.push](/docs/comparisons/dataLayer.md): Compare walkerOS HTML tagging with manual dataLayer.push calls. - [walkerOS vs. Google Tag Manager](/docs/comparisons/gtm.md): Compare walkerOS and Google Tag Manager to understand which solution fits your needs. - [walkerOS vs. Jentis](/docs/comparisons/jentis.md): Compare walkerOS and Jentis to understand which solution fits your needs. - [walkerOS vs. Jitsu](/docs/comparisons/jitsu.md): Compare walkerOS and Jitsu to understand which solution fits your needs. - [walkerOS vs. RudderStack](/docs/comparisons/rudderstack.md): Compare walkerOS and RudderStack to understand which solution fits your needs. - [walkerOS vs. Segment](/docs/comparisons/segment.md): Compare walkerOS and Twilio Segment to understand which solution fits your needs. - [walkerOS vs. Snowplow](/docs/comparisons/snowplow.md): Compare walkerOS and Snowplow to understand which solution fits your needs. - [walkerOS vs. Stape](/docs/comparisons/stape.md): Compare walkerOS and Stape to understand which solution fits your needs. ### contributing Learn how to contribute to walkerOS - [Contributing](/docs/contributing.md): Learn how to contribute to walkerOS ### core Core libraries and utilities for walkerOS - [Core utilities](/docs/core.md): Core libraries and utilities for walkerOS - [Server Core Utilities](/docs/core/server.md): Server-specific utilities and extensions for walkerOS - [Web Core Utilities](/docs/core/web.md): Web-specific utilities and extensions for walkerOS ### migrate - [Migrating to @walkeros/cli 4.x](/docs/migrate/cli-4x.md): nft-traced bundler, output filename, Dockerfile updates ### migrating - [Migrating flow.json from v3 to v4](/docs/migrating/v3-to-v4.md): Upgrade existing flow.json files to the v4 schema introduced in walkerOS Flow v4. ### plans - [FlowMap Diagrams Implementation Plan](/docs/plans/2026-01-14-flowmap-diagrams.md): For Claude Use superpowers:executing-plans to ### upgrading How walkerOS clients negotiate versions with the app, and what to do when you see a 426 Upgrade Required response. - [Upgrading walkerOS clients](/docs/upgrading.md): How walkerOS clients negotiate versions with the app, and what to do when you see a 426 Upgrade Required response. ## skills Index of walkerOS skills, the source of truth for AI assistants and developers. - [walkerOS Skills](/skills.md): Index of walkerOS skills, the source of truth for AI assistants and developers. ### walkeros-create-cmp-source Use when creating a new walkerOS CMP (consent management platform) source. Structured fill-in-the-blanks workflow that turns any CMP's consent API into a walkerOS source package. Covers CookieFirst, Usercentrics, CookiePro/OneTrust patterns and generalizes to any CMP. - [Create a CMP Source](/skills/walkeros-create-cmp-source.md): Use when creating a new walkerOS CMP (consent management platform) source. Structured fill-in-the-blanks workflow that turns any CMP's consent API into a walkerOS source package. Covers CookieFirst, Usercentrics, CookiePro/OneTrust patterns and generalizes to any CMP. ### walkeros-create-destination Use when creating a new walkerOS destination to send events to a vendor or API (GA4/gtag, Meta/Facebook Pixel, Mixpanel, Amplitude, a custom HTTP API, Measurement Protocol), web or server-side. Example-driven workflow: research the vendor SDK and define step examples before implementing the destination interface, env pattern, and mappings. - [Create a New Destination](/skills/walkeros-create-destination.md): Use when creating a new walkerOS destination to send events to a vendor or API (GA4/gtag, Meta/Facebook Pixel, Mixpanel, Amplitude, a custom HTTP API, Measurement Protocol), web or server-side. Example-driven workflow: research the vendor SDK and define step examples before implementing the destination interface, env pattern, and mappings. ### walkeros-create-source Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern. - [Create a New Source](/skills/walkeros-create-source.md): Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern. ### walkeros-create-transformer Use when creating a new walkerOS transformer to modify events in the pipeline (validate, enrich, or redact) at a before/next chain position. Example-driven workflow for the transformer interface, return values, and chaining. - [Create a New Transformer](/skills/walkeros-create-transformer.md): Use when creating a new walkerOS transformer to modify events in the pipeline (validate, enrich, or redact) at a before/next chain position. Example-driven workflow for the transformer interface, return values, and chaining. ### walkeros-debugging Use when walkerOS events aren't reaching destinations, debugging event flow, or troubleshooting mapping issues. Covers common problems and debugging strategies. - [Debugging walkerOS Events](/skills/walkeros-debugging.md): Use when walkerOS events aren't reaching destinations, debugging event flow, or troubleshooting mapping issues. Covers common problems and debugging strategies. ### walkeros-mapping-configuration Use when configuring walkerOS event mappings for specific use cases. Provides recipes for GA4, Meta, custom APIs, and common transformation patterns. - [Mapping Configuration Recipes](/skills/walkeros-mapping-configuration.md): Use when configuring walkerOS event mappings for specific use cases. Provides recipes for GA4, Meta, custom APIs, and common transformation patterns. ### walkeros-mcp-actions Use when an AI agent calls walkerOS MCP tools from code (code execution / Code Mode, a Worker, or an app sandbox) to validate, simulate, bundle, or inspect flows and packages, and wants to filter large results in code instead of issuing many separate tool calls. - [Calling walkerOS MCP Tools from Code](/skills/walkeros-mcp-actions.md): Use when an AI agent calls walkerOS MCP tools from code (code execution / Code Mode, a Worker, or an app sandbox) to validate, simulate, bundle, or inspect flows and packages, and wants to filter large results in code instead of issuing many separate tool calls. ### walkeros-testing-strategy Use when writing tests, reviewing test code, or discussing testing approach for walkerOS packages. Covers env pattern, dev examples, and package-specific strategies. - [walkerOS Testing Strategy](/skills/walkeros-testing-strategy.md): Use when writing tests, reviewing test code, or discussing testing approach for walkerOS packages. Covers env pattern, dev examples, and package-specific strategies. ### walkeros-understanding-destinations Use when working with walkerOS destinations, understanding the destination interface, or learning about env pattern and configuration. Covers interface, lifecycle, env mocking, and paths. - [Understanding walkerOS Destinations](/skills/walkeros-understanding-destinations.md): Use when working with walkerOS destinations, understanding the destination interface, or learning about env pattern and configuration. Covers interface, lifecycle, env mocking, and paths. ### walkeros-understanding-development Use when contributing to walkerOS, before writing code, or when unsure about project conventions. Covers build/test/lint workflow, XP principles, folder structure, and package usage. - [Understanding walkerOS Development](/skills/walkeros-understanding-development.md): Use when contributing to walkerOS, before writing code, or when unsure about project conventions. Covers build/test/lint workflow, XP principles, folder structure, and package usage. ### walkeros-understanding-events Use when creating walkerOS events, understanding event structure, or working with event properties. Covers entity-action naming, event properties, statelessness, and vendor-agnostic design. - [Understanding walkerOS Events](/skills/walkeros-understanding-events.md): Use when creating walkerOS events, understanding event structure, or working with event properties. Covers entity-action naming, event properties, statelessness, and vendor-agnostic design. ### walkeros-understanding-flow Use when learning walkerOS architecture, understanding data flow, or designing composable event pipelines. Covers Source→Collector→Destination pattern and separation of concerns. - [Understanding walkerOS Flow](/skills/walkeros-understanding-flow.md): Use when learning walkerOS architecture, understanding data flow, or designing composable event pipelines. Covers Source→Collector→Destination pattern and separation of concerns. ### walkeros-understanding-mapping Use when transforming walkerOS events in the flow (source→collector or collector→destination), configuring data/map/loop/set/condition/policy, or using $code: syntax in JSON configs. - [Understanding walkerOS Mapping](/skills/walkeros-understanding-mapping.md): Use when transforming walkerOS events in the flow (source→collector or collector→destination), configuring data/map/loop/set/condition/policy, or using $code: syntax in JSON configs. ### walkeros-understanding-sources Use when working with walkerOS sources, understanding event capture, or learning about the push interface. Covers browser, dataLayer, and server source patterns. - [Understanding walkerOS Sources](/skills/walkeros-understanding-sources.md): Use when working with walkerOS sources, understanding event capture, or learning about the push interface. Covers browser, dataLayer, and server source patterns. ### walkeros-understanding-stores Use when working with walkerOS stores, understanding key-value storage in flows, or learning about store injection via env. Covers interface, lifecycle, $store. wiring, and available store packages. - [Understanding walkerOS Stores](/skills/walkeros-understanding-stores.md): Use when working with walkerOS stores, understanding key-value storage in flows, or learning about store injection via env. Covers interface, lifecycle, $store. wiring, and available store packages. ### walkeros-understanding-transformers Use when working with walkerOS transformers, understanding event validation/enrichment/redaction, or learning about transformer chaining. Covers interface, return values, and pipeline integration. - [Understanding walkerOS Transformers](/skills/walkeros-understanding-transformers.md): Use when working with walkerOS transformers, understanding event validation/enrichment/redaction, or learning about transformer chaining. Covers interface, return values, and pipeline integration. ### walkeros-using-cli Use when bundling walkerOS flows, testing events with simulate/push, running local servers, validating configs, or configuring Flow JSON files. - [Using the walkerOS CLI](/skills/walkeros-using-cli.md): Use when bundling walkerOS flows, testing events with simulate/push, running local servers, validating configs, or configuring Flow JSON files. ### walkeros-using-logger Use when working with walkerOS sources/destinations to understand standard logging patterns, replace console.log, or add logging to external API calls. Covers DRY principles, when to log, and migration patterns. - [Using the walkerOS Logger](/skills/walkeros-using-logger.md): Use when working with walkerOS sources/destinations to understand standard logging patterns, replace console.log, or add logging to external API calls. Covers DRY principles, when to log, and migration patterns. ### walkeros-using-step-examples Use when writing, simulating, validating, or testing with walkerOS step examples. Covers the complete lifecycle from authoring examples to CI integration. - [Using Step Examples](/skills/walkeros-using-step-examples.md): Use when writing, simulating, validating, or testing with walkerOS step examples. Covers the complete lifecycle from authoring examples to CI integration. ### walkeros-using-store-cache Use when adding read-through caching to a walkerOS store, memoizing a slow API/Sheets backing, composing multi-tier cache chains, or deduplicating concurrent store reads. Covers recipes, TTL choice, error policy, and observability counters. - [Using store-level cache](/skills/walkeros-using-store-cache.md): Use when adding read-through caching to a walkerOS store, memoizing a slow API/Sheets backing, composing multi-tier cache chains, or deduplicating concurrent store reads. Covers recipes, TTL choice, error policy, and observability counters. ### walkeros-using-transformer-ga4 Use when wiring `@walkeros/transformer-ga4` into a server flow, overriding default GA4 event mappings, dropping events, adding custom event keys, or troubleshooting GA4 Measurement Protocol decoding. Covers the `before`-chain wiring contract, configuration recipes, and per-field patching with extend/remove. - [Using @walkeros/transformer-ga4](/skills/walkeros-using-transformer-ga4.md): Use when wiring `@walkeros/transformer-ga4` into a server flow, overriding default GA4 event mappings, dropping events, adding custom event keys, or troubleshooting GA4 Measurement Protocol decoding. Covers the `before`-chain wiring contract, configuration recipes, and per-field patching with extend/remove. ### walkeros-writing-documentation Use when writing or updating walkerOS documentation - README, website docs, or skills. Covers quality standards, example validation, and DRY patterns. - [Writing Documentation](/skills/walkeros-writing-documentation.md): Use when writing or updating walkerOS documentation - README, website docs, or skills. Covers quality standards, example validation, and DRY patterns. --- # Full Documentation Content # What is walkerOS walkerOS is an **open-source event data collection platform** you run as code. Capture events from anywhere, transform them with mapping, and route them to any destination, all with built-in consent management. No fragile UI configs. No vendor lock-in. Just tracking you can version, test, and trust. It's built for analytics engineers, data leads, and developers who own analytics on a custom-built app and want to replace a duct-taped GA4 and server-side tag manager stack with a pipeline they actually control. ## Try it now[​](#try-it-now "Direct link to Try it now") **[Send your first event in five minutes → Quickstart](/docs/getting-started/quickstart/.md)** Then: [send it to GA4 with mapping →](/docs/getting-started/ga4-ecommerce.md) Working with an AI assistant? Build and simulate a flow locally with the walkerOS MCP, no account needed: [With an AI assistant](/docs/apps/mcp.md). ## Why walkerOS?[​](#why-walkeros "Direct link to Why walkerOS?") * **Open-Source:** MIT licensed, free forever, no restrictions * **Self-hostable:** Run it on your own infrastructure * **Config-as-Code:** Version control, PRs, deploy with confidence * **Composable:** Mix sources, destinations, and mappings freely * **Developer-first:** TypeScript and declarative tagging, with an optional hosted app and MCP/AI-assisted management *** ## How it works[​](#how-it-works "Direct link to How it works") Every walkerOS setup follows one pattern: * **Sources:** Where events come from (browser DOM, dataLayer, HTTP endpoints, custom code) * **Transformers:** Middleware that validates, enriches, routes, or redacts events before and after the collector * **Collector:** The processing engine (consent, enrichment, routing) * **Destinations:** Where events go (GA4, your API, BigQuery, any service) * **Stores:** Shared key-value storage used by transformers and destinations (caching, file serving, session state) A "flow" connects sources to destinations through a collector. You can have one flow or many, client-side, server-side, or both working together. ## The centerpiece: mapping[​](#the-centerpiece-mapping "Direct link to The centerpiece: mapping") **Mapping** transforms events between stages. It's how walkerOS adapts to any source or destination format. One transformation language works everywhere. This is what makes walkerOS truly composable. [Learn more about mapping →](/docs/mapping/.md) *** ## Core concepts[​](#core-concepts "Direct link to Core concepts") | Concept | What It Is | Learn More | | ---------------- | --------------------------------------------------------------------- | ---------------------------------------------------- | | **Flow** | Sources + collector + destinations configuration | [Flow configuration](/docs/getting-started/flow/.md) | | **Event model** | How events are structured (`entity action` format) | [Event model](/docs/getting-started/event-model.md) | | **Mapping** | Event transformation rules | [Mapping](/docs/mapping/.md) | | **Consent** | Privacy controls at destination and property level | [Consent management](/docs/guides/consent/.md) | | **Sources** | Event capture from browser, server, or custom code | [Sources](/docs/sources/.md) | | **Transformers** | Middleware for validating, enriching, routing, and redacting events | [Transformers](/docs/transformers/.md) | | **Destinations** | Event delivery to analytics, APIs, warehouses | [Destinations](/docs/destinations/.md) | | **Stores** | Shared key-value storage for caching, file serving, and session state | [Stores](/docs/stores/.md) | *** ## Two ways to use walkerOS[​](#two-ways-to-use-walkeros "Direct link to Two ways to use walkerOS") Both modes use the same architecture. You choose at deploy time, after your first event, see [operating modes](/docs/getting-started/modes/.md). ### Integrated mode[​](#integrated-mode "Direct link to Integrated mode") The collector lives **inside your application code**. ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser } }, }); ``` **Best for:** Building into your app, full TypeScript control, dynamic configuration. [**Get started with integrated mode →**](/docs/getting-started/modes/integrated.md) *** ### Bundled mode[​](#bundled-mode "Direct link to Bundled mode") The collector is a **separate artifact** you configure with JSON. ``` { "version": 4, "flows": { "default": { "config": { "platform": "web" }, "sources": { "browser": { "package": "@walkeros/web-source-browser" } } } } } ``` Then: `walkeros bundle flow.json` → standalone file. **Best for:** Config-as-code, Docker deployments, separate tracking scripts. [**Get started with bundled mode →**](/docs/getting-started/modes/bundled.md) *** ## Quick links[​](#quick-links "Direct link to Quick links") **Deploying to production?** * [Deploy](/docs/getting-started/deploy.md): self-hosted or managed * [CLI Reference](/docs/apps/cli.md): Build and test commands **Already using walker.js?** * [Walker.js Docs](/docs/apps/walkerjs.md): Pre-built convenience bundle **Need help choosing?** * [Operating Modes](/docs/getting-started/modes/.md): Detailed comparison --- # Apps --- # walkerOS CLI The walkerOS CLI (`@walkeros/cli`) is a command-line tool for building, testing, and running event collection flows. It handles the complete workflow from configuration to deployment. It bundles flows into optimized JavaScript, testing with simulated events, and running collection servers locally or in Docker. ## Installation[​](#installation "Direct link to Installation") ### Global Installation (Recommended)[​](#global-installation-recommended "Direct link to Global Installation (Recommended)") Install globally to use the `walkeros` command anywhere: ``` npm install -g @walkeros/cli # Verify installation walkeros --version ``` ### Local Installation[​](#local-installation "Direct link to Local Installation") Install in your project for team consistency: ``` npm install --save-dev @walkeros/cli # Use with npx npx walkeros --version ``` ## Commands overview[​](#commands-overview "Direct link to Commands overview") | Command | Purpose | Use Case | | ---------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | `bundle` | Build production-ready bundle from flow config | Create deployable JavaScript from configuration | | `push` | Execute event with real API calls (or `--simulate` for mocked) | Testing and production validation | | `setup` | Run the optional `setup()` lifecycle on one component | Provision external resources (BigQuery datasets, Pub/Sub topics, SQLite tables, webhook registrations) | | `validate` | Validate events, flows, mappings, contracts, or entries | Check configuration before bundling | | `run` | Start HTTP event collection server | Accept incoming events via HTTP POST | | `cache` | Manage CLI package and build caches | Clear stale caches, view cache statistics | | `auth` | Authentication and identity | Log in, log out, check identity | | `projects` | Manage walkerOS projects | Create, list, update, delete projects | | `flows` | Manage walkerOS flows | Create, list, update, delete, duplicate flows | | `deploy` | Create and manage deployments | Deploy flows to walkerOS cloud or self-hosted | | `previews` | Manage preview bundles | Test flow changes on live sites before deploying | The `push` command accepts either a config JSON or a pre-built bundle as input. ## Configuration types[​](#configuration-types "Direct link to Configuration types") The CLI uses types from `@walkeros/core`: * **`Flow.Json`** - Root config file format (`version`, `flows`) * **`Flow`** - Single flow (has `config`, `sources`, `destinations`, `transformers`, `collector`) * **`Flow.Config`** - Per-flow config block (`platform`, `url`, `settings`, `bundle`) * **`Collector.InitConfig`** - Runtime type passed to `startFlow()` The CLI transforms `Flow.Json` → `Flow` (per flow) → bundled code that uses `Collector.InitConfig` at runtime. ## Packages and overrides[​](#packages-and-overrides "Direct link to Packages and overrides") Build-time concerns live under each flow's `config.bundle` block. Three fields are supported: * **`packages`**: the list of npm packages (or local paths) the bundler should install for this flow. Pacote downloads them based on this list. You do **not** need to run `npm install` yourself for step packages; the CLI handles it transparently. * **`overrides`**: pin transitive dependency versions, matching npm's `overrides` semantics. Useful when an upstream package declares an overly narrow version range that conflicts with a newer transitive version in the same tree. * **`traceInclude`** (server flows only): an escape hatch for the file tracer. See below. ``` "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/collector": {} }, "overrides": { "@amplitude/analytics-types": "2.11.1" } } } } } ``` ### Server bundles use nft tracing[​](#server-bundles-use-nft-tracing "Direct link to Server bundles use nft tracing") Server flows (`platform: "server"`) are traced automatically with [`@vercel/nft`](https://github.com/vercel/nft). The bundler externalizes every step package, then nft walks the entry to figure out which files are actually reachable at runtime, and copies only those files into `dist/node_modules/`. Native add-ons, `.proto` files, and `__dirname`-loaded assets are picked up. There is no `walkerOS.bundle.external` annotation: nft figures out the externalization automatically. ### Escape hatch: `traceInclude`[​](#escape-hatch-traceinclude "Direct link to escape-hatch-traceinclude") If nft cannot statically reach a file (rare: dynamic `require(somePath)` constructed from a runtime variable, dynamically loaded data files), declare it explicitly with `flow..config.bundle.traceInclude`. Paths and globs are both supported (matched via picomatch). Paths resolve against the install root, not your project directory. ``` "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-destination-gcp": {} }, "traceInclude": [ "node_modules/some-pkg/data/*.json", "node_modules/another-pkg/lib/runtime-loaded.js" ] } } } } ``` ### Range conflict resolution[​](#range-conflict-resolution "Direct link to Range conflict resolution") When two transitive consumers declare incompatible ranges for the same dependency (e.g., `arrify@^3.0.0` from one package, `arrify@^2.0.0` from another), the bundler now resolves the chosen range to a concrete version via the npm registry and nests any non-satisfying spec under its consumer's `node_modules/`. This kills classes of upstream chaos (like `arrify is not a function` from `@google-cloud/common`) before they can reach runtime. After install, a defense-in-depth check warns when any installed package's declared deps don't match what's reachable on its resolution path; if you see those warnings, add an `overrides` entry to your flow's `config.bundle.overrides` to pin the offending dep. If the npm registry is unreachable during validation, set `BUNDLER_STRICT_RANGES=0` to bypass strict validation with a warning instead of failing the build. ### Schema version[​](#schema-version "Direct link to Schema version") The flow\.json schema stays at `version: 4`. Build-time fields live under `flow..config.bundle.{packages, overrides, traceInclude}`. The `flow..config.bundle.external` sub-field is no longer supported in @walkeros/cli\@4.x. ## Package imports[​](#package-imports "Direct link to Package imports") ### Implicit collector[​](#implicit-collector "Direct link to Implicit collector") The collector is added automatically. To pin a specific version (recommended for production), add the collector explicitly: ``` "packages": { "@walkeros/collector": { "version": "1.0.0" }, "@walkeros/web-source-browser": {}, "@walkeros/destination-demo": {} } ``` ### Default exports (sources & destinations)[​](#default-exports-sources--destinations "Direct link to Default exports (sources & destinations)") Sources and destinations automatically use their **default export** - no `imports` needed: ``` "packages": { "@walkeros/web-source-browser": {}, // Uses default export "@walkeros/destination-demo": {} // Uses default export } ``` ### Utility imports[​](#utility-imports "Direct link to Utility imports") Use `imports` when you need specific utility functions from a package: ``` "packages": { "@walkeros/core": { "imports": ["getId", "clone"] }, // Optional utilities "@walkeros/web-source-browser": { "imports": ["createTagger"] // Additional utility } } ``` ### Named-export selection with `import`[​](#named-export-selection-with-import "Direct link to named-export-selection-with-import") For packages without a default export, or to pick a specific named export, set `import` on the step alongside `package`. The bundler adds the name to the `imports` list automatically. ``` "sources": { "custom": { "package": "@my/custom-package", "import": "namedExport" } } ``` The legacy `code: "namedExport"` form is no longer accepted. Validation raises `OBSOLETE_CODE_STRING` with a rename hint pointing to `import`. ## Local packages[​](#local-packages "Direct link to Local packages") By default, the CLI downloads packages from npm. For development or testing unpublished packages, you can use local packages instead by specifying a `path` property. ### Configuration[​](#configuration "Direct link to Configuration") Add a `path` property to any package to use a local directory instead of npm: ``` { "packages": { "@my/custom-destination": { "path": "./my-custom-destination" } } } ``` ### Resolution Rules[​](#resolution-rules "Direct link to Resolution Rules") * **`path` takes precedence** - When both `path` and `version` are specified, `path` is used * **Relative paths** - Resolved relative to the config file's directory * **Absolute paths** - Used as-is * **dist folder** - If a `dist/` folder exists, it's used; otherwise the package root is used ### Dependency Resolution[​](#dependency-resolution "Direct link to Dependency Resolution") When a local package has dependencies on other packages that are also specified with local paths, the CLI will use the local versions for those dependencies too. This prevents npm versions from overwriting your local packages. ``` { "packages": { "@walkeros/core": { "path": "../packages/core" }, "@walkeros/collector": { "path": "../packages/collector" } } } ``` In this example, even though `@walkeros/collector` depends on `@walkeros/core`, the local version of core will be used (not downloaded from npm). This is essential when testing changes across multiple interdependent packages. ### Use Cases[​](#use-cases "Direct link to Use Cases") **Development of custom packages:** ``` { "packages": { "@my-org/destination-custom": { "path": "../my-destination" } } } ``` **Testing local changes to walkerOS packages:** ``` { "packages": { "@walkeros/collector": { "path": "../../packages/collector" } } } ``` When ready for production, simply remove the `path` property to use the published npm version. ## Getting started[​](#getting-started "Direct link to Getting started") Before using the CLI, you need a [flow configuration file](/docs/getting-started/modes/bundled.md). Here's a minimal example: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-source-express": {}, "@walkeros/destination-demo": {} } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "settings": { "path": "/collect", "port": 8080 } } } }, "destinations": { "console": { "package": "@walkeros/destination-demo", "config": { "settings": { "name": "Console Logger" } } } }, "collector": { "run": true } } } } ``` Implicit collector The `@walkeros/collector` package is automatically added when your flow has sources or destinations. Save this as `flow.json`. ## Bundle command[​](#bundle-command "Direct link to Bundle command") The `bundle` command builds production-ready JavaScript bundles from flow configurations. ### Use Case[​](#use-case "Direct link to Use Case") You've defined your sources, destinations, and transformations in a flow configuration file. Now you need to: * Download the required npm packages * Bundle everything into a single optimized JavaScript file * Deploy it to production (Docker, Cloud Run, serverless functions) The bundle command handles all of this. ### Basic Usage[​](#basic-usage "Direct link to Basic Usage") ``` walkeros bundle flow.json ``` This writes the optimized bundle to stdout (web flows only). Use `-o` to write to a file or directory (e.g., `-o ./dist/walker.js` for web, `-o ./dist/` for server, which produces `dist/{flow.mjs, package.json, node_modules/}`). For server flows you can also write a `.tar.gz`/`.tgz` archive (e.g., `-o ./flow.tar.gz`), which packs the server bundle directory (`flow.mjs`, `package.json`, `node_modules/`) into a single gzip file. Web single-file bundles do not support archive output. ### Step-by-Step Guide[​](#step-by-step-guide "Direct link to Step-by-Step Guide") **1. Create a flow configuration** Create `server-collect.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-source-express": {}, "@walkeros/destination-demo": {} } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "settings": { "path": "/collect", "port": 8080, "cors": true } } } }, "destinations": { "console": { "package": "@walkeros/destination-demo", "config": { "settings": { "name": "Event Logger", "values": ["name", "data.title", "timestamp"] } } } }, "collector": { "run": true } } } } ``` **2. Bundle the flow** ``` walkeros bundle server-collect.json --stats ``` Output: ``` 📦 Downloading packages from npm... ✓ @walkeros/collector@latest ✓ @walkeros/server-source-express@latest ✓ @walkeros/destination-demo@latest 🔨 Bundling... ✓ Bundle created: ./dist/flow.mjs 📊 Bundle Statistics: Size: 45.2 KB (minified) Packages: 3 Format: ESM ``` **3. Review the bundle** ``` ls -lh dist/ # -rw-r--r-- 1 user staff 45K flow.mjs # -rw-r--r-- 1 user staff 152B package.json # drwxr-xr-x 12 user staff 384B node_modules file dist/flow.mjs # dist/flow.mjs: JavaScript source, UTF-8 Unicode text ``` The bundle is now ready to deploy! ### Options[​](#options "Direct link to Options") ``` walkeros bundle [options] ``` | Option | Description | | --------------------- | ------------------------------------------------- | | `-o, --output ` | Write bundle to file, directory, or presigned URL | | `-f, --flow ` | Build specific flow (for multi-flow configs) | | `--all` | Build all flows | | `-s, --stats` | Show bundle statistics | | `--json` | Output statistics as JSON (for CI/CD) | | `--no-cache` | Skip package cache, download fresh | | `-v, --verbose` | Detailed logging | When `--output` is a URL (e.g., a presigned S3 URL), the CLI bundles to a temp file and uploads via HTTP PUT. This is used by the walkerOS cloud service for remote builds. For server flows, `-o` resolves to a directory: `dist/{flow.mjs, package.json, node_modules/}`. When `-o` ends in `.tar.gz` or `.tgz`, that same directory is packed into a single gzip archive instead (server flows only). For multi-flow `--all`, each flow gets its own subdirectory: `dist//...`. ### Multi-Flow Example[​](#multi-flow-example "Direct link to Multi-Flow Example") ``` # Bundle development flow walkeros bundle config.json --flow development # Bundle production with stats walkeros bundle config.json --flow production --stats # Bundle all flows walkeros bundle config.json --all ``` ## Push command[​](#push-command "Direct link to Push command") The `push` command executes your flow with a real event. By default it makes actual API calls to your configured destinations. Use `--simulate` to mock specific steps for safe testing. It accepts either a config JSON (which gets bundled) or a pre-built bundle. ### Use Case[​](#use-case-1 "Direct link to Use Case") You want to: * Test event processing with mocked destinations (`--simulate`) * Test with real third-party APIs (GA4, Meta, BigQuery, etc.) * Verify production credentials and endpoints work * Debug actual API responses and errors * Perform integration testing before deployment * Execute a pre-built bundle without rebuilding Push handles both safe local testing (with `--simulate`) and real integration testing. ### Basic Usage[​](#basic-usage-1 "Direct link to Basic Usage") ``` # With config JSON (auto-bundled) walkeros push flow.json --event '{"name":"page view","data":{"title":"Home"}}' # With pre-built bundle walkeros push dist/flow.mjs --event '{"name":"page view"}' ``` ### Step-by-Step Guide[​](#step-by-step-guide-1 "Direct link to Step-by-Step Guide") **1. Create a flow configuration** Create `api-flow.json` with a destination that makes real HTTP calls: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/web-destination-api": {} } } }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "https://your-endpoint.com/events", "method": "POST" } } } }, "collector": { "run": true } } } } ``` **2. Create an event file** Create `event.json`: ``` { "name": "order complete", "data": { "id": "ORD-12345", "total": 299.99, "currency": "USD" }, "user": { "id": "user_abc123" } } ``` **3. Push the event** ``` walkeros push api-flow.json --event event.json --verbose ``` **4. Review the output** ``` 📥 Loading event... 📦 Loading flow configuration... 🔨 Bundling flow configuration... 🖥️ Executing in server environment (Node.js)... Pushing event: order complete ✅ Event pushed successfully Event ID: 1701234567890-abc12-1 Entity: order Action: complete Duration: 1234ms ``` The event was sent to your real API endpoint! ### Options[​](#options-1 "Direct link to Options") ``` walkeros push [options] ``` | Option | Description | | --------------------------- | -------------------------------------------------------------------------------------------------------------- | | `-e, --event ` | **Required.** Event to push (JSON string, file path, or URL) | | `-f, --flow ` | Flow name (for multi-flow configs) | | `-p, --platform ` | Platform override (`web` or `server`) | | `--simulate ` | Simulate a step (repeatable). Mocks the step's push, captures result. Use `destination.NAME` or `source.NAME`. | | `--mock ` | Mock a step with a specific return value (repeatable). Use `destination.NAME=VALUE`. | | `--snapshot ` | JS file to eval before execution. Sets global state (`window.dataLayer`, `process.env`, etc.). | | `-o, --output ` | Write result to file | | `--json` | Output results as JSON | | `-v, --verbose` | Verbose output with debug information | | `-s, --silent` | Suppress output (for CI/CD) | ### Input Types[​](#input-types "Direct link to Input Types") The CLI auto-detects the input type by attempting to parse as JSON: * **Config JSON** - Bundled and executed * **Pre-built bundle** (`.js`/`.mjs`) - Executed directly When using pre-built bundles, platform is detected from file extension: * `.mjs` → server (ESM, Node.js) * `.js` → web (IIFE, JSDOM) Use `--platform` to override if extension doesn't match intended runtime. ### Event Input Formats[​](#event-input-formats "Direct link to Event Input Formats") The `--event` parameter accepts three formats: **Inline JSON string:** ``` walkeros push flow.json --event '{"name":"page view","data":{"title":"Home"}}' ``` **File path:** ``` walkeros push flow.json --event ./events/order.json ``` **URL:** ``` walkeros push flow.json --event https://example.com/sample-event.json ``` ### Push modes[​](#push-modes "Direct link to Push modes") | Mode | Flag | API Calls | Use Case | | -------- | ------------ | ------------------ | ------------------------------------------ | | Real | (none) | Real HTTP requests | Integration testing, production validation | | Simulate | `--simulate` | Mocked (captured) | Safe local testing | | Mock | `--mock` | Returns mock value | Controlled testing | **Recommended workflow:** * Use `push --simulate` first to validate configuration without side effects * Use `push` (without flags) to verify real integrations work before deployment ### JSON Output[​](#json-output "Direct link to JSON Output") For CI/CD pipelines, use `--json` for machine-readable output: ``` walkeros push flow.json --event '{"name":"page view"}' --json --silent ``` Output: ``` { "success": true, "event": { "id": "1701234567890-abc12-1", "name": "page view", "entity": "page", "action": "view" }, "duration": 1234 } ``` On error: ``` { "success": false, "error": "Connection refused: https://your-endpoint.com/events", "duration": 5023 } ``` ### Multi-Flow[​](#multi-flow "Direct link to Multi-Flow") Push to specific flows: ``` # Push to staging walkeros push flow.json --event event.json --flow staging # Push to production walkeros push flow.json --event event.json --flow production ``` ## Setup command[​](#setup-command "Direct link to Setup command") The `setup` command runs the optional `setup()` lifecycle on a single component to provision external resources before traffic flows. Use it for one-time provisioning steps like creating a BigQuery dataset and table, declaring a Pub/Sub topic and subscription, creating SQLite tables, or registering a webhook with a third-party API. For the conceptual overview of the lifecycle (when to implement `setup()` in a component, how it relates to the runtime hot path, and the operator separation rationale), see [Setup lifecycle](/docs/destinations/create-your-own.md#setup-lifecycle-optional). ### Use case[​](#use-case-2 "Direct link to Use case") Setup is for the operator-time, "do this once before traffic" step that sits between writing config and accepting events: * A BigQuery destination needs its dataset and table to exist with the right schema before rows can be streamed in. * A Pub/Sub source needs a subscription against a topic before it can pull messages. * A SQLite store needs its tables created before reads and writes work. * A webhook integration needs to register its callback URL with the upstream service. Setup is **explicit only**. It is never triggered by `push`, `simulate`, `deploy`, or the long-running runtime. You run it once when provisioning, then again only when external resources need to be re-provisioned (for example, after changing the table schema in config). The runtime never tries to "fix" missing resources on its own. ### Basic usage[​](#basic-usage-2 "Direct link to Basic usage") ``` # Provision the BigQuery dataset and table for the destination named "bigquery" walkeros setup destination.bigquery ``` ### Target syntax[​](#target-syntax "Direct link to Target syntax") The target uses the same `.` syntax as `walkeros push --simulate`. Valid kinds are `source`, `destination`, and `store`. Transformers are pure functions and have no setup. | Target | Resolves to | | ---------------------- | ----------------------------------------------------- | | `destination.bigquery` | The destination named `bigquery` in the resolved flow | | `source.events-in` | The source named `events-in` | | `store.session` | The store named `session` | ### Options[​](#options-2 "Direct link to Options") ``` walkeros setup [options] ``` | Option | Description | | --------------------- | ----------------------------------------------------- | | `-c, --config ` | Flow config file (default: `./flow.json`) | | `-f, --flow ` | Flow name for multi-flow configs | | `--json` | Output as JSON (passed to the package's setup logger) | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ### What "skip" means[​](#what-skip-means "Direct link to What \"skip\" means") The framework narrates one of three reasons when nothing runs and exits successfully: | Reason | Trigger | | ----------------------- | -------------------------------------------------------------------- | | `no setup function` | The package does not export a `setup` function on its default export | | `config.setup is false` | The flow config explicitly opts out with `"setup": false` | | `config.setup is unset` | The flow config does not declare `setup` at all | In all three cases the exit code is `0`. Setup is a single-purpose command, not a tiered status check. ### Result output[​](#result-output "Direct link to Result output") When the package's `setup()` returns a non-undefined value, the CLI emits it as JSON on stdout for `jq`-style scripting. This lets you script around what was provisioned: ``` walkeros setup destination.bigquery --json | jq .datasetCreated ``` If the package returns nothing, no JSON is emitted; you only see the framework narration. ### Exit codes[​](#exit-codes "Direct link to Exit codes") | Code | Meaning | | -------- | ----------------------------------------------------------------- | | 0 | Success or skip (any of the three skip reasons above) | | non-zero | Failure (resolution error, missing default export, package threw) | ### IAM note[​](#iam-note "Direct link to IAM note") Setup typically needs higher permissions than runtime push: "create dataset" or "create subscription" vs "write rows" or "pull messages". Operators commonly run setup with a separate service account, or different credentials, than the runtime uses. Plan your IAM accordingly: grant the runtime only what it needs to push, and grant setup the broader provisioning scope. ### Examples[​](#examples "Direct link to Examples") ``` # Provision the BigQuery dataset and table for the destination named "bigquery" # in the default flow file. walkeros setup destination.bigquery # Same but pointing to a specific flow in a multi-flow config. walkeros setup destination.bigquery --flow analytics # Custom config path. walkeros setup destination.bigquery --config ./flows/prod.json # Pipe the structured result to jq. walkeros setup destination.bigquery --json | jq .datasetCreated # Provision a Pub/Sub source's subscription. walkeros setup source.events-in ``` ## Validate command[​](#validate-command "Direct link to Validate command") The `validate` command checks the structure and correctness of events, flow configurations, mapping configurations, contracts, or individual flow entries before bundling or deployment. ### Use case[​](#use-case-3 "Direct link to Use case") Before bundling or deploying your flow, you want to: * Catch configuration errors early * Verify event structure follows walkerOS conventions * Check mapping patterns are valid * Validate contracts define proper entity-action schemas * Validate a specific destination, source, or transformer entry against its package's published JSON Schema * Integrate validation into CI/CD pipelines Validate gives you fast feedback without the overhead of bundling. ### Basic usage[​](#basic-usage-3 "Direct link to Basic usage") ``` # Validate a flow configuration (schema, references, cross-step examples) walkeros validate flow.json # Validate an event structure walkeros validate event.json --type event # Validate a mapping configuration walkeros validate mapping.json --type mapping # Validate a contract walkeros validate contract.json --type contract # Validate a specific entry against its package schema walkeros validate flow.json --path destinations.snowplow ``` ### Validation types[​](#validation-types "Direct link to Validation types") The `--type` option accepts four validation types. Default is `flow`: | Type | What it validates | | ---------------- | -------------------------------------------------------------------------------------------------- | | `event` | Event structure: `name` field exists, follows "entity action" format with space, valid data types | | `flow` (default) | Flow\.Json config: schema, references, packages, and cross-step example compatibility | | `mapping` | Mapping rules: event patterns use "entity action" or wildcard format, rule structures are valid | | `contract` | Contract structure: entity-action entries map to JSON Schema objects, optional `$tagging` metadata | Use `--path` for entry validation against package schemas (e.g., `--path destinations.snowplow`). ### Step-by-step guide[​](#step-by-step-guide-2 "Direct link to Step-by-step guide") **1. Validate a flow configuration** ``` walkeros validate flow.json ``` Output (valid): ``` Validating flow... Validation Results: ✓ All checks passed Summary: 0 error(s), 0 warning(s) ``` **2. Validate with errors** ``` # Create an invalid event echo '{"name":"pageview"}' > bad-event.json walkeros validate bad-event.json --type event ``` Output: ``` Validating event... Validation Results: ✗ name: Event name must be "entity action" format with space (e.g., "page view") Summary: 1 error(s), 0 warning(s) ``` **3. Validate with warnings** ``` # Event without consent object echo '{"name":"page view","data":{"title":"Home"}}' > event.json walkeros validate event.json --type event ``` Output: ``` Validating event... Validation Results: ✓ All checks passed ⚠ consent: No consent object provided → Consider adding a consent object for GDPR/privacy compliance Summary: 0 error(s), 1 warning(s) ``` ### Options[​](#options-3 "Direct link to Options") ``` walkeros validate [input] [options] ``` | Option | Description | | --------------------- | ------------------------------------------------------------------------------------ | | `-t, --type ` | Validation type (default: `flow`). Also: `event`, `mapping`, `contract` | | `--path ` | Validate a specific entry against its package schema (e.g., `destinations.snowplow`) | | `-f, --flow ` | Flow name to validate (for multi-flow configs) | | `-o, --output ` | Write result to file | | `--strict` | Treat warnings as errors (exit code 2) | | `--json` | Output results as JSON | | `-v, --verbose` | Show detailed validation information | | `-s, --silent` | Suppress banner output | ### Exit codes[​](#exit-codes-1 "Direct link to Exit codes") | Code | Meaning | | ---- | ------------------------------------------------ | | 0 | Valid (no errors) | | 1 | Validation errors found | | 2 | Warnings found (with `--strict`) | | 3 | Input error (file not found, invalid JSON, etc.) | ### CI/CD integration[​](#cicd-integration "Direct link to CI/CD integration") Use `--json` and exit codes for automated pipelines: ``` # Fail on any error walkeros validate flow.json --json || exit 1 # Fail on warnings too (strict mode) walkeros validate flow.json --strict --json || exit 1 ``` JSON output format: ``` { "valid": true, "type": "flow", "errors": [], "warnings": [ { "path": "packages.@walkeros/destination-demo", "message": "Package \"@walkeros/destination-demo\" has no version specified", "suggestion": "Consider specifying a version for reproducible builds" } ], "details": { "flowNames": ["default"], "flowCount": 1, "packageCount": 2 } } ``` ### Multi-flow validation[​](#multi-flow-validation "Direct link to Multi-flow validation") Validate a specific flow in a multi-flow configuration: ``` # Validate only the "production" flow walkeros validate flow.json --flow production ``` If the flow doesn't exist: ``` Validation Results: ✗ flows: Flow "production" not found. Available: default, staging Summary: 1 error(s), 0 warning(s) ``` ### Event validation details[​](#event-validation-details "Direct link to Event validation details") Event validation checks (see [event.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/src/commands/validate/validators/event.ts)): 1. **Name field exists** - Required field 2. **Name is non-empty** - Cannot be empty string 3. **Entity-action format** - Must contain space (e.g., `"page view"` not `"pageview"`) 4. **Schema validation** - Data types match expected structure 5. **Best practices** - Warns if consent object is missing ``` # Valid event walkeros validate '{"name":"product view","data":{"id":"P123"}}' --type event # Invalid: missing space in name walkeros validate '{"name":"productview"}' --type event ``` ### Mapping validation details[​](#mapping-validation-details "Direct link to Mapping validation details") Mapping validation checks (see [mapping.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/src/commands/validate/validators/mapping.ts)): 1. **Object structure** - Must be an object with event patterns as keys 2. **Event patterns** - Must be `"entity action"` format or contain wildcard (`*`) 3. **Rule structure** - Each rule must be an object or array of objects 4. **Catch-all position** - Warns if `*` is not the last pattern ``` # Valid mapping walkeros validate '{ "page view": {"name": "pageview"}, "*": {"name": "generic_event"} }' --type mapping ``` ### Contract validation details[​](#contract-validation-details "Direct link to Contract validation details") Contract validation checks (see [contract.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/src/commands/validate/validators/contract.ts)): 1. **Root structure** - Must be an object (not array or primitive) 2. **`$tagging` metadata** - If present, must be a non-negative integer 3. **Entity keys** - Cannot be empty strings 4. **Action entries** - Each entity must contain an object of action entries 5. **Schema entries** - Each action value must be a JSON Schema object ``` # Valid contract walkeros validate '{ "page": { "view": { "type": "object", "properties": { "title": { "type": "string" } } } } }' --type contract ``` ### Entry validation (`--path`)[​](#entry-validation---path "Direct link to entry-validation---path") Entry validation checks a specific destination, source, or transformer in your flow config against the package's published JSON Schema. The CLI fetches the schema from the package on the CDN and validates the `config.settings` object using AJV. ``` # Validate a specific destination entry walkeros validate flow.json --path destinations.snowplow # Validate a specific source entry walkeros validate flow.json --path sources.browser # Short form (works if the key is unique across sections) walkeros validate flow.json --path snowplow ``` The entry validator: 1. Resolves the entry from the flow config (first flow is used) 2. Reads the `package` field to identify the npm package 3. Fetches the package's JSON Schema from the CDN 4. Validates the entry's `config.settings` against the schema ## Auth commands[​](#auth-commands "Direct link to Auth commands") The `auth` command group manages authentication with the walkerOS cloud service. Authentication is required for cloud commands (`projects`, `flows`, `deploy`). ### Login[​](#login "Direct link to Login") Log in to walkerOS via an OAuth browser flow. The CLI requests a device code, opens your browser for authorization, and polls for the resulting token. ``` walkeros auth login ``` Output: ``` ! Your one-time code: ABCD-1234 Authorize here: https://app.walkeros.io/auth/device?code=ABCD-1234 Opening browser... Waiting for authorization... (press Ctrl+C to cancel) ✓ Logged in as user@example.com Token stored in ~/.config/walkeros/config.json ``` The token is stored in `~/.config/walkeros/config.json` with `0600` permissions. You can also set the `WALKEROS_TOKEN` environment variable instead of using `auth login`. | Option | Description | | --------------- | --------------------------------------------------- | | `--url ` | Custom app URL (default: `https://app.walkeros.io`) | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ### Logout[​](#logout "Direct link to Logout") Remove stored credentials from disk. ``` walkeros auth logout ``` | Option | Description | | --------------- | --------------- | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ### Whoami[​](#whoami "Direct link to Whoami") Show the current authenticated user's identity, including email, user ID, and project ID (if the token is project-scoped). ``` walkeros auth whoami ``` Output: ``` user@example.com User: usr_abc123 Project: proj_def456 ``` | Option | Description | | --------------------- | -------------------- | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ### Token resolution[​](#token-resolution "Direct link to Token resolution") The CLI resolves authentication tokens in this order: 1. `WALKEROS_TOKEN` environment variable 2. Config file (`~/.config/walkeros/config.json`, written by `auth login`) 3. Not authenticated (cloud commands will fail) ## Projects commands[​](#projects-commands "Direct link to Projects commands") The `projects` command group manages walkerOS cloud projects. All commands require authentication (see [Auth commands](#auth-commands)). ### List projects[​](#list-projects "Direct link to List projects") ``` walkeros projects list ``` | Option | Description | | -------------- | ---------------------------------------------------------------- | | `--cursor ` | Resume from a pagination cursor returned by a previous list call | | `--limit ` | Maximum number of projects to return | ### Get project details[​](#get-project-details "Direct link to Get project details") ``` # Get by explicit ID walkeros projects get proj_abc123 # Uses WALKEROS_PROJECT_ID if omitted walkeros projects get ``` ### Create a project[​](#create-a-project "Direct link to Create a project") ``` walkeros projects create "My Analytics Project" ``` ### Update a project[​](#update-a-project "Direct link to Update a project") ``` walkeros projects update proj_abc123 --name "New Name" ``` ### Delete a project[​](#delete-a-project "Direct link to Delete a project") ``` walkeros projects delete proj_abc123 ``` ### Common options[​](#common-options "Direct link to Common options") All `projects` subcommands support: | Option | Description | | --------------------- | -------------------- | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ## Flows commands[​](#flows-commands "Direct link to Flows commands") The `flows` command group manages flow configurations within a project. All commands require authentication and a project context (either `--project` or `WALKEROS_PROJECT_ID`). ### List flows[​](#list-flows "Direct link to List flows") ``` walkeros flows list ``` | Option | Description | | ------------------- | ---------------------------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `--cursor ` | Resume from a pagination cursor returned by a previous list call | | `--limit ` | Maximum number of flows to return | | `--sort ` | Sort by: `name`, `updated_at`, `created_at` | | `--order ` | Sort order: `asc`, `desc` | | `--include-deleted` | Include soft-deleted flows | ### Get a flow[​](#get-a-flow "Direct link to Get a flow") Retrieves a flow with its full `Flow.Json` content. ``` walkeros flows get cfg_abc123 ``` | Option | Description | | ---------------- | ---------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | ### Create a flow[​](#create-a-flow "Direct link to Create a flow") ``` # With inline JSON walkeros flows create "My Flow" --content '{"version":4,"flows":{"default":{"config":{"platform":"server"}}}}' # From a file via stdin cat flow.json | walkeros flows create "My Flow" ``` | Option | Description | | ---------------------- | ---------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `-c, --content ` | Flow\.Json string or file path | ### Update a flow[​](#update-a-flow "Direct link to Update a flow") ``` # Update name only walkeros flows update cfg_abc123 --name "Updated Flow" # Update content walkeros flows update cfg_abc123 --content '{"version":4,"flows":{"default":{"config":{"platform":"server"}}}}' ``` | Option | Description | | ---------------------- | ---------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `--name ` | New flow name | | `-c, --content ` | New Flow\.Config JSON string or file path | ### Delete a flow[​](#delete-a-flow "Direct link to Delete a flow") Soft-deletes a flow configuration. ``` walkeros flows delete cfg_abc123 ``` ### Duplicate a flow[​](#duplicate-a-flow "Direct link to Duplicate a flow") Creates a copy of an existing flow configuration. ``` walkeros flows duplicate cfg_abc123 --name "Flow Copy" ``` | Option | Description | | ---------------- | ---------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `--name ` | Name for the copy (defaults to "Copy of ...") | ### Common options[​](#common-options-1 "Direct link to Common options") All `flows` subcommands support: | Option | Description | | --------------------- | -------------------- | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ## Deploy commands[​](#deploy-commands "Direct link to Deploy commands") The `deploy` command group handles deploying flows to the walkerOS cloud or managing self-hosted deployments with heartbeat registration. All commands require authentication. ### deploy create[​](#deploy-create "Direct link to deploy create") Create a new deployment. The CLI infers the deployment type (`web` or `server`) from the flow configuration. ``` # From a local config file walkeros deploy create flow.json # From a remote flow ID walkeros deploy create cfg_abc123 # With a specific flow name (multi-config flows) walkeros deploy create flow.json --flow production ``` On success, the CLI displays the deployment ID, slug, and type. It also shows example commands for running the deployment locally or via Docker. The create response does not include a deploy token: deploy tokens are minted separately, so the Docker instructions tell you to create one in the app (Settings, Self-hosted deploy token) and set it as `WALKEROS_DEPLOY_TOKEN`. | Option | Description | | --------------------- | ---------------------------------------------- | | `--label ` | Human-readable label for the deployment | | `-f, --flow ` | Flow name for multi-flow configs | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ### deploy start[​](#deploy-start "Direct link to deploy start") Deploy a remote flow to walkerOS cloud infrastructure. Auto-detects whether to use web (script hosting) or server (container) deployment based on the flow content. Streams deployment progress via SSE. ``` # Deploy a flow walkeros deploy start cfg_abc123 # Deploy a specific config in a multi-config flow walkeros deploy start cfg_abc123 --flow production # Deploy without waiting for completion walkeros deploy start cfg_abc123 --no-wait ``` Output (web deployment): ``` Building bundle... Publishing to web... ✓ Published: https://cdn.walkeros.io/proj_xxx/walker.js ``` Output (server deployment): ``` Building bundle... Deploying container... Starting container... ✓ Active: https://collect-abc123.walkeros.io ``` | Option | Description | | --------------------- | ----------------------------------------------------------------------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `-f, --flow ` | Flow name for multi-config flows | | `--no-wait` | Return immediately after triggering (do not stream progress) | | `--timeout ` | Override the client-side wait budget (defaults to 12 minutes, which covers the server's full deploy budget) | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | By default `deploy start` streams progress until the deploy completes. The client-side wait budget defaults to 12 minutes so a slow but healthy long deploy is never aborted early and misread as a failure in CI; `--timeout` overrides it. Each invocation sends a fresh idempotency key, so retrying after a failed attempt starts a new deploy instead of replaying the previous result. When a deploy fails, the CLI prints a stable, machine-readable `error.code` alongside the message so CI can branch on the code without matching prose. A rate-limited trigger (`429`) is reported as retryable with the `Retry-After` hint; under `--wait` the CLI honors one bounded retry automatically. ### deploy list[​](#deploy-list "Direct link to deploy list") List all deployments in a project. ``` walkeros deploy list # Filter by type walkeros deploy list --type server # Filter by status walkeros deploy list --status active ``` | Option | Description | | ------------------- | ---------------------------------------------------------------- | | `--project ` | Project ID (defaults to `WALKEROS_PROJECT_ID`) | | `--cursor ` | Resume from a pagination cursor returned by a previous list call | | `--limit ` | Maximum number of deployments to return | | `--type ` | Filter by type: `web`, `server` | | `--status ` | Filter by status | ### deploy status[​](#deploy-status "Direct link to deploy status") Get deployment details by ID or slug. ``` walkeros deploy status dep_abc123 # Or by slug walkeros deploy status my-collector ``` | Option | Description | | ---------------- | ----------- | | `--project ` | Project ID | ### deploy delete[​](#deploy-delete "Direct link to deploy delete") Delete a deployment by ID or slug. ``` walkeros deploy delete dep_abc123 ``` | Option | Description | | ---------------- | -------------- | | `--project ` | Project ID | | `--json` | Output as JSON | ### Common options[​](#common-options-2 "Direct link to Common options") All `deploy` subcommands support: | Option | Description | | --------------------- | -------------------- | | `-o, --output ` | Write result to file | | `--json` | Output as JSON | | `-v, --verbose` | Verbose output | | `-s, --silent` | Suppress output | ## Previews command[​](#previews-command "Direct link to Previews command") The `previews` command group manages preview bundles: short-lived flow bundles used to test configuration changes on a real production site before deploying. Creating a preview mints an app-signed, origin-bound activation grant; opening `?elbPreview={grant}` on your site verifies the grant locally, with no network round trip, and swaps in the preview bundle for that browser for the life of the preview session. Activation links minted for an Observe session additionally carry an `elbPreviewSession` companion grant, which the previewed page uses to forward server-bound events to that session's isolated container. All commands require authentication and a project (set via `WALKEROS_PROJECT_ID` or `--project`). ### How activation works[​](#how-activation-works "Direct link to How activation works") A managed web bundle that supports preview activation is produced by the wrap step's `preview` option, which bakes in a public key keyring, the issuing environment, and an opaque per-project binding (never a project ID or secret). The emitted bundle imports `browserSwapActivator` from `@walkeros/core` to verify an activation grant and swap in the preview artifact. `preview` is the current wrap option for this; it replaces the earlier `previewOrigin` / `previewScope` options. A companion `previewGrantTargets` option exists for the preview artifact itself: it forwards the stored grant as an `X-Walkeros-Preview` header to named server-bound destinations, so a server flow can be previewed too. `preview` and `previewGrantTargets` are mutually exclusive on a single wrap invocation: a host bundle activates a preview, a preview artifact injects its grant, not both. ### previews list[​](#previews-list "Direct link to previews list") List all previews for a flow. ``` walkeros previews list [--project ] ``` | Option | Description | | ---------------- | ------------------------------ | | `--project ` | Project ID (overrides default) | ### previews get[​](#previews-get "Direct link to previews get") Get details for a single preview. ``` walkeros previews get [--project ] ``` | Option | Description | | ---------------- | ------------------------------ | | `--project ` | Project ID (overrides default) | ### previews create[​](#previews-create "Direct link to previews create") Create a preview bundle for a specific flow settings entry. ``` walkeros previews create [options] ``` | Option | Description | | ------------------------ | -------------------------------------------------------------- | | `-f, --flow ` | Flow settings name (resolved to an ID) | | `-s, --settings-id ` | Flow settings ID (alternative to `--flow`) | | `-u, --url ` | Your site URL; prints a ready-to-open activation URL on stdout | | `--project ` | Project ID (overrides default) | Without `--url`, the CLI prints the activation fragment (`?elbPreview=...`) to stdout: append it to any URL on your site to activate preview mode. With `--url`, it prints an activation URL for your site on stdout, plus a deactivation URL on stderr. Most modern terminals make the URL clickable. You must provide either `--flow` or `--settings-id` to pick which flow settings the preview should bundle. ### previews delete[​](#previews-delete "Direct link to previews delete") Delete a preview. Removes both the DB row and the S3 bundle immediately; the production walker on visitors' browsers self-heals on the next page load by clearing the stored `elbPreview` grant and loading the production flow. ``` walkeros previews delete [options] ``` | Option | Description | | ---------------- | ----------------------------------------------------- | | `-y, --yes` | Skip confirmation (required to run non-interactively) | | `--project ` | Project ID (overrides default) | ### Example[​](#example "Direct link to Example") ``` # Create a preview for the `demo` flow settings with a ready-to-open URL walkeros previews create flow_abc123 \ --flow demo \ --url https://example.com # List previews for a flow walkeros previews list flow_abc123 # Delete when you're done walkeros previews delete flow_abc123 prv_xyz456 --yes ``` ## Run command[​](#run-command "Direct link to Run command") The `run` command starts an HTTP server that accepts events and processes them through your flow. ### Use Case[​](#use-case-4 "Direct link to Use Case") You need an HTTP endpoint to: * Receive events from browser clients, mobile apps, or server-side sources * Process events through your collector and destinations * Test the full event pipeline locally before deploying to production This is similar to running a Segment or Jitsu collection endpoint. ### Basic Usage[​](#basic-usage-4 "Direct link to Basic Usage") ``` walkeros run flow.json --port 8080 ``` ### Step-by-Step Guide[​](#step-by-step-guide-3 "Direct link to Step-by-Step Guide") **1. Create a collection flow** Create `collect.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-source-express": {}, "@walkeros/destination-demo": {} } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "settings": { "path": "/collect", "port": 8080, "cors": true } } } }, "destinations": { "console": { "package": "@walkeros/destination-demo", "config": { "settings": { "name": "Event Collector", "values": ["name", "data", "user.id", "timestamp"] } } } }, "collector": { "run": true, "globals": { "environment": "development" } } } } } ``` **2. Start the collector** ``` walkeros run flow.json ``` Output: ``` 📦 Bundling flow... ✓ Bundle ready 🚀 Starting collection server... ✓ Server running on http://localhost:8080 ✓ Endpoint: POST http://localhost:8080/collect ✓ Health check: GET http://localhost:8080/health ``` **3. Send test events** Open a new terminal and send events: ``` # Page view event curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Home Page", "path": "/" }, "user": { "id": "user123" } }' # Product view event curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{ "name": "product view", "data": { "id": "P123", "name": "Laptop", "price": 999 } }' ``` **4. See events in console** The collector terminal shows: ``` [Event Collector] page view data: {"title":"Home Page","path":"/"} user.id: user123 timestamp: 1701234567890 [Event Collector] product view data: {"id":"P123","name":"Laptop","price":999} timestamp: 1701234567891 ``` ### Options[​](#options-4 "Direct link to Options") ``` walkeros run [options] ``` The input can be a flow config, a pre-built bundle, or a `.tar.gz`/`.tgz` flow archive (local file or URL). | Option | Description | | -------------------------------- | ----------------------------------------------------------- | | `-p, --port ` | Server port (default: 8080) | | `-h, --host
` | Host address (default: 0.0.0.0) | | `--deploy ` | Deployment ID or slug (enables heartbeat to walkerOS cloud) | | `--project ` | Project ID (used with `--deploy`) | | `--url ` | Public URL of this server (used with `--deploy`) | | `--health-endpoint ` | Health check path (default: `/health`) | | `--heartbeat-interval ` | Heartbeat interval in seconds (default: 60) | | `--json` | Output as JSON | | `-v, --verbose` | Detailed logging | | `-s, --silent` | Suppress output | Heartbeat registration When `--deploy` is provided, the collector registers itself with the walkerOS cloud via periodic heartbeats. This makes the running instance visible in the project dashboard and enables remote management. ``` walkeros run flow.json \ --deploy dep_abc123 \ --project proj_def456 \ --url https://collect.example.com \ --heartbeat-interval 30 ``` The heartbeat sends instance ID, uptime, CLI version, and mode. The server can respond with `update` (triggering a bundle refresh) or `stop` (graceful shutdown). ### Running Pre-Built Bundles[​](#running-pre-built-bundles "Direct link to Running Pre-Built Bundles") You can also run pre-built bundles directly: ``` # First, bundle walkeros bundle collect.json # Then run the bundle walkeros run dist/flow.mjs --port 8080 ``` ### Running flow archives[​](#running-flow-archives "Direct link to Running flow archives") `run` also accepts a `.tar.gz`/`.tgz` flow archive, either a local file or a URL. The CLI fetches or reads the gzip, extracts the bundle and its sibling `node_modules/`, and runs the entry. This lets server flows whose step packages are external resolve those packages at runtime from the extracted `node_modules/`. ``` # Bundle to an archive walkeros bundle collect.json -o flow.tar.gz # Run a local archive walkeros run flow.tar.gz --port 8080 # Run an archive from a URL walkeros run https://example.com/flow.tar.gz ``` ## Complete example: Web → Server flow[​](#complete-example-web--server-flow "Direct link to Complete example: Web → Server flow") This example demonstrates a complete analytics pipeline: * Browser events captured by web flow * Sent to server collection endpoint * Logged to console (swap for BigQuery in production) ### 1. Create Server Collection Flow[​](#1-create-server-collection-flow "Direct link to 1. Create Server Collection Flow") Create `server-collect.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-source-express": {}, "@walkeros/destination-demo": {} } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "settings": { "path": "/collect", "port": 8080, "cors": true } } } }, "destinations": { "console": { "package": "@walkeros/destination-demo", "config": { "settings": { "name": "Server Logger" } } } }, "collector": { "run": true } } } } ``` ### 2. Create Web Tracking Flow[​](#2-create-web-tracking-flow "Direct link to 2. Create Web Tracking Flow") Create `web-track.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "web", "bundle": { "packages": { "@walkeros/web-source-browser": {}, "@walkeros/web-destination-api": {} } } }, "sources": { "browser": { "package": "@walkeros/web-source-browser", "config": { "settings": { "pageview": true, "elb": "track" } } } }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "http://localhost:8080/collect", "method": "POST" } } } }, "collector": { "run": true } } } } ``` ### 3. Start Collection Server[​](#3-start-collection-server "Direct link to 3. Start Collection Server") Terminal 1: ``` walkeros run server-collect.json ``` ### 4. Start Web Server[​](#4-start-web-server "Direct link to 4. Start Web Server") Terminal 2: ``` walkeros run web-track.json --port 3000 ``` ### 5. Test in Browser[​](#5-test-in-browser "Direct link to 5. Test in Browser") Create `demo.html`: ```

Welcome to walkerOS

``` Open in browser. Terminal 1 shows: ``` [Server Logger] page view [Server Logger] promotion view [Server Logger] promotion cta [Server Logger] custom event ``` ## Cache command[​](#cache-command "Direct link to Cache command") The `cache` command manages the CLI's package and build caches. ### Use Case[​](#use-case-5 "Direct link to Use Case") The CLI caches downloaded npm packages and compiled builds to speed up repeated operations. You may need to: * Clear stale cached packages when debugging version issues * Free up disk space by removing old cached builds * View cache statistics to understand cache usage ### How Caching Works[​](#how-caching-works "Direct link to How Caching Works") **Package Cache** (`.tmp/cache/packages/`): * Mutable versions (`latest`, `^`, `~`) are re-checked daily * Exact versions (`0.4.1`) are cached indefinitely * Saves network time on repeated builds **Build Cache** (`.tmp/cache/builds/`): * Caches compiled bundles based on flow\.json content + date * Identical configs reuse cached builds within the same day * Dramatically speeds up repeated builds (\~100x faster) ### Basic Usage[​](#basic-usage-5 "Direct link to Basic Usage") ``` # View cache statistics walkeros cache info # Clear all caches walkeros cache clear ``` ### Commands[​](#commands "Direct link to Commands") **View cache info:** ``` walkeros cache info ``` Output: ``` Cache directory: .tmp/cache Cached packages: 12 Cached builds: 5 ``` **Clear all caches:** ``` walkeros cache clear ``` **Clear only package cache:** ``` walkeros cache clear --packages ``` **Clear only build cache:** ``` walkeros cache clear --builds ``` ### Bypassing Cache[​](#bypassing-cache "Direct link to Bypassing Cache") To skip the cache for a single build operation: ``` walkeros bundle flow.json --no-cache ``` This downloads fresh packages and rebuilds without using or updating the cache. ### Options[​](#options-5 "Direct link to Options") | Option | Description | | ------------ | ---------------------------- | | `--packages` | Clear only the package cache | | `--builds` | Clear only the build cache | ## Global options[​](#global-options "Direct link to Global options") These options work with all commands: ``` walkeros [command] [options] ``` | Option | Description | | ----------- | -------------------------- | | `--verbose` | Show detailed logs | | `--silent` | Suppress output | | `--json` | Output as JSON (for CI/CD) | | `--help` | Show help for command | | `--version` | Show CLI version | ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Purpose | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `WALKEROS_TOKEN` | API token for cloud commands (overrides `auth login` config) | | `WALKEROS_PROJECT_ID` | Default project ID for `projects`, `flows`, and `deploy` commands | | `WALKEROS_APP_URL` | Base URL override (default: `https://app.walkeros.io`) | | `WALKEROS_DEPLOY_TOKEN` | Deploy token for container heartbeat authentication | | `WALKEROS_CLIENT_TYPE` | Override client identity sent to the app. Defaults to `cli`; set to `runner` for long-lived flow runners. Used by the runtime image. | | `WALKEROS_OBSERVE_LEVEL` | Baseline telemetry level for `walkeros run` (`off`, `standard`, or `trace`). Invalid values are warned about and ignored. | | `WALKEROS_CONFIG_FROZEN` | Set to `1` or `true` to make `walkeros run` serve the bundle as an immutable snapshot: secrets are still injected at boot, but config hot-swap and heartbeat are disabled. | ## API version compatibility[​](#api-version-compatibility "Direct link to API version compatibility") Official clients (CLI, MCP, runner image) send three headers on every request to the walkerOS app so the server can tell them apart and enforce minimum versions per endpoint: * `User-Agent`: e.g. `walkeros-cli/1.4.0 (node/18.19.0; linux)` * `X-WalkerOS-Client`: `cli`, `mcp`, or `runner` * `X-WalkerOS-Client-Version`: the installed package version If you call an endpoint that has been updated in a way that requires a newer client, the app responds with `426 Upgrade Required`. The CLI prints the required minimum version and the upgrade command, then exits with code 2: ``` Error: This walkerOS app endpoint requires @walkeros/cli >= 1.5.0 (you have 1.4.0). Upgrade with: npm install -g @walkeros/cli@latest See https://walkeros.io/docs/upgrading for details. ``` The MCP server surfaces the same information in tool error responses so AI assistants can tell the user to upgrade. See [Upgrading](/docs/upgrading.md) for the full version-negotiation rules. Set `WALKEROS_CLIENT_TYPE=runner` to identify the CLI binary as a long-lived flow runner instead of an interactive CLI session. The official runner Docker image sets this automatically. ## CI/CD integration[​](#cicd-integration-1 "Direct link to CI/CD integration") ### GitHub Actions[​](#github-actions "Direct link to GitHub Actions") ``` name: Build Flows on: [push] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: '18' - name: Install CLI run: npm install -g @walkeros/cli - name: Bundle flows run: walkeros bundle flow.json --all --stats --json > stats.json - name: Upload bundles uses: actions/upload-artifact@v3 with: name: flows path: dist/ ``` ### Docker Build[​](#docker-build "Direct link to Docker Build") The canonical multi-stage Dockerfile installs the CLI in the build stage, bundles the flow into a `dist/` directory, then copies the directory into the runtime image. You do not need to declare step packages in `package.json`; pacote installs them transparently from `flow.json`. ``` FROM node:22.23.0-alpine AS builder WORKDIR /build RUN npm init -y && npm install --save-dev @walkeros/cli COPY flow.json ./ RUN npx walkeros bundle flow.json -o dist/ FROM walkeros/flow:4 WORKDIR /app/flow COPY --from=builder /build/dist/ ./ ENV PORT=8080 EXPOSE 8080 ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Package Download Issues[​](#package-download-issues "Direct link to Package Download Issues") If packages fail to download: ``` # Clear cache and retry walkeros bundle flow.json --no-cache # Check npm registry access npm ping ``` ### Build Issues[​](#build-issues "Direct link to Build Issues") If you encounter build issues: ``` # Clear cache and retry walkeros cache clear # Retry with verbose output walkeros bundle flow.json --verbose ``` ### Port Already in Use[​](#port-already-in-use "Direct link to Port Already in Use") If the port is already in use: ``` # Use a different port walkeros run flow.json --port 8081 # Or kill the process using the port lsof -ti:8080 | xargs kill ``` ### Authentication Issues[​](#authentication-issues "Direct link to Authentication Issues") If cloud commands fail with authentication errors: ``` # Check current identity walkeros auth whoami # Re-authenticate walkeros auth login # Or set token directly export WALKEROS_TOKEN=sk-walkeros-xxx ``` ### nft cannot trace a runtime asset[​](#nft-cannot-trace-a-runtime-asset "Direct link to nft cannot trace a runtime asset") If a runtime asset is missing in `dist/node_modules/` (typically because the dep loads it via a dynamically constructed path), declare it explicitly under the flow's `config.bundle.traceInclude`. Paths and globs both work, resolved against the install root: ``` "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-destination-gcp": {} }, "traceInclude": [ "node_modules/some-pkg/data/*.json" ] } } } } ``` ### Caching pacote downloads in CI[​](#caching-pacote-downloads-in-ci "Direct link to Caching pacote downloads in CI") The bundler caches downloads under `process.env.NPM_CACHE_DIR` (default `/cache/npm`). On CI, persist that path with `actions/cache`: ``` - uses: actions/cache@v4 with: path: .walkeros-cache/npm key: walkeros-${{ hashFiles('**/flow.json') }} - run: WALKEROS_TMP_DIR=.walkeros-cache npx walkeros bundle flow.json -o dist/ ``` ### Bundle output is large[​](#bundle-output-is-large "Direct link to Bundle output is large") `dist/node_modules/` for a server flow with the GCP destination typically holds 30-50MB across 10k+ files (grpc, protobufjs, etc.). Recommendations: * Use `.dockerignore` to exclude irrelevant build output from non-bundle contexts. * Prefer `COPY --from=builder /build/dist/ /app/flow/` over file-by-file in your Dockerfile. * Expect \~30s artifact upload time on slow CI. ## Next steps[​](#next-steps "Direct link to Next steps") * **[Flow Configuration](/docs/getting-started/modes/bundled.md)** - Learn about flow config structure * **[Docker Deployment](/docs/apps/docker.md)** - Deploy flows to production * **[Runner](/docs/apps/runner.md)** - Self-hosted runner with config polling and hot-swap * **[MCP Server](/docs/apps/mcp.md)** - Use CLI tools from AI assistants * **[Sources](/docs/sources/.md)** - Explore event sources * **[Destinations](/docs/destinations/.md)** - Configure analytics tools * **[Mapping](/docs/mapping/.md)** - Transform events for destinations *** ## See Also[​](#see-also "Direct link to See Also") **Using Integrated mode instead?** The CLI uses JSON configuration (Bundled mode). If you prefer TypeScript and want the collector built into your application, see [Integrated Mode](/docs/getting-started/modes/integrated.md) and the [Collector documentation](/docs/collector/.md). Both approaches use the same underlying architecture. The difference is how you configure and deploy. --- # walkerOS Docker The walkerOS Docker image (`walkeros/flow`) is a **pure runtime container** for executing pre-built flow bundles in production. It's designed for fast startup (< 1 second), minimal footprint (\~150-200MB), and cloud-native deployment. **Key philosophy**: Docker handles runtime, [CLI](/docs/apps/cli.md) handles build-time. You bundle with the CLI, deploy with Docker. ## What it does[​](#what-it-does "Direct link to What it does") * ✅ **Executes pre-built `.mjs` bundles** from the CLI * ✅ **Runs collection servers** (HTTP endpoints for event ingestion) * ✅ **Serves static files** (web tracking scripts) * ✅ **Sub-1 second startup** - No npm downloads, no build steps * ✅ **Production-ready** - Health checks, non-root user, signal handling ## What it doesn't do[​](#what-it-doesnt-do "Direct link to What it doesn't do") * ❌ **No bundling** - Flows must be pre-built with the CLI * ❌ **No package downloads** - All dependencies must be bundled * ❌ **No configuration generation** - Configuration comes from your bundle Think of it as the **deployment target**, not the build tool. ## Build → deploy workflow[​](#build--deploy-workflow "Direct link to Build → deploy workflow") ``` ┌─────────────────────────────────────────────────────────────────┐ │ DEVELOPMENT │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ flow.json ──► walkeros bundle ──► flow.mjs │ │ │ (CLI) │ │ │ │ │ │ │ Config as code Pre-built bundle │ │ (sources, destinations) (all deps included) │ │ │ └────────────────────────────────┬────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ PRODUCTION │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ flow.mjs ──► Docker Container ──► HTTP Server │ │ │ (runtime only) │ │ │ │ │ │ │ Mount or bake in < 1 second startup │ │ /health, /collect │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` This separation means: * **CLI** handles complexity (npm resolution, bundling, TypeScript) * **Docker** stays simple (just runs JavaScript) * **Startup is instant** (no npm install, no build step) ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Docker installed ([Get Docker](https://docs.docker.com/get-docker/)) * (Optional) Pre-built flow bundle from the [CLI](/docs/apps/cli.md) ## Quick start with demos[​](#quick-start-with-demos "Direct link to Quick start with demos") The Docker image includes demo bundles for instant testing. ### Demo: event collection[​](#demo-event-collection "Direct link to Demo: event collection") ``` docker run -p 8080:8080 \ -e FLOW=/app/demos/demo-collect.mjs \ walkeros/flow:latest ``` Output: ``` ✓ Loading flow: /app/demos/demo-collect.mjs ✓ Starting collection server on port 8080 ✓ Health check available at /health [Demo Output] page view [Demo Output] product add ``` The demo is self-contained and generates test events automatically. ### Demo: static file serving[​](#demo-static-file-serving "Direct link to Demo: static file serving") ``` docker run -p 3000:3000 \ -e PORT=3000 \ -e STATIC_DIR=/app/demos \ walkeros/flow:latest ``` Visit `http://localhost:3000/demo-serve.js` to download the demo web bundle. ## Configuration[​](#configuration "Direct link to Configuration") The Docker container is configured via environment variables: ### Server flow[​](#server-flow "Direct link to Server flow") ``` docker run -p 8080:8080 \ -v $(pwd)/flow.mjs:/app/flow.mjs \ -e FLOW=/app/flow.mjs \ -e PORT=8080 \ -e HOST=0.0.0.0 \ walkeros/flow:latest ``` | Environment Variable | Required | Default | Description | | -------------------- | -------- | --------- | -------------------------------------------------------- | | `BUNDLE` | Yes | - | Path, URL, or piped content of a pre-built `.mjs` bundle | | `PORT` | No | `8080` | Server port | | `HOST` | No | `0.0.0.0` | Bind address | ### Web flow[​](#web-flow "Direct link to Web flow") ``` docker run -p 3000:3000 \ -v $(pwd)/dist:/app/static \ -e STATIC_DIR=/app/static \ -e PORT=3000 \ walkeros/flow:latest ``` | Environment Variable | Required | Default | Description | | -------------------- | -------- | ----------- | ------------------ | | `STATIC_DIR` | No | `/app/dist` | Directory to serve | | `PORT` | No | `3000` | Server port | | `HOST` | No | `0.0.0.0` | Bind address | ### Bundle input methods[​](#bundle-input-methods "Direct link to Bundle input methods") The runtime accepts bundles from three sources (checked in priority order): **File path** (default): mount or bake the bundle into the container: ``` docker run -p 8080:8080 \ -v $(pwd)/flow.mjs:/app/flow/bundle.mjs \ walkeros/flow:latest ``` **URL**: fetch the bundle from a remote location at startup: ``` docker run -p 8080:8080 \ -e BUNDLE=https://cdn.example.com/flow.mjs \ walkeros/flow:latest ``` Works with presigned S3/GCS URLs, CDNs, or any HTTP(S) endpoint. **Stdin pipe**: pipe the bundle directly into the container: ``` cat flow.mjs | docker run -i -p 8080:8080 walkeros/flow:latest ``` Stdin takes highest priority. If data is piped, the `BUNDLE` env var is ignored. ## Including files[​](#including-files "Direct link to Including files") The `include` field in your flow config copies folders into the bundle output directory at build time. When combined with `--dockerfile`, these folders are automatically added as `COPY` instructions in the generated Dockerfile. ### The `include` field[​](#the-include-field "Direct link to the-include-field") ``` { "version": 4, "include": ["./public"], "flows": { "default": { "config": { "platform": "server" }, "stores": { "files": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./public" } } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "next": [ { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "next": "file" } ] } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.files" } } } } } } ``` Avoid circular includes The `include` field copies folders into the **output directory** (where the bundle is written). If your output is `dist/bundle.mjs` and you set `include: ["./dist"]`, the bundler will try to copy `dist/` into `dist/dist/`, a circular operation that will error. **Fix:** Only include folders that are **not** the output directory: * `"include": ["./shared"]`: copies `shared/` into the output directory * `"include": ["./credentials"]`: copies `credentials/` into the output directory Build with Dockerfile generation: ``` walkeros bundle flow.json -o dist/ --dockerfile ``` The generated `dist/Dockerfile` includes COPY lines for both the bundle and the included folder: ``` FROM walkeros/flow:latest COPY bundle.mjs /app/flow/bundle.mjs COPY public/ /app/flow/public/ ENV BUNDLE=/app/flow/bundle.mjs EXPOSE 8080 ``` ### Volume mount for development[​](#volume-mount-for-development "Direct link to Volume mount for development") During development, mount folders directly instead of baking them into the image: ``` docker run -v ./public:/app/flow/public -p 8080:8080 walkeros/flow ``` ### Directory separation[​](#directory-separation "Direct link to Directory separation") Keep served files and credentials in separate directories: * **`public/`**: Files served to clients (static assets, templates) * **`credentials/`**: Secret files (API keys, service accounts) The fs store only serves files under its configured `basePath`. Never point it at a directory containing secrets. ## Deployment workflows[​](#deployment-workflows "Direct link to Deployment workflows") ### Workflow 1: volume mount (development)[​](#workflow-1-volume-mount-development "Direct link to Workflow 1: volume mount (development)") Mount pre-built bundles directly into the container. **1. Build with CLI** ``` walkeros bundle server-collect.json ``` This creates `dist/flow.mjs`. **2. Run with Docker** ``` docker run -d \ -p 8080:8080 \ -v $(pwd)/dist/flow.mjs:/app/flow.mjs \ -e FLOW=/app/flow.mjs \ --name walkeros-flow \ walkeros/flow:latest ``` **3. Test it** ``` curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{"name":"page view","data":{"title":"Home"}}' ``` **4. View logs** ``` docker logs -f walkeros-flow ``` ### Workflow 2: custom Docker image (production)[​](#workflow-2-custom-docker-image-production "Direct link to Workflow 2: custom Docker image (production)") Build a custom Docker image with your bundle baked in. **1. Build with CLI and generate Dockerfile** ``` walkeros bundle production.json --dockerfile ``` This creates `dist/bundle.mjs` and `dist/Dockerfile`. **2. Build image** ``` cd dist docker build -t my-analytics:v1.0.0 . ``` **3. Run it** ``` docker run -d -p 8080:8080 my-analytics:v1.0.0 ``` **4. Push to registry** ``` docker tag my-analytics:v1.0.0 gcr.io/my-project/analytics:v1.0.0 docker push gcr.io/my-project/analytics:v1.0.0 ``` ### Workflow 3: multi-stage build[​](#workflow-3-multi-stage-build "Direct link to Workflow 3: multi-stage build") Build and bundle in a single Dockerfile. ``` # Build stage FROM node:18-alpine AS builder WORKDIR /build RUN npm install -g @walkeros/cli COPY flow.json . RUN walkeros bundle flow.json --output flow.mjs # Runtime stage FROM walkeros/flow:latest COPY --from=builder /build/flow.mjs /app/flow.mjs ENV BUNDLE=/app/flow.mjs EXPOSE 8080 ``` Build and run: ``` docker build -t analytics-collector . docker run -d -p 8080:8080 analytics-collector ``` ## Docker Compose[​](#docker-compose "Direct link to Docker Compose") ### Single service[​](#single-service "Direct link to Single service") ``` version: '3.8' services: collector: image: walkeros/flow:latest environment: FLOW: /app/flow.mjs PORT: 8080 volumes: - ./dist/flow.mjs:/app/flow.mjs ports: - "8080:8080" restart: unless-stopped healthcheck: test: ["CMD", "curl", "-f", "http://localhost:8080/health"] interval: 30s timeout: 3s retries: 3 ``` Run it: ``` docker-compose up -d docker-compose logs -f ``` ### Multiple services (web + server)[​](#multiple-services-web--server "Direct link to Multiple services (web + server)") ``` version: '3.8' services: # Server-side event collection collector: image: walkeros/flow:latest environment: FLOW: /app/server-flow.mjs volumes: - ./dist/server-flow.mjs:/app/server-flow.mjs ports: - "8080:8080" restart: unless-stopped # Web bundle serving tracker: image: walkeros/flow:latest environment: STATIC_DIR: /app/static PORT: 3000 volumes: - ./dist:/app/static ports: - "3000:3000" restart: unless-stopped ``` ## Cloud deployment[​](#cloud-deployment "Direct link to Cloud deployment") ### Google Cloud Run[​](#google-cloud-run "Direct link to Google Cloud Run") Cloud Run is ideal for serverless deployment of walkerOS flows. #### Deploy collection endpoint[​](#deploy-collection-endpoint "Direct link to Deploy collection endpoint") **1. Create server flow** Create `bigquery-collect.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/collector": { "version": "latest" }, "@walkeros/server-source-express": { "version": "latest" }, "@walkeros/server-destination-gcp": { "version": "latest" }, "@walkeros/destination-demo": { "version": "latest" } } } }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "settings": { "path": "/collect", "port": 8080, "cors": true } } } }, "destinations": { "bigquery": { "package": "@walkeros/server-destination-gcp", "config": { "settings": { "projectId": "my-project", "datasetId": "analytics", "tableId": "events" } } }, "console": { "package": "@walkeros/destination-demo", "config": { "settings": { "name": "Logger" } } } }, "collector": { "run": true } } } } ``` Starting with Console The config above includes **both** BigQuery and console destinations. This is a best practice: * Console logs help debug issues in Cloud Run logs * Start with console only, verify events flow correctly * Then uncomment BigQuery destination for production To start console-only: remove the `bigquery` destination block. **2. Bundle the flow** ``` walkeros bundle bigquery-collect.json --output flow.mjs ``` **3. Create Dockerfile** ``` FROM walkeros/flow:latest COPY flow.mjs /app/flow.mjs ENV BUNDLE=/app/flow.mjs ENV PORT=8080 ``` **4. Build and push to Google Container Registry** ``` # Set your GCP project export PROJECT_ID=my-project export SERVICE_NAME=analytics-collector export REGION=us-central1 # Build and push docker build -t gcr.io/$PROJECT_ID/$SERVICE_NAME . docker push gcr.io/$PROJECT_ID/$SERVICE_NAME ``` **5. Deploy to Cloud Run** ``` gcloud run deploy $SERVICE_NAME \ --image gcr.io/$PROJECT_ID/$SERVICE_NAME \ --platform managed \ --region $REGION \ --allow-unauthenticated \ --port 8080 \ --memory 512Mi \ --cpu 1 \ --max-instances 10 \ --timeout 60s ``` **6. Get the URL** ``` gcloud run services describe $SERVICE_NAME \ --platform managed \ --region $REGION \ --format 'value(status.url)' # Output: https://analytics-collector-xxxxx-uc.a.run.app ``` **7. Test it** ``` curl -X POST https://analytics-collector-xxxxx-uc.a.run.app/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Landing Page", "path": "/landing" }, "user": { "id": "user123" } }' ``` **8. View logs** ``` gcloud run logs read $SERVICE_NAME \ --region $REGION \ --limit 50 ``` You should see console logs showing the received events. #### Deploy web bundle server[​](#deploy-web-bundle-server "Direct link to Deploy web bundle server") **1. Create web flow** Create `web-tracker.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "web", "bundle": { "packages": { "@walkeros/collector": { "version": "latest" }, "@walkeros/web-source-browser": { "version": "latest" }, "@walkeros/web-destination-api": { "version": "latest" } } } }, "sources": { "browser": { "package": "@walkeros/web-source-browser", "config": { "settings": { "pageview": true, "elb": "track" } } } }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "https://analytics-collector-xxxxx-uc.a.run.app/collect", "method": "POST" } } } }, "collector": { "run": true } } } } ``` **2. Bundle** ``` walkeros bundle web-tracker.json ``` This creates `tracker.js`. **3. Create Dockerfile for serving** ``` FROM walkeros/flow:latest COPY tracker.js /app/static/tracker.js ENV STATIC_DIR=/app/static ENV PORT=8080 ``` **4. Deploy to Cloud Run** ``` # Build and push docker build -t gcr.io/$PROJECT_ID/tracker-server . docker push gcr.io/$PROJECT_ID/tracker-server # Deploy gcloud run deploy tracker-server \ --image gcr.io/$PROJECT_ID/tracker-server \ --platform managed \ --region $REGION \ --allow-unauthenticated \ --port 8080 ``` **5. Use in your website** ``` ``` Production Considerations For production, consider: * **Use a CDN**: Serve static files from Cloud Storage + Cloud CDN instead of Cloud Run * **Custom domain**: Map your own domain (e.g., `cdn.example.com/tracker.js`) * **Versioning**: Include version in filename (e.g., `tracker.v1.2.3.js`) * **Caching**: Set appropriate cache headers ### AWS (ECS/Fargate)[​](#aws-ecsfargate "Direct link to AWS (ECS/Fargate)") Similar workflow for AWS: ``` # Push to ECR aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $AWS_ACCOUNT.dkr.ecr.us-east-1.amazonaws.com docker tag my-analytics-collector:latest $AWS_ACCOUNT.dkr.ecr.us-east-1.amazonaws.com/analytics-collector:latest docker push $AWS_ACCOUNT.dkr.ecr.us-east-1.amazonaws.com/analytics-collector:latest # Create task definition and service using ECS console or CLI # Configure port 8080, environment variables, and health check ``` ### Kubernetes[​](#kubernetes "Direct link to Kubernetes") Deploy to any Kubernetes cluster: ``` apiVersion: apps/v1 kind: Deployment metadata: name: walkeros-flow spec: replicas: 3 selector: matchLabels: app: walkeros-flow template: metadata: labels: app: walkeros-flow spec: containers: - name: collector image: gcr.io/my-project/analytics-collector:v1.0.0 ports: - containerPort: 8080 env: - name: FLOW value: "/app/flow.mjs" - name: PORT value: "8080" livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 10 periodSeconds: 30 resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "512Mi" cpu: "500m" --- apiVersion: v1 kind: Service metadata: name: walkeros-flow spec: selector: app: walkeros-flow ports: - protocol: TCP port: 80 targetPort: 8080 type: LoadBalancer ``` Apply: ``` kubectl apply -f deployment.yaml kubectl get services walkeros-flow ``` ## Health checks[​](#health-checks "Direct link to Health checks") The Docker container provides a `/health` endpoint: ``` curl http://localhost:8080/health ``` Response: ``` { "status": "ok", "timestamp": 1701234567890, "file": "/app/flow.mjs" } ``` Use this for: * Docker health checks * Kubernetes liveness/readiness probes * Load balancer health checks * Monitoring systems ## Monitoring and logs[​](#monitoring-and-logs "Direct link to Monitoring and logs") ### View Docker logs[​](#view-docker-logs "Direct link to View Docker logs") ``` # Follow logs docker logs -f walkeros-flow # Last 100 lines docker logs --tail 100 walkeros-flow # With timestamps docker logs -t walkeros-flow ``` ### Cloud Run logs[​](#cloud-run-logs "Direct link to Cloud Run logs") ``` # Recent logs gcloud run logs read $SERVICE_NAME --region $REGION --limit 50 # Tail logs gcloud run logs tail $SERVICE_NAME --region $REGION # Filter by severity gcloud run logs read $SERVICE_NAME --region $REGION --log-filter="severity>=ERROR" ``` ### Structured logging[​](#structured-logging "Direct link to Structured logging") For better observability, use structured logging in your destinations: ``` { "destinations": { "structured-logger": { "package": "@walkeros/destination-demo", "config": { "settings": { "format": "json" } } } } } ``` ## Security[​](#security "Direct link to Security") The Docker image follows security best practices: * ✅ **Non-root user** - Runs as `walker` (UID 1001) * ✅ **Minimal base** - Alpine Linux (\~150-200MB) * ✅ **No build tools** - Production dependencies only * ✅ **Signal handling** - Graceful shutdown with Tini * ✅ **Health checks** - Built-in endpoint for orchestrators ### Environment secrets[​](#environment-secrets "Direct link to Environment secrets") For sensitive configuration (API keys, credentials), use environment variables: ``` # Docker docker run -e BIGQUERY_KEY="$(cat key.json)" walkeros/flow # Cloud Run gcloud run deploy $SERVICE_NAME \ --set-secrets="BIGQUERY_KEY=bigquery-key:latest" # Kubernetes kubectl create secret generic analytics-secrets \ --from-file=bigquery-key=key.json ``` Reference secrets in your flow configuration using environment variable substitution. ## Performance[​](#performance "Direct link to Performance") ### Startup time[​](#startup-time "Direct link to Startup time") * **Cold start**: < 1 second (flow already bundled) * **Warm start**: < 100ms (container reuse) ### Resource usage[​](#resource-usage "Direct link to Resource usage") Typical resource consumption: | Metric | Idle | Active (1000 req/min) | | ------- | ------ | --------------------- | | Memory | \~50MB | \~100-150MB | | CPU | < 1% | 5-15% | | Startup | < 1s | < 1s | ### Scaling[​](#scaling "Direct link to Scaling") The container is designed for horizontal scaling: ``` # Docker Swarm docker service scale walkeros-flow=5 # Kubernetes kubectl scale deployment walkeros-flow --replicas=5 # Cloud Run (auto-scaling) gcloud run services update $SERVICE_NAME --max-instances=100 ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Container won't start[​](#container-wont-start "Direct link to Container won't start") ``` # Check logs docker logs walkeros-flow # Common issues: # - FLOW path doesn't exist # - Port already in use ``` ### Events not being received[​](#events-not-being-received "Direct link to Events not being received") ``` # Verify container is running docker ps | grep walkeros # Check health endpoint curl http://localhost:8080/health # Test event submission curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{"name":"test event","data":{}}' # Check logs for errors docker logs walkeros-flow ``` ### Port conflicts[​](#port-conflicts "Direct link to Port conflicts") ``` # Use a different host port docker run -p 8081:8080 walkeros/flow # Or find and kill the conflicting process lsof -ti:8080 | xargs kill ``` ### Bundle not found[​](#bundle-not-found "Direct link to Bundle not found") ``` # Verify volume mount docker run -v $(pwd)/dist/flow.mjs:/app/flow.mjs walkeros/flow # Check if file exists in container docker run --rm -it walkeros/flow ls -la /app/ # Verify FLOW environment variable docker run -e FLOW=/app/flow.mjs walkeros/flow ``` ## Next steps[​](#next-steps "Direct link to Next steps") * **[CLI](/docs/apps/cli.md)** - Learn how to build flows * **[Flow Configuration](/docs/getting-started/modes/bundled.md)** - Understand flow structure * **[Sources](/docs/sources/.md)** - Configure event sources * **[Destinations](/docs/destinations/.md)** - Set up analytics destinations * **[Docker Hub](https://hub.docker.com/r/walkeros/flow)** - Official Docker images --- # MCP servers walkerOS provides [Model Context Protocol](https://modelcontextprotocol.io/) servers for AI assistant integration. | Package | Purpose | | ------------------------------ | ----------------------------------------------------------------------- | | `@walkeros/mcp` | Flow development: tools, reference resources, guided prompts, cloud API | | `@walkeros/mcp-source-browser` | HTML tagging: generate, parse, and validate `data-elb` attributes | ## Claude Code plugin[​](#claude-code-plugin "Direct link to Claude Code plugin") The recommended way to get started in Claude Code. One plugin installs both MCP servers **and** 18 skills that teach Claude how to build sources, destinations, transformers, and flows. **Step 1: add the marketplace:** ``` /plugin marketplace add elbwalker/walkerOS ``` **Step 2: install the plugin:** ``` /plugin install walkeros@elbwalker-walkerOS ``` That's it. Claude Code will reload with the MCP tools and skills active. ## Quick start[​](#quick-start "Direct link to Quick start") For Claude Desktop or other MCP clients, add servers to your configuration manually: ``` { "mcpServers": { "walkeros-flow": { "command": "npx", "args": ["@walkeros/mcp"] }, "walkeros-source-browser": { "command": "npx", "args": ["@walkeros/mcp-source-browser"] } } } ``` Each server starts on STDIO and registers its tools automatically. ### Your first flow via AI[​](#your-first-flow-via-ai "Direct link to Your first flow via AI") Everything in this loop runs locally, no account: 1. Install the plugin: `/plugin install walkeros@elbwalker-walkerOS` 2. Ask: "Create a web flow with a GA4 destination, validate it, and simulate a page view." 3. The assistant runs `flow_load` → `flow_validate` → `flow_simulate` and shows you the result. ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Used by | Required | Default | Purpose | | --------------------- | ------- | -------- | ------------------------- | -------------------------------------------------------- | | `WALKEROS_TOKEN` | mcp | No | none | Bearer token fallback (alternative to `auth` tool login) | | `WALKEROS_PROJECT_ID` | mcp | No | none | Active project ID (`proj_...`) | | `WALKEROS_APP_URL` | mcp | No | `https://app.walkeros.io` | Base URL override | `@walkeros/mcp-source-browser` works without any environment variables. All tools are always registered. To authenticate with the walkerOS cloud, use the `auth` tool (device code flow) or set `WALKEROS_TOKEN` as a fallback. *** ## @walkeros/mcp (flow development)[​](#walkerosmcp-flow-development "Direct link to @walkeros/mcp (flow development)") Unified server for flow development, package discovery, reference resources, guided prompts, and cloud API. Replaces the previous separate `@walkeros/mcp-cli` and `@walkeros/mcp-api` packages. ### Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/mcp ``` ### Programmatic usage[​](#programmatic-usage "Direct link to Programmatic usage") `@walkeros/mcp` exports a transport-agnostic server factory so host applications can mount the MCP protocol over HTTP (e.g., from a Next.js Route Handler) instead of running the stdio binary: ``` import { createWalkerOSMcpServer, HttpToolClient, createStreamableHttpHandler, } from '@walkeros/mcp'; const server = createWalkerOSMcpServer({ client: new HttpToolClient(), version: '1.0.0', }); const handler = createStreamableHttpHandler(server, { sessionIdGenerator: () => crypto.randomUUID(), }); // In a Next.js Route Handler: export const POST = handler; ``` To use the raw tool registry without the MCP protocol (e.g., with the Vercel AI SDK), import \`TOOL\_DEFINITIONS\` and provide your own \`ToolClient\` implementation. The stdio binary stays available via \`@walkeros/mcp/stdio\` and the \`walkeros-mcp\` bin entry, unchanged for end users. ### Tools[​](#tools "Direct link to Tools") #### Local tools (no account)[​](#local-tools-no-account "Direct link to Local tools (no account)") ##### `flow_load`[​](#flow_load "Direct link to flow_load") Load an existing flow configuration from a local file path, URL, inline JSON, or the walkerOS API, or create a new empty flow by specifying a platform. | Parameter | Type | Required | Description | | ---------- | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------- | | `source` | string | No | Flow source: file path, URL, inline JSON, or API flow/config ID (`flow_...` / `cfg_...`). Omit to create new. | | `platform` | `"web"` \| `"server"` | No | Platform for new flows. Required when source is omitted. | Passing a `flow_...` or `cfg_...` ID loads that flow from the API (the same source `flow_manage` reads from). Returned configs are round-trip safe: structural values such as package names, platform, and IDs come back literally, so a loaded config can be edited and sent straight back to `flow_manage({ action: "update" })` without altering anything you did not change. ##### `flow_validate`[​](#flow_validate "Direct link to flow_validate") Validate walkerOS events, flow configurations, mapping rules, or data contracts. | Parameter | Type | Required | Description | | --------- | ---------------------------------------------------- | -------- | ------------------------------------------ | | `type` | `"event"` \| `"flow"` \| `"mapping"` \| `"contract"` | Yes | Validation type | | `input` | string | Yes | JSON string, file path, or URL to validate | | `flow` | string | No | Flow name for multi-flow configs | | `path` | string | No | Entry path for package schema validation | ##### `flow_bundle`[​](#flow_bundle "Direct link to flow_bundle") Bundle a walkerOS flow configuration into deployable JavaScript. | Parameter | Type | Required | Description | | ------------ | ------- | -------- | --------------------------------------------------------------------------------------------------- | | `configPath` | string | Yes | Flow source: file path, or an API flow/config ID (`flow_...` / `cfg_...`) resolved like `flow_load` | | `flow` | string | No | Flow name for multi-flow configs | | `stats` | boolean | No | Return bundle statistics (default: `true`) | | `output` | string | No | Output file path | `configPath` accepts a cloud flow id, so you can bundle a saved flow directly without loading it to a file first. Bundle stats report the real total bundle size and the included package names (no per-package size estimate). ##### `flow_simulate`[​](#flow_simulate "Direct link to flow_simulate") Simulate events through a walkerOS flow without making real API calls. Returns summarized per-destination results. | Parameter | Type | Required | Description | | ------------ | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------ | | `configPath` | string | Yes | Flow source: file path, or an API flow/config ID (`flow_...` / `cfg_...`) resolved like `flow_load` | | `event` | string \| object | Yes | Event input (see shapes below). JSON string, file path, or URL also accepted. | | `step` | string | Yes | Target step as `"type.name"` (e.g. `"source.demo"`, `"collector.default"`, `"destination.gtag"`, `"transformer.router"`) | | `flow` | string | No | Flow name for multi-flow configs | | `platform` | `"web"` \| `"server"` | No | Override platform detection | | `verbose` | boolean | No | Include full payload per destination (default: `false`) | There are four step types: `source`, `collector`, `transformer`, and `destination`. The `event` shape depends on the step type: * **Destinations / transformers:** a walkerOS event, `{ name: "entity action", data: {...} }`. Add `consent` (e.g. `{ marketing: true }`) when the destination requires it. * **Collector:** the enrichment step. It takes a post-`next` partial event plus an optional state snapshot `{ consent?, user?, globals?, timing? }`, applies the collector's `createEvent`, and returns the fully enriched event. * **Sources:** `{ content, trigger? }`, where `content` is the walkerOS event `{ name, data }` and the optional `trigger` is `{ type?, options? }`. There is no `env` field in the source-step event. Sources can be simulated as a step, including the `@walkeros/source-demo` demo source. When `configPath` is a cloud flow id, you can simulate a saved flow without a manual file round-trip. Repeated simulations of the same configuration reuse a prebuilt bundle, so successive calls run faster (local file paths always rebuild). A `transformer` step also accepts an optional `ingest` field, a raw ingest without `_meta`. Supply it to test a request decoder standalone, for example a GA4 decoder that reads `ctx.ingest.url`: pass `ingest: { url: "..." }` alongside the event. ##### `flow_push`[​](#flow_push "Direct link to flow_push") Push a real event through a walkerOS flow to actual destinations. **This makes real API calls.** Best for server-side flows. Web destinations require browser globals not available in Node.js. | Parameter | Type | Required | Description | | ------------ | --------------------- | -------- | --------------------------------------- | | `configPath` | string | Yes | Path to flow configuration file | | `event` | string | Yes | Event as JSON string, file path, or URL | | `flow` | string | No | Flow name for multi-flow configs | | `platform` | `"web"` \| `"server"` | No | Override platform detection | ##### `flow_examples`[​](#flow_examples "Direct link to flow_examples") List all step examples in a walkerOS flow configuration. | Parameter | Type | Required | Description | | ------------ | ------- | -------- | --------------------------------------------------------------------------------------------------- | | `configPath` | string | Yes | Flow source: file path, or an API flow/config ID (`flow_...` / `cfg_...`) resolved like `flow_load` | | `flow` | string | No | Flow name for multi-flow configs | | `step` | string | No | Filter to a specific step (e.g., `"destination.gtag"`) | | `full` | boolean | No | Return full in/out/mapping data (default: `false`, metadata only) | When a step has no inline examples, `flow_examples` falls back to the examples shipped with that step's package, tagged by source so you can tell inline examples from package-provided ones. ##### `package_search`[​](#package_search "Direct link to package_search") Browse walkerOS packages or look up a specific one. Without package name: returns catalog filtered by type/platform. With package name: returns metadata. | Parameter | Type | Required | Description | | ---------- | ------------------------------------------------------------- | -------- | --------------------------------- | | `package` | string | No | Exact npm package name for lookup | | `type` | `"source"` \| `"destination"` \| `"transformer"` \| `"store"` | No | Filter by type (browse) | | `platform` | `"web"` \| `"server"` | No | Filter by platform (browse) | | `version` | string | No | Package version (default: latest) | In browse mode the tool returns the complete catalog. If it has to fall back to a partial source or omit packages, the response includes a `warnings` array explaining what is incomplete rather than silently returning a partial list. ##### `package_get`[​](#package_get "Direct link to package_get") Fetch walkerOS package details from npm. By default returns schemas + hint texts + example summaries. Use `section` parameter for full content. | Parameter | Type | Required | Description | | --------- | ------------------------------------ | -------- | ----------------------------------- | | `package` | string | Yes | Exact npm package name | | `version` | string | No | Package version (default: latest) | | `section` | `"hints"` \| `"examples"` \| `"all"` | No | Section to expand with full content | ##### `diagnostics`[​](#diagnostics "Direct link to diagnostics") Report the MCP runtime surface. Read-only, takes no parameters, and works even when logged out. Reach for it when a request fails, to see which versions and backend you are on. The response includes the MCP version, the CLI version, the resolved app URL and whether it came from `WALKEROS_APP_URL` or the default, app `/api/health` reachability, the bundled OpenAPI contract version, and which source served the last package catalog lookup. This tool has no parameters. #### Cloud tools[​](#cloud-tools "Direct link to Cloud tools") ☁️walkerOS Cloud These tools manage flows in the hosted app: shared projects, deploys, secrets, and live-site previews. Self-hosting? Local `flow.json` files and the CLI cover the same flow-development loop. [What Cloud adds →](https://app.walkeros.io) ##### `auth`[​](#auth "Direct link to auth") Authenticate with the walkerOS cloud. Uses device code flow — the user receives a URL to open in a browser to complete login. | Parameter | Type | Required | Description | | ------------ | ------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------- | | `action` | `"status"` \| `"login"` \| `"logout"` | Yes | Auth action | | `deviceCode` | string | No | Device code from a previous pending login. Provide with `action: "login"` to resume polling without requesting a new code. | ##### `project_manage`[​](#project_manage "Direct link to project_manage") Manage walkerOS projects in the cloud. | Parameter | Type | Required | Description | | ----------- | -------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------ | | `action` | `"list"` \| `"get"` \| `"create"` \| `"update"` \| `"delete"` \| `"set_default"` | Yes | Project action | | `projectId` | string | No | Project ID (`proj_...`). Required for: get, update, delete, set\_default. Falls back to `WALKEROS_PROJECT_ID` env. | | `name` | string | No | Name for create/update operations | ##### `flow_manage`[​](#flow_manage "Direct link to flow_manage") Manage walkerOS flow configurations and previews in the cloud. | Parameter | Type | Required | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `action` | `"list"` \| `"get"` \| `"create"` \| `"update"` \| `"delete"` \| `"duplicate"` \| `"preview_list"` \| `"preview_get"` \| `"preview_create"` \| `"preview_delete"` \| `"preview_regrant"` | Yes | Flow action | | `projectId` | string | No | Project ID (`proj_...`). Optional filter for list. Required for create if no default project set. Falls back to `WALKEROS_PROJECT_ID` env. | | `flowId` | string | No | Flow ID (`flow_...`) or config ID (`cfg_...`). Required for: get, update, delete, duplicate, preview\_list, preview\_get, preview\_create, preview\_delete, preview\_regrant. | | `name` | string | No | Flow name. Required for create. Optional for update (to rename) and duplicate. | | `content` | object | No | Flow\.Json content. Used for create and update. | | `patch` | boolean | No | Merge-patch for update (default: `true`). When true, only provided fields are updated. | | `fields` | string\[] | No | Dot-path selectors for get to return only specific fields. | | `sort` | `"name"` \| `"updated_at"` \| `"created_at"` | No | Sort field for list. | | `order` | `"asc"` \| `"desc"` | No | Sort order for list. | | `includeDeleted` | boolean | No | Include soft-deleted flows in list results. | | `previewId` | string | No | Preview ID (`prv_...`). Required for: preview\_get, preview\_delete, preview\_regrant. | | `flowName` | string | No | Flow settings name. Used by preview\_create as an alternative to `flowSettingsId`. | | `flowSettingsId` | string | No | Flow settings ID. Used by preview\_create as an alternative to `flowName`. | | `source` | object | No | What preview\_create should run: `{ "kind": "draft" }` (default) or `{ "kind": "deployment-version", "deploymentVersionId": "..." }` to preview a deployed version's stored config ("preview what's live"). | | `siteUrl` | string | No | Optional site URL for preview\_create. When provided, an app-signed activation grant is minted for that origin and the response's `activationUrl` works there; otherwise `activationUrl` is `null` until a grant is minted (see preview\_regrant). | | `origins` | string\[] | No | Site origins (bare `https://host[:port]`) to mint a preview activation grant for. Used by preview\_regrant; the returned `activationUrl` targets the first origin. | Configs returned by `get` are round-trip safe: structural values (package names, platform, IDs) are returned literally, so a returned config can be edited and sent back to `update` unchanged. ##### `deploy_manage`[​](#deploy_manage "Direct link to deploy_manage") Deploy walkerOS flows and manage deployments. | Parameter | Type | Required | Description | | ----------- | ----------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `action` | `"deploy"` \| `"list"` \| `"get"` \| `"delete"` | Yes | Deployment action | | `flowId` | string | Conditional | Flow ID. Required for: `deploy`, `get`, `delete`. Optional filter for `list`. | | `slug` | string | No | Deployment slug. Optional disambiguator for `get`/`delete` when the flow has multiple active deployments. | | `projectId` | string | No | Project ID. Optional; falls back to the default project. | | `type` | `"web"` \| `"server"` | No | Deployment type filter for `list`. | | `status` | string | No | Status filter for `list`. | | `wait` | boolean | No | Wait for the deployment to reach a terminal status (default: `true`), with a 120-second budget. Set `false` to return the deployment id immediately. Only used with `deploy`. | | `flowName` | string | No | Flow name for multi-settings flows. Only used with `deploy`. | | `cursor` | string | No | Pagination cursor from a previous `list` response. Only used with `list`. | | `limit` | number | No | Max items per page (1-100). Only used with `list`. | A finished `deploy` carries its status and, on failure, an `errorMessage` with the user-facing reason; use the `get` action to re-read it. When a flow has more than one active deployment and no `slug` is supplied, `get` and `delete` return a `MULTIPLE_DEPLOYMENTS` error with a `details[]` list so the caller can pick a specific deployment. Soft-deleted deployments are always excluded. ``` { "error": "Flow flow_abc has 2 active deployments; pass slug to disambiguate", "code": "MULTIPLE_DEPLOYMENTS", "details": [ { "slug": "abc123456789", "type": "web", "status": "active", "updatedAt": "2026-04-20T00:00:00.000Z" }, { "slug": "def987654321", "type": "web", "status": "active", "updatedAt": "2026-04-21T00:00:00.000Z" } ] } ``` ##### `secret_manage`[​](#secret_manage "Direct link to secret_manage") Manage a flow's secrets, the `$env.` values its steps reference at deploy and run time. | Parameter | Type | Required | Description | | ----------- | ----------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------- | | `action` | `"list"` \| `"set"` \| `"update"` \| `"delete"` | Yes | Secret action | | `flowId` | string | Yes | Flow ID (`flow_...`) or config ID (`cfg_...`). Secrets are flow-scoped, so it is required for every action. | | `projectId` | string | No | Project ID. Optional; falls back to the default project. | | `name` | string | Conditional | Secret name (UPPER\_SNAKE\_CASE, referenced as `$env.`). Required for `set`. | | `value` | string | Conditional | Secret value (1-65536 chars). Required for `set` and `update`. Write-only: never returned or logged. | | `secretId` | string | Conditional | Secret ID (`sec_...`). Required for `update` and `delete`. Use `list` to find it. | Required fields per action: | Action | Required fields | Effect | | -------- | ----------------------------- | ----------------------------------- | | `list` | `flowId` | Return secret metadata for the flow | | `set` | `flowId`, `name`, `value` | Create a new secret | | `update` | `flowId`, `secretId`, `value` | Rotate an existing secret's value | | `delete` | `flowId`, `secretId` | Remove a secret | Secrets are write-mostly. Values are encrypted at rest and are NEVER returned, listed, or echoed back: `set` and `update` respond with metadata only, and `list` returns names, ids, and timestamps but no values. The only way to learn a secret's value is to rotate it with a new one. Reference a secret from any flow step as `$env.`, for example `$env.API_TOKEN`. ``` { "action": "set", "flowId": "flow_abc", "name": "API_TOKEN", "value": "the-secret-value" } ``` ### Resources[​](#resources "Direct link to Resources") | URI | Description | | ---------------------------------- | ----------------------------------------------------------- | | `walkeros://reference/flow-schema` | Flow\.Json structure and connection rules | | `walkeros://reference/event-model` | Event naming, properties, auto-populated fields | | `walkeros://reference/mapping` | Mapping syntax (data/map/loop/set/condition/consent/policy) | | `walkeros://reference/consent` | Consent model (destination/rule/field level) | | `walkeros://reference/variables` | Variable patterns ($var/$env/$code/$store) | | `walkeros://reference/contract` | Event schemas, wildcards, inheritance | | `walkeros://reference/openapi` | OpenAPI 3.1 specification | | `walkeros://reference/packages` | Full package catalog | | `walkeros://schema/{packageName}` | Per-package JSON schemas | ### Prompts[​](#prompts "Direct link to Prompts") | Prompt | Description | | ----------------- | ----------------------------------------------------------- | | `add-step` | Add a source, destination, transformer, or store to a flow | | `setup-mapping` | Configure event mapping for a step | | `manage-contract` | Create/update event contracts (bidirectional with mappings) | *** ## @walkeros/mcp-source-browser (HTML tagging tools)[​](#walkerosmcp-source-browser-html-tagging-tools "Direct link to @walkeros/mcp-source-browser (HTML tagging tools)") Generate, parse, and validate walkerOS `data-elb` HTML attributes using real DOM parsing (JSDOM). No API token or CLI dependency required. ### Installation[​](#installation-1 "Direct link to Installation") ``` npm install @walkeros/mcp-source-browser ``` ### Tools (3)[​](#tools-3 "Direct link to Tools (3)") #### `generate_tagging`[​](#generate_tagging "Direct link to generate_tagging") Generate walkerOS `data-elb` HTML attributes from structured input. Returns attribute key-value pairs and an example HTML snippet. | Parameter | Type | Required | Description | | --------- | ------ | -------- | -------------------------------------------------------------------------- | | `entity` | string | No | Entity name (creates `data-elb="entity"`) | | `data` | object | No | Entity properties as key:value pairs | | `action` | object | No | Trigger:action pairs for `data-elbaction` (nearest entity) | | `actions` | object | No | Trigger:action pairs for `data-elbactions` (all entities) | | `context` | object | No | Context properties for `data-elbcontext` | | `globals` | object | No | Global properties for `data-elbglobals` | | `link` | object | No | Link relationships for `data-elblink` | | `prefix` | string | No | Custom prefix (default: `data-elb`) | At least one parameter must be provided. #### `parse_tagging`[​](#parse_tagging "Direct link to parse_tagging") Parse HTML with `data-elb` attributes using real DOM parsing (JSDOM). Extracts all walkerOS events and globals. | Parameter | Type | Required | Description | | --------- | ------ | -------- | --------------------------------------- | | `html` | string | Yes | HTML snippet with `data-elb` attributes | | `prefix` | string | No | Custom prefix (default: `data-elb`) | #### `validate_tagging`[​](#validate_tagging "Direct link to validate_tagging") Validate HTML `data-elb` tagging for common mistakes. Checks for orphan actions, missing entities, unknown triggers, orphan properties, and entities without actions. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------------------------------- | | `html` | string | Yes | HTML snippet to validate | | `prefix` | string | No | Custom prefix (default: `data-elb`) | ### Resources[​](#resources-1 "Direct link to Resources") | URI | Description | | ----------------------------------------- | --------------------------------------------------- | | `walkeros://docs/tagging/html-attributes` | Complete guide to `data-elb` HTML attribute tagging | | `walkeros://docs/tagging/tagger` | `createTagger()` fluent API reference | *** ## Example workflows[​](#example-workflows "Direct link to Example workflows") ### Create and validate a flow[​](#create-and-validate-a-flow "Direct link to Create and validate a flow") Ask your AI assistant: > "Create a new web flow, add a GA4 destination, then validate it." The assistant uses `flow_load` to create a skeleton, the `add-step` prompt to add GA4, and `flow_validate` to check the result. ### Simulate events[​](#simulate-events "Direct link to Simulate events") > "Simulate a page view event through my flow at ./flow\.json." The assistant calls `flow_simulate` and returns per-destination results showing which destinations received the event. ### Deploy a flow[​](#deploy-a-flow "Direct link to Deploy a flow") ☁️walkerOS Cloud Requires a walkerOS account. The local loop (create, validate, simulate) needs none. > "Deploy flow cfg\_abc123 and wait for it to finish." The assistant calls `deploy_manage({ action: "deploy", flowId: "cfg_abc123" })` and streams progress updates through bundling, publishing, and activation. ### Preview a flow on a live site[​](#preview-a-flow-on-a-live-site "Direct link to Preview a flow on a live site") ☁️walkerOS Cloud Requires a walkerOS account. The local loop (create, validate, simulate) needs none. > "Create a preview of my demo settings on flow\_abc123 and give me the link to open on ." The assistant calls `flow_manage({ action: "preview_create", flowId: "flow_abc123", flowName: "demo", siteUrl: "https://example.com" })` and returns the grant-based `activationUrl` the user clicks to activate preview mode on their site. To activate on additional origins later, the assistant calls `flow_manage({ action: "preview_regrant", flowId: "flow_abc123", previewId: "prv_...", origins: [...] })` to mint a fresh grant. Running `flow_manage({ action: "preview_delete", … })` later removes the bundle; the production walker self-heals on visitors' next page load. ### Set up event mapping[​](#set-up-event-mapping "Direct link to Set up event mapping") > "Help me set up mapping for the gtag destination in my flow." The assistant uses the `setup-mapping` prompt, reads the mapping reference resource, fetches package examples, and generates mapping rules. ### Generate HTML tagging[​](#generate-html-tagging "Direct link to Generate HTML tagging") > "Generate data-elb attributes for a promotion entity with name 'Summer Sale' and a click action." The assistant calls `generate_tagging` with `entity: "promotion"`, `data: { name: "Summer Sale" }`, and `action: { click: "click" }`, returning ready-to-use HTML attributes. ### Discover a package[​](#discover-a-package "Direct link to Discover a package") > "What configuration does the Snowplow destination need?" The assistant calls `package_search` for `@walkeros/web-destination-snowplow`, then `package_get` to fetch schemas, hints, and examples. ## Next steps[​](#next-steps "Direct link to Next steps") * **[CLI](/docs/apps/cli.md)**: Learn about the underlying CLI commands * **[Flow configuration](/docs/getting-started/modes/bundled.md)**: Understand flow structure * **[HTML attributes](/docs/sources/web/browser/tagging/html-attributes.md)**: Learn about `data-elb` tagging * **[MCP specification](https://modelcontextprotocol.io/)**: Learn about the Model Context Protocol --- # walkerOS Runner The walkerOS runner is the `walkeros/flow` Docker image in **self-bundling mode**. Point it at a flow config (local file or API) and it bundles, runs, and keeps itself up to date. ``` docker run -v ./flow.json:/app/flow.json -e BUNDLE=/app/flow.json -p 8080:8080 walkeros/flow ``` ## Runner vs Docker runtime[​](#runner-vs-docker-runtime "Direct link to Runner vs Docker runtime") | | [Runner](/docs/apps/runner.md) (this page) | [Docker runtime](/docs/apps/docker.md) | | ------------------- | ------------------------------------------ | -------------------------------------- | | **Input** | Flow config (JSON) | Pre-built bundle (.mjs) | | **Bundling** | Self-bundles internally | None; expects pre-built | | **API integration** | Heartbeat, config polling, hot-swap | None | | **Use case** | Self-hosted production, managed deploy | Simple deploy, CI/CD pipelines | Use the **runner** when you want API visibility, remote config, or hot-swap. Use the **Docker runtime** when you pre-build bundles in CI and want minimal images. ## Deployment modes[​](#deployment-modes "Direct link to Deployment modes") Same image, same config format. Each mode adds one env var: | Mode | What you set | What happens | | -------------------- | --------------------------------- | ---------------------------------------------------------------- | | **A. Local** | `BUNDLE` only | Bundles config, runs. No API. | | **B. Registered** | + `WALKEROS_TOKEN` + `PROJECT_ID` | Same as A, plus heartbeat. Visible in dashboard. | | **C. Remote config** | + `FLOW_ID` | Fetches config from API. Polls for updates. Hot-swaps on change. | | **D. Managed** | (set by walkerOS) | Same as C, on walkerOS infrastructure. | ### Mode A: fully local[​](#mode-a-fully-local "Direct link to Mode A: fully local") No signup, no token, no API: ``` docker run -v ./flow.json:/app/flow.json \ -e BUNDLE=/app/flow.json \ -p 8080:8080 \ walkeros/flow ``` ### Mode B: local config + dashboard[​](#mode-b-local-config--dashboard "Direct link to Mode B: local config + dashboard") Adds heartbeat registration. The runner appears in your project dashboard: ``` docker run -v ./flow.json:/app/flow.json \ -e BUNDLE=/app/flow.json \ -e WALKEROS_TOKEN=sk-walkeros-xxx \ -e PROJECT_ID=proj_xxx \ -p 8080:8080 \ walkeros/flow ``` ### Mode C: remote config with hot-swap[​](#mode-c-remote-config-with-hot-swap "Direct link to Mode C: remote config with hot-swap") Config is fetched from the API. Polls for updates, hot-swaps on new versions: ``` docker run \ -e WALKEROS_TOKEN=sk-walkeros-xxx \ -e PROJECT_ID=proj_xxx \ -e FLOW_ID=flow_xxx \ -v runner-cache:/app/cache \ -p 8080:8080 \ walkeros/flow ``` ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Default | Description | | -------------------- | ------------------------- | ------------------------------------------------------------- | | `MODE` | `collect` | `collect` (HTTP event server) or `serve` (static file server) | | `PORT` | `8080` | Server port | | `BUNDLE` | `/app/flow/bundle.mjs` | Local config/bundle path or URL | | `WALKEROS_TOKEN` | none | API token for registration and config fetch | | `PROJECT_ID` | none | Project ID (required with `WALKEROS_TOKEN`) | | `FLOW_ID` | none | Flow ID for remote config (requires token + project) | | `FLOW_NAME` | none | Flow name for multi-flow configs (selects which flow to run) | | `CACHE_DIR` | `/app/cache` | Directory for last-known-good bundle cache | | `POLL_INTERVAL` | `30` | Seconds between config polls (mode C/D) | | `HEARTBEAT_INTERVAL` | `60` | Seconds between heartbeats (mode B/C/D) | | `WALKEROS_APP_URL` | `https://app.walkeros.io` | API base URL | ### Validation rules[​](#validation-rules "Direct link to Validation rules") Invalid combinations fail fast with actionable error messages: | Vars set | Result | | ------------------------------------------- | ----------------------------------------------------------------------------------- | | Nothing or `BUNDLE` only | Mode A: local bundle, no API | | `WALKEROS_TOKEN` + `PROJECT_ID` | Mode B: local config + heartbeat | | `WALKEROS_TOKEN` + `PROJECT_ID` + `FLOW_ID` | Mode C/D: remote config + polling | | `FLOW_ID` without token | Error: `FLOW_ID requires WALKEROS_TOKEN and PROJECT_ID` | | `WALKEROS_TOKEN` without `PROJECT_ID` | Error: `WALKEROS_TOKEN requires PROJECT_ID` | | `BUNDLE` + `FLOW_ID` | Starts with local bundle, polls for remote. Hot-swaps when remote version is newer. | ## Pipeline[​](#pipeline "Direct link to Pipeline") After config is resolved, all modes converge into the same pipeline: ``` 1. Validate env ─────────── all modes ────────── 2. Fetch secrets ──────────── if flow_id set ─── 3. Resolve config local | API fetch ↓ ┌──────────────────┐ │ config (Flow.Json) │ └──────────────────┘ ↓ 4. Bundle ───────────────── esbuild ──────────── 5. Cache ────────────── write last-known-good ── 6. Run ──────────────────── all modes ────────── 7. Heartbeat ────────────── if token set ─────── 8. Poll + hot-swap ──────── if flow_id set ───── ``` ## Secrets[​](#secrets "Direct link to Secrets") When `FLOW_ID` is set, the runner fetches secrets from the walkerOS cloud API and injects them into `process.env` before the flow starts. This lets you use `$env.SECRET_NAME` references in your flow config without manually setting environment variables on the container. Secrets are managed per-flow in the walkerOS dashboard. The runner handles them automatically: 1. **At startup**: secrets are fetched and injected into the environment before config resolution and bundling. The `$env.*` references in your flow config resolve to the fetched values. 2. **On hot-swap**: when a new config version is detected, secrets are re-fetched before re-bundling. This ensures updated secrets take effect alongside the new config. ### Error handling[​](#error-handling "Direct link to Error handling") The runner treats secret fetch failures differently depending on the cause: | Error | Behavior | | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | **401/403 (auth failure)** | Fatal: the runner exits. An invalid token means config fetch will also fail. | | **404 (no secrets configured)** | Warning: the runner continues. The flow may not use `$env.*` references. | | **500 (server error)** | Warning: the runner continues without secrets. | | **Failure during hot-swap** | The swap is skipped entirely. The runner keeps running the current flow with existing secrets and retries on the next poll. | Transient fetch failures (request timeouts, network errors, and 5xx responses) are retried a few times with bounded, jittered backoff before the behavior above applies. This covers the secret, config, and bundle fetches at startup, so a brief blip while the container starts does not fail the run. The secret and bundle fetches are also each bounded by a timeout, so an unresponsive API cannot stall startup indefinitely. note Secrets only apply in mode C/D (remote config). In local modes, set environment variables directly on the container with `-e` or in your Docker Compose file. ## Polling and hot-swap[​](#polling-and-hot-swap "Direct link to Polling and hot-swap") When `FLOW_ID` is set, the runner polls for config changes using ETags: 1. Every `POLL_INTERVAL` seconds, checks for a new config version 2. `304 Not Modified`: no action 3. New version detected: re-fetches secrets, downloads config, bundles locally, atomically swaps the running flow, updates cache, reports version via heartbeat If the new bundle fails to load, the current flow stays active (safe swap). If secret refresh fails during a poll, the entire swap is skipped and the runner retries on the next poll cycle. ## Caching and resilience[​](#caching-and-resilience "Direct link to Caching and resilience") The runner writes the last working bundle to `CACHE_DIR` (`/app/cache`). This provides cold-start resilience: * API unreachable at startup: falls back to cached bundle * Bundling fails: uses cached bundle instead * Mount as a Docker volume to persist across restarts ``` # Volume mount for cache persistence docker run \ -e WALKEROS_TOKEN=sk-walkeros-xxx \ -e PROJECT_ID=proj_xxx \ -e FLOW_ID=flow_xxx \ -v runner-cache:/app/cache \ walkeros/flow ``` ## Health checks[​](#health-checks "Direct link to Health checks") The runner provides its own health server on the configured `PORT`, independent of flow sources. Health endpoints are always available, even during flow hot-swaps or when no flow handler is loaded. | Endpoint | Description | | ------------- | ------------------------------------------------------------------------------ | | `GET /health` | Liveness check: always returns `200` | | `GET /ready` | Readiness check: returns `200` when a flow handler is mounted, `503` otherwise | ``` curl http://localhost:8080/health ``` ``` { "status": "ok" } ``` All other requests are delegated to the flow's HTTP handler (e.g., Express source routes). The Docker image includes a built-in `HEALTHCHECK` that polls `/health` every 30 seconds with a 30-second start period. ## Observability and runtime tracing[​](#observability-and-runtime-tracing "Direct link to Observability and runtime tracing") A flow's baseline observability is set by the `flow..config.observe` block: | Level | What the runtime emits | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `off` | No baseline records are emitted. In a managed deployment the observer and trace poll stay wired, so runtime trace can still elevate to `trace`; a self-hosted bundle built with no telemetry option makes no fetches at all. | | `standard` (default) | Structural FlowState records, without inbound or outbound event payloads. | | `trace` | Full payloads on every hop (inbound and outbound), sample rate `1`. | When `observe` is omitted, managed deployments default to `standard`. ### Activating trace at runtime (no redeploy)[​](#activating-trace-at-runtime-no-redeploy "Direct link to Activating trace at runtime (no redeploy)") Trace can be turned on and off for a running deployment without rebuilding or redeploying. The mechanism is a poll, not the heartbeat: 1. The deployment carries a trace window timestamp (`traceUntil`). 2. The observer exposes `GET /trace/:deploymentId` (authenticated with the ingest token), returning `{ "traceUntil": "" | null }`. 3. Both the server runtime (`@walkeros/cli run`) and the deployed browser bundle poll that endpoint about every 15 seconds and feed the returned value into `resolveTelemetryOptions({ traceUntil })` in `@walkeros/core`. The resolver applies the value per emit: * A `traceUntil` that parses to a **future** ISO timestamp forces `level: trace` with full inbound and outbound payloads and `sample: 1`, overriding the flow's `observe` block. * A `null` or **past** value reverts to the flow's `observe` baseline. * The window self-expires once the timestamp passes, so trace stops on its own with no further action. Only a `200` with a parseable body changes the active trace state. A network error, a non-200, or an unparseable response leaves the current window untouched, so a transient observer outage never silently disables (or enables) tracing. ### Journey correlation across flows[​](#journey-correlation-across-flows "Direct link to Journey correlation across flows") Every FlowState record carries correlation fields so a single observer can stitch one request across web and server flows. These fields survive every level, including `standard`; they are never gated behind `trace`. | Field | Meaning | | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `traceId` | W3C 32-hex trace id of the originating run. Resolved as `event.source.trace`, falling back to an inbound header's trace, then the run-scoped collector trace. Never synthesized per hop. | | `eventId` | W3C span id of the walker event at this hop (`event.id`). | | `parentEventId` | The upstream runtime's `event.id`, present when this flow was entered across a `$flow` crossing (from an inbound `traceparent`). | | `sourceId` | The originating source id. | | `seq` | Per-poster monotonic counter, stamped on transport (see [Loss-visible transport](#loss-visible-transport)). | Correlation crosses the wire via the W3C `traceparent` header: * **Emit.** The web `api` destination adds `traceparent: 00---01` to each non-batched send when the event carries both ids. A `traceparent` you configure yourself always wins (matched case-insensitively). Batched sends carry no `traceparent`, since one batch may aggregate distinct upstream traces. * **Adopt.** Any server source with a request header bag (for example an Express or fetch handler) reads a valid inbound `traceparent` and threads its trace and parent span into the ingest context, so the receiving flow's records share the caller's `traceId` and link back via `parentEventId`. An absent or malformed header is ignored. The AWS and GCP handlers do not yet adopt inbound request identity. ### Journey status[​](#journey-status "Direct link to Journey status") Records sharing a `traceId` assemble into a single journey, and each journey carries a status derived from the hops observed so far: | Status | Meaning | | ---------- | -------------------------------------------------------------------------------------------------------- | | `pending` | The journey is still settling: a record arrived recently, or an expected hop has no terminal record yet. | | `complete` | Every expected hop reached a terminal state (`out`, `error`, or `skip`). | | `partial` | The journey has settled, but an expected hop never produced a terminal record. | "Expected" comes from the flow's step topology when one is known (only steps downstream of a processed hop are expected); without a topology, a journey is `complete` once every observed hop is terminal. A separate `lossy` flag is orthogonal to status: a journey is `lossy` when a `seq` gap (see [Loss-visible transport](#loss-visible-transport)) overlaps its window, so a journey can be `complete` and `lossy`, or `partial` and `lossy`, at once. ### Vendor-call capture[​](#vendor-call-capture "Direct link to Vendor-call capture") At `trace` level a destination can record the outgoing vendor calls it makes, so an observer sees not just the mapped event but the exact call that left for the vendor. A destination opts in by declaring its observable callables as dot-paths (the `gtag` destination declares `call:window.gtag`); the runtime then wraps those callables and records each `{ fn, args, ts }` onto the record's outbound payload, with arguments sanitized to JSON-safe markers. Capture works whenever the destination reaches its callable through the env. Server and simulate runs carry the callable in the env directly. On live web a destination resolves a real global (for example `window.gtag`) through `@walkeros/web-core`'s `getEnv`; the runtime wraps the declared callables at that resolution point, so calls are recorded without the destination or the page changing. The batched delivery path is not captured: batched sends flush against the unwrapped env, so a batching destination shows its per-event input and output but not its vendor calls. Recorded calls are payload-class: they appear only when the outbound payload does (`trace`, or an explicit `includeOut`). ### Loss-visible transport[​](#loss-visible-transport "Direct link to Loss-visible transport") The batched poster stamps each record with a monotonic `seq` before sending. A dropped or rejected POST is never re-sequenced, so a gap in `seq` at the observer makes lost telemetry visible instead of silent. The poster also splits a batch by real UTF-8 byte size (`maxBodyBytes`, default 60000) rather than letting an oversized body fail with a 413. Docker Compose examples ### Mode A: local config[​](#mode-a-local-config "Direct link to Mode A: local config") ``` services: runner: image: walkeros/flow:latest volumes: - ./flow.json:/app/flow.json environment: BUNDLE: /app/flow.json PORT: "8080" ports: - "8080:8080" ``` ### Mode C: remote config with cache[​](#mode-c-remote-config-with-cache "Direct link to Mode C: remote config with cache") ``` services: runner: image: walkeros/flow:latest volumes: - runner-cache:/app/cache environment: WALKEROS_TOKEN: ${WALKEROS_TOKEN} PROJECT_ID: ${PROJECT_ID} FLOW_ID: ${FLOW_ID} PORT: "8080" ports: - "8080:8080" restart: unless-stopped volumes: runner-cache: ``` Heartbeat details When `WALKEROS_TOKEN` and `PROJECT_ID` are set, the runner sends periodic heartbeats: * **Endpoint**: `POST /api/projects/:projectId/runners/heartbeat` * **Interval**: Every `HEARTBEAT_INTERVAL` seconds (default 60) * **Payload**: instance ID, flow ID, config version, CLI version, uptime, mode, event counters (delta since last successful heartbeat), plus recent runtime errors and recent log output (redacted at the runner before sending) * **Fire-and-forget**: Failures are logged, never crash the runner. Counter deltas accumulate on failure and are included in the next successful heartbeat. The recent errors and logs let the dashboard surface a deployed flow's runtime errors and log output without any external log tooling. They are captured in bounded in-memory buffers and secrets are redacted before they leave the runner. Runners appear in the project dashboard after their first heartbeat. Building a custom image ``` cd packages/cli npm run docker:build:flow ``` Pin a specific CLI version: ``` docker build --build-arg CLI_VERSION=1.3.0 -t walkeros/flow:1.3.0 -f Dockerfile . ``` ## File paths at runtime[​](#file-paths-at-runtime "Direct link to File paths at runtime") When the runner loads a bundle, it sets the working directory (`process.cwd()`) to the **bundle's directory**. All file paths in your flow config settings resolve relative to the bundle, not your project root. For example, if your bundle is at `dist/bundle.mjs` and a transformer reads a file: ``` { "settings": { "filePath": "./data.json" } } ``` At runtime, this resolves to `dist/data.json`, not `./data.json` from the project root. Use the `include` field in your flow config to copy files alongside the bundle so they're accessible at runtime. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **Runner won't start**: Check the error message. Invalid env var combinations produce specific errors like `FLOW_ID requires WALKEROS_TOKEN and PROJECT_ID`. **Runner exits with "Secrets fetch failed (auth)"**: The API token is invalid or expired. Generate a new token in the walkerOS dashboard. Auth failures (401/403) are fatal because config fetch would also fail. **"Warning: Could not fetch secrets" but runner continues**: A non-auth error (404, 500) occurred. The runner continues without secrets. If your flow uses `$env.*` references, those values will be missing or fall back to their defaults. **API fetch fails at startup**: The runner falls back to the cached bundle. If no cache exists, it exits. Mount `CACHE_DIR` as a volume for resilience. **Hot-swap not working**: Verify `FLOW_ID` is set (polling only runs in mode C/D). Check logs for `Config unchanged` or `Poll error`. Confirm the flow was updated in the walkerOS UI. **Hot-swap skipped with "Failed to refresh secrets"**: A transient secrets fetch error during polling skips the swap. The runner keeps the current flow and retries on the next poll. Check API connectivity and token validity. **Bundle fails after config fetch**: The runner continues with the current flow and logs the error. The cached bundle is not overwritten. ## Next steps[​](#next-steps "Direct link to Next steps") * **[Docker runtime](/docs/apps/docker.md)**: Pre-built bundle deployment * **[CLI](/docs/apps/cli.md)**: Build and test flows locally * **[Flow configuration](/docs/getting-started/modes/bundled.md)**: Flow\.Json format reference --- # Storybook addon The walkerOS Storybook addon integrates event tracking visualization into Storybook. It enables developers to inspect walker attributes, monitor real-time events, and visually highlight tagged elements while building components. ## Installation[​](#installation "Direct link to Installation") ``` npm install --save-dev @walkeros/storybook-addon ``` ## Setup[​](#setup "Direct link to Setup") Register the addon in your Storybook configuration: ``` // .storybook/main.ts const config: StorybookConfig = { addons: ['@walkeros/storybook-addon'], }; export default config; ``` ## Configuration[​](#configuration "Direct link to Configuration") Optionally customize the addon behavior in your preview file: ``` // .storybook/preview.ts const preview: Preview = { parameters: { walkerOS: { prefix: 'data-elb', // Custom data attribute prefix autoRefresh: true, // Auto-refresh on story changes }, }, }; export default preview; ``` | Option | Type | Default | Description | | ------------- | --------- | ------------ | ---------------------------------------------------------- | | `prefix` | `string` | `'data-elb'` | Data attribute prefix for walker attributes | | `autoRefresh` | `boolean` | `true` | Automatically refresh events when story or controls change | ## Features[​](#features "Direct link to Features") The addon panel provides three tabs for inspecting walkerOS data: ### Events tab[​](#events-tab "Direct link to Events tab") Displays all detected walkerOS events in the current story's DOM. Click "Update events" to manually refresh the event list. ### Live events tab[​](#live-events-tab "Direct link to Live events tab") Captures events in real-time as you interact with components. Shows timestamps for each event and stores up to 50 events. Use the "Clear" button to reset. ### Skeleton tab[​](#skeleton-tab "Direct link to Skeleton tab") Shows a hierarchical tree of the entities found in the DOM, with each entity's properties resolved onto it the way the walker reads them: * Generic (`data-elb-`) and scoped (`data-elb_`) values are shown on the entities that consume them, labeled by origin (`G` for generic, `S` for scoped; directly declared properties are unlabeled) * Expandable/collapsible entity structure * Click to view element markup * Attribute counts by type ### Visual highlighting[​](#visual-highlighting "Direct link to Visual highlighting") Toggle buttons highlight elements with different attribute types: | Attribute | Color | Data Attribute | | --------- | ------ | ------------------------- | | Globals | Cyan | `data-elbglobals` | | Context | Yellow | `data-elbcontext` | | Entity | Green | `data-elb` | | Property | Red | `data-elb-*`, `data-elb_` | | Action | Purple | `data-elbaction` | Multiple highlights can be active simultaneously, displaying layered outlines. ## Usage in stories[​](#usage-in-stories "Direct link to Usage in stories") Import `dataElbArgTypes` to add walkerOS controls to your stories: ``` import { dataElbArgTypes } from '@walkeros/storybook-addon'; export default { title: 'Components/Button', argTypes: { ...dataElbArgTypes, }, }; ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **No events detected:** * Verify components have `data-elb` attributes * Check the prefix matches your configuration * Click "Update events" to manually refresh **Highlights not showing:** * Ensure the story iframe has loaded * Toggle highlights off and on again * Check browser console for errors ## Next steps[​](#next-steps "Direct link to Next steps") * **[Storybook Demo](https://storybook.walkeros.io/)**: See the addon in action * **[HTML Attributes](/docs/sources/web/browser/tagging/html-attributes.md)**: Learn walker attribute syntax * **[Browser Source](/docs/sources/web/browser/.md)**: Understand DOM event capture --- # Telemetry walkerOS `@walkeros/cli` and `@walkeros/mcp` support anonymous usage telemetry to help us improve the tools. Telemetry is **off by default** and requires explicit opt-in. Nothing is sent from your machine until you run `walkeros telemetry enable`. This page describes the data contract, how to enable or disable it, and the legal basis. Status In the current release, no ingest endpoint is shipped. Opting in records your consent locally; actual data collection begins in a future release and will continue to honor your stored choice. ## Why opt-in[​](#why-opt-in "Direct link to Why opt-in") walkerOS is a privacy-aware data-collection product built in the EU. The ePrivacy Directive (Article 5(3)) requires prior consent for any storing of or access to information on a user's terminal equipment that is not strictly necessary for the service the user asked for. Installing a persistent installation UUID on your machine to help us improve the tools is not strictly necessary to run `walkeros bundle`, so we ask first and we default off. This is the same consent model we encourage on the web. ## The three states[​](#the-three-states "Direct link to The three states") Your choice is stored in `~/.config/walkeros/config.json` under `telemetryEnabled`: | State | Meaning | | ------------ | ------------------------------------------------------------------------------------------------------------------------- | | field absent | You have not made a choice yet. Default: nothing is collected, no installation UUID is written. | | `true` | You opted in via `walkeros telemetry enable`. An installation UUID is generated at that moment and the emitter is active. | | `false` | You opted out via `walkeros telemetry disable`. Nothing is collected. | ## Commands[​](#commands "Direct link to Commands") ``` # Opt in walkeros telemetry enable # Opt out walkeros telemetry disable # Show current state walkeros telemetry status ``` ## What we would collect (after opt-in)[​](#what-we-would-collect-after-opt-in "Direct link to What we would collect (after opt-in)") Every event is an `entity action` (the same event model the rest of walkerOS uses). The emitted events: | Event | When | | ------------- | ------------------------------------------------------------- | | `mcp start` | MCP server starts up and completes the `initialize` handshake | | `cmd invoke` | A CLI command or MCP tool is invoked | | `error throw` | An unhandled error is thrown during a command or tool call | For each event we attach: * **Installation UUID**: generated at opt-in, stored in `~/.config/walkeros/config.json`, not linked to any account or user. * **Environment**: OS, OS version, Node.js version, language, timezone, whether the process is running in CI. * **Command or tool name**: for example `bundle`, `push`, `flow_validate`. * **Outcome**: success or error. Never the error message. * **Duration**: how long the command or tool call took, in milliseconds. * **Client version**: the installed `@walkeros/cli` or `@walkeros/mcp` version. ## What we do NOT collect[​](#what-we-do-not-collect "Direct link to What we do NOT collect") * File paths, file names, or file contents * Environment variable names or values * Error messages, stack traces, or code snippets * Project names, flow names, destination settings, or any flow config content * Account identifiers, email addresses, or auth tokens * IP addresses ## Source of truth[​](#source-of-truth "Direct link to Source of truth") The full contract, with JSON Schemas for every event and every property, lives in the CLI source tree: [`packages/cli/src/telemetry/flow.json`](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/src/telemetry/flow.json) If it is not declared there, we do not emit it. ## Headless environments[​](#headless-environments "Direct link to Headless environments") MCP servers, CI jobs, piped invocations, and any other non-interactive process inherit the stored choice but cannot prompt. In practice: * If you have opted in via `walkeros telemetry enable` on the same machine, MCP and CI invocations emit telemetry normally. * If you have never opted in on the machine, those surfaces stay silent. There is no environment variable to force-enable telemetry on a machine that has not opted in. That is intentional: consent must be an explicit act. ## Env var overrides[​](#env-var-overrides "Direct link to Env var overrides") Two legacy env vars force telemetry off regardless of your stored choice: ``` export DO_NOT_TRACK=1 export WALKEROS_TELEMETRY_DISABLED=1 ``` These are honored for backward compatibility and for users who want a belt-and-braces guarantee even after opting in. They are not a substitute for opt-in: telemetry is already off by default. ## Debug mode[​](#debug-mode "Direct link to Debug mode") To inspect exactly what would be sent after opt-in, set: ``` export WALKEROS_TELEMETRY_DEBUG=1 ``` In debug mode the CLI and MCP print each telemetry payload to `stderr` as JSON and do not send anything to the network. ## Where it goes (future)[​](#where-it-goes-future "Direct link to Where it goes (future)") When an ingest endpoint ships, telemetry events will be sent to the walkerOS managed endpoint operated by elbwalker GmbH on EU infrastructure. We will not share the data with any third party and we will not use it for advertising. It exists to help us prioritize fixes and improvements to the open-source tools. Until then, opting in records your consent locally and emits nothing. --- # walker.js Walker.js is a pre-built walkerOS application that combines both the [browser](/docs/sources/web/browser/.md) and [dataLayer](/docs/sources/web/dataLayer/.md) sources with the [collector](/docs/collector/.md) and a default `dataLayer` destination into a pre-build package. It's designed for users who want instant web tracking without complex setup or configuration. ## Installation[​](#installation "Direct link to Installation") ### Option 1: npm package[​](#option-1-npm-package "Direct link to Option 1: npm package") ``` npm install @walkeros/walker.js ``` ### Option 2: CDN[​](#option-2-cdn "Direct link to Option 2: CDN") ``` ``` ## Basic setup[​](#basic-setup "Direct link to Basic setup") ### 1. Add event queueing (recommended)[​](#1-add-event-queueing-recommended "Direct link to 1. Add event queueing (recommended)") Add this script before walker.js loads to queue events during initialization: ``` ``` ### 2. Include walker.js[​](#2-include-walkerjs "Direct link to 2. Include walker.js") ``` ``` ### 3. Configure destinations[​](#3-configure-destinations "Direct link to 3. Configure destinations") ``` ``` ## Configuration options[​](#configuration-options "Direct link to Configuration options") Walker.js supports multiple configuration approaches with different priorities: 1. **Script tag `data-elbconfig`** (highest priority) 2. **`window.elbConfig`** (default fallback) 3. **Manual initialization** (when `run: false`) ### Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | ---------------- | ----------------------------------------------------------------------- | ---- | | `collector` | `any` | Collector configuration (Collector.InitConfig) | | | `browser` | `any` | Browser source configuration (Partial\) | | | `dataLayer` | `boolean \| any` | DataLayer configuration (boolean \| Partial\) | | | `elb` | `string` | Name for the global elb function (default: "elb") | | | `name` | `string` | Name for the global instance | | | `run` | `boolean` | Auto-run on initialization (default: true) | | #### Browser source settings[​](#browser-source-settings "Direct link to Browser source settings") | Property | Type | Description | More | | ---------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `prefix` | `string` | Prefix for data attributes (default: data-elb) | | | `scope` | `string` | DOM scope for event tracking (default: document) | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `capture` | `boolean` | Read click/submit triggers in the capture phase so tagged elements are resolved at click time before app handlers run (set false for the previous bubble-phase behavior) | | | `elb` | `string` | Name for global elb function | | | `name` | `string` | Custom name for source instance | | | `elbLayer` | `boolean \| string \| any` | Enable elbLayer for async command queuing (boolean, string, or Elb.Layer) | | #### DataLayer settings[​](#datalayer-settings "Direct link to DataLayer settings") | Property | Type | Description | More | | -------- | ---------- | ------------------------------------------------------------------------ | ---- | | `name` | `string` | DataLayer variable name (default: dataLayer) | | | `prefix` | `string` | Event prefix for filtering which events to process | | | `filter` | `function` | Custom filter function: (event: unknown) => boolean \| Promise\ | | #### Collector settings[​](#collector-settings "Direct link to Collector settings") | Property | Type | Description | More | | --------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------- | ---- | | `run` | `boolean` | Whether to run collector automatically on initialization | | | `globalsStatic` | `WalkerOS.Properties` | Static global properties that persist across collector runs | | | `sessionStatic` | `Collector.SessionStatic` | Static session data that persists across collector runs | | | `logger` | `Logger.Config` | Logger configuration (level, handler) | | | `queueMax` | `number` | Maximum events retained in collector.queue (late-registration replay). FIFO drop on overflow. Default 1000. | | | `name` | `string` | Flow name; keys this flow's entry in event.source.release and the observer flowId. | | | `release` | `string` | Config release id stamped into event.source.release for this flow. | | | `consent` | `WalkerOS.Consent` | Initial consent state | | | `user` | `any` | Initial user data | | | `globals` | `WalkerOS.Properties` | Initial global properties | | | `sources` | `Source.InitSources` | Source configurations | | | `destinations` | `Destination.InitDestinations` | Destination configurations | | | `transformers` | `Transformer.Configs` | Transformer configurations | | | `stores` | `Store.Configs` | Store configurations | | | `custom` | `WalkerOS.Properties` | Initial custom implementation-specific properties | | | `hooks` | `Collector.Hooks` | Pipeline observation hooks | | ### Full configuration object[​](#full-configuration-object "Direct link to Full configuration object") ``` window.elbConfig = { // Global settings elb: 'elb', // Global function name (default: 'elb') name: 'walkerjs', // Global instance name run: true, // Auto-initialize (default: true) // Browser source settings browser: { run: true, // Auto-start DOM tracking session: true, // Enable session tracking scope: document.body, // Tracking scope pageview: true, // Enable automatic page views }, // DataLayer integration dataLayer: true, // Enable dataLayer // or detailed config: // dataLayer: { // name: 'dataLayer', // DataLayer variable name // prefix: 'dataLayer', // Event prefix // }, // Collector configuration collector: { consent: { functional: true }, // Default consent state destinations: { // Your destinations here console: { push: (event) => console.log('Event:', event), }, }, }, }; ``` ### Inline configuration[​](#inline-configuration "Direct link to Inline configuration") Configure directly in the script tag using simple key:value pairs: ``` ``` ### Named configuration object[​](#named-configuration-object "Direct link to Named configuration object") Use a custom configuration object name: ``` ``` ## Usage[​](#usage "Direct link to Usage") ### Automatic DOM tracking[​](#automatic-dom-tracking "Direct link to Automatic DOM tracking") Walker.js automatically tracks events based on HTML data attributes: ```
``` For detailed information on data attributes, see the [Browser Source documentation](/docs/sources/web/browser/tagging/html-attributes.md). ### Manual event tracking[​](#manual-event-tracking "Direct link to Manual event tracking") Use the global `elb` function for manual tracking: ``` // Simple event elb('button click', { label: 'interesting', }); ``` ### DataLayer integration[​](#datalayer-integration "Direct link to DataLayer integration") Walker.js can integrate with existing dataLayer implementations: ``` // Enable dataLayer integration window.elbConfig = { dataLayer: true, // Uses window.dataLayer by default }; // Existing dataLayer events will be processed dataLayer.push({ event: 'purchase', ecommerce: { transaction_id: '12345', value: 25.42, }, }); ``` ## Advanced features[​](#advanced-features "Direct link to Advanced features") ### Async loading and event queueing[​](#async-loading-and-event-queueing "Direct link to Async loading and event queueing") Walker.js handles async loading gracefully with automatic event queueing: ``` ``` ### Build variants[​](#build-variants "Direct link to Build variants") Walker.js provides multiple build formats for different environments: * `walker.js` - Standard IIFE bundle for browsers * `index.es5.js` - GTM-compatible ES2015 build * `index.mjs` - ES modules for modern bundlers * `index.js` - CommonJS for Node.js environments ### Programmatic usage[​](#programmatic-usage "Direct link to Programmatic usage") Use walker.js programmatically in applications: ``` import { createWalkerjs } from '@walkeros/walker.js'; const { collector, elb } = await createWalkerjs({ collector: { destinations: { console: { push: console.log }, }, }, browser: { session: true, pageview: true, }, }); ``` ## Destination configuration[​](#destination-configuration "Direct link to Destination configuration") Configure multiple destinations for your events: ``` window.elbConfig = { collector: { destinations: { // Console logging for development console: { push: (event) => console.log('Walker.js Event:', event), }, // Custom API endpoint api: { push: async (event) => { await fetch('/api/events', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(event), }); }, }, }, }, }; ``` For comprehensive destination options, see the [Destinations documentation](/docs/destinations/.md). ## Growing beyond walker.js[​](#growing-beyond-walkerjs "Direct link to Growing beyond walker.js") Walker.js is a **convenience bundle** that combines Browser Source, DataLayer Source, and Collector into a single file. It's perfect for getting started, but you may outgrow it. ### When to stay with walker.js[​](#when-to-stay-with-walkerjs "Direct link to When to stay with walker.js") * ✅ Simple tracking needs (pageviews, clicks, forms) * ✅ Static sites or traditional server-rendered pages * ✅ Quick proof-of-concept or MVP * ✅ Teams who prefer minimal setup ### When to upgrade[​](#when-to-upgrade "Direct link to When to upgrade") Consider moving to full [Integrated](/docs/getting-started/modes/integrated.md) or [Bundled](/docs/getting-started/modes/bundled.md) mode when you need: | Need | Solution | | ------------------------------- | --------------------------------- | | Server-side tracking | Bundled mode with Docker | | Custom sources (HTTP, AWS, GCP) | Both modes support server sources | | Multiple flows per page | Full Collector configuration | | TypeScript integration | Integrated mode | | CI/CD versioned configs | Bundled mode | | Event mapping/transformation | Both modes with mapping config | ### Migration path[​](#migration-path "Direct link to Migration path") Walker.js configuration maps directly to full modes: * Integrated * Bundled ``` // walker.js equivalent in Integrated mode import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true } }, }, }, destinations: { // Your destinations here }, run: true, }); ``` ``` // walker.js equivalent in Bundled mode (flow.json) { "version": 4, "flows": { "default": { "config": { "platform": "web" }, "sources": { "browser": { "package": "@walkeros/web-source-browser", "config": { "settings": { "pageview": true, "session": true } } } }, "destinations": { // Your destinations here }, "collector": { "run": true } } } } ``` The mental model is the same (sources, collector, destinations), just with more control. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common issues[​](#common-issues "Direct link to Common issues") **Events not firing:** Check that walker.js loaded and configuration is valid. **Missing events:** Ensure event queueing function is added before walker.js. **Configuration not applied:** Verify `data-elbconfig` points to the correct object name. ## API reference[​](#api-reference "Direct link to API reference") ### Factory function[​](#factory-function "Direct link to Factory function") ``` createWalkerjs(config?: Config): Promise ``` Creates a new walker.js instance with the provided configuration. ### Instance properties[​](#instance-properties "Direct link to Instance properties") * `collector` - The walkerOS collector instance * `elb` - Browser push function for event tracking ### Utility functions[​](#utility-functions "Direct link to Utility functions") ``` import { getAllEvents, getEvents, getGlobals } from '@walkeros/walker.js'; // Get all trackable events on the page const events = getAllEvents(); // Get events for a specific element and trigger const button = document.querySelector('button'); const clickEvents = getEvents(button, 'click'); // Get global properties from the page const globals = getGlobals(); ``` ## Related documentation[​](#related-documentation "Direct link to Related documentation") * **[Browser Source](/docs/sources/web/browser/.md)** - Detailed DOM tracking capabilities * **[Collector](/docs/collector/.md)** - Event processing and routing * **[Destinations](/docs/destinations/.md)** - Available destination options * **[DataLayer Source](/docs/sources/web/dataLayer/.md)** - DataLayer integration details Walker.js combines all these components into a single, easy-to-use package perfect for getting started with walkerOS quickly. --- # Collector The collector is the central **processing engine** of walkerOS that receives events from sources, **enriches** them with additional data, applies consent rules, and **routes** them to destinations. It acts as the **intelligent middleware** between event capture and event delivery. #### What it does[​](#what-it-does "Direct link to What it does") The Collector transforms raw events into enriched, compliant data streams by: * **Event processing** - Validates, normalizes, and enriches incoming events * **Consent management** - Applies privacy rules and user consent preferences * **Data enrichment** - Adds session data, user context, and custom properties * **Destination routing** - Sends processed events to configured analytics platforms #### Key features[​](#key-features "Direct link to Key features") * **Compatibility** - Works in both web browsers and server environments * **Privacy-first** - Built-in consent management and data protection * **Event validation** - Ensures data quality and consistency * **Flexible routing** - Send events to multiple destinations simultaneously * **Delivery status** - Built-in per-source and per-destination delivery tracking #### Role in architecture[​](#role-in-architecture "Direct link to Role in architecture") In the walkerOS data flow, the collector sits between sources and destinations: Sources capture events and send them to the collector, which processes and routes them to your chosen destinations like Google Analytics, custom APIs, or data warehouses. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/collector ``` ## Basic setup[​](#basic-setup "Direct link to Basic setup") ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; const { collector, elb } = await startFlow({ consent: { functional: true }, sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, }, }, }, }, destinations: { console: { code: true, // Built-in code destination config: { settings: { push: "console.log('Event:', event)", }, }, }, }, }); ``` ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------- | ---- | | `run` | `boolean` | Whether to run collector automatically on initialization | | | `globalsStatic` | `WalkerOS.Properties` | Static global properties that persist across collector runs | | | `sessionStatic` | `Collector.SessionStatic` | Static session data that persists across collector runs | | | `logger` | `Logger.Config` | Logger configuration (level, handler) | | | `queueMax` | `number` | Maximum events retained in collector.queue (late-registration replay). FIFO drop on overflow. Default 1000. | | | `name` | `string` | Flow name; keys this flow's entry in event.source.release and the observer flowId. | | | `release` | `string` | Config release id stamped into event.source.release for this flow. | | | `consent` | `WalkerOS.Consent` | Initial consent state | | | `user` | `any` | Initial user data | | | `globals` | `WalkerOS.Properties` | Initial global properties | | | `sources` | `Source.InitSources` | Source configurations | | | `destinations` | `Destination.InitDestinations` | Destination configurations | | | `transformers` | `Transformer.Configs` | Transformer configurations | | | `stores` | `Store.Configs` | Store configurations | | | `custom` | `WalkerOS.Properties` | Initial custom implementation-specific properties | | | `hooks` | `Collector.Hooks` | Pipeline observation hooks | | ## Advanced setup[​](#advanced-setup "Direct link to Advanced setup") ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { destinationAPI } from '@walkeros/web-destination-api'; const { collector, elb } = await startFlow({ consent: { functional: true }, sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, }, }, }, }, destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://analytics.example.com/events', }, }, }, }, logger: { level: 'DEBUG', // Show all logs (ERROR, INFO, DEBUG) }, }); ``` ## Event transformation[​](#event-transformation "Direct link to Event transformation") The collector works with **mapping** to transform events as they flow through the system. Mapping is configured at the destination level and controls how walkerOS events are converted to vendor-specific formats. For example, transforming a `product add` event into GA4's `add_to_cart`: ``` destinations: { ga4: { code: destinationGtag, config: { mapping: { product: { add: { name: 'add_to_cart', data: { map: { currency: { value: 'USD' }, value: 'data.price', items: { loop: ['nested', { map: { item_id: 'data.id' } }] } } } } } } } } } ``` [**Learn more about mapping →**](/docs/mapping/.md) ## Status[​](#status "Direct link to Status") The collector tracks delivery metrics on `collector.status`, giving you real-time visibility into event flow without external monitoring: ``` const { collector } = await startFlow({ /* ... */ }); // After events have been processed console.log(collector.status); // { // startedAt: 1707580800000, // in: 100, // Total events received // out: 95, // Total events delivered // failed: 5, // Total failures // sources: { // browser: { count: 100, lastAt: 1707580900000, duration: 1200 } // }, // destinations: { // ga4: { count: 95, failed: 0, lastAt: 1707580900000, duration: 800, // queuePushSize: 0, dlqSize: 0 }, // meta: { count: 0, failed: 5, lastAt: 1707580850000, duration: 400, // queuePushSize: 0, dlqSize: 5 } // }, // // Process-wide drop counters keyed by stepId. See `stepId()` in // // `@walkeros/core`. Entries are created lazily on first drop. // dropped: { // // "collector" is the reserved stepId for the collector-level buffer. // // "destination." keys each destination's drop counters. // // Each entry exposes optional { queue?, dlq? } counts. // } // } // Compute average destination push time const ga4 = collector.status.destinations.ga4; const avgMs = ga4.duration / (ga4.count + ga4.failed); ``` ### What `failed` counts[​](#what-failed-counts "Direct link to what-failed-counts") `collector.status.failed` is the single counter for any walkerOS-internal pipeline failure. Contributing sites: * a destination's `push` threw or returned an error, * an exception escaped the inner pipeline of `collector.push` or `collector.command` (any uncaught error inside the boundary), * mapping outer-wrap failures (an internal throw inside `processMappingValue`), * source startup failures (`code()` threw, `init()` threw, queued-on flush threw), * transformer init failures, * destination init failures. User-supplied callback throws are visibility-only: they log at error level but do NOT increment `status.failed`. This keeps the counter a clean pipeline-health signal. Sites in this group: * `on` subscriptions (`destination.on`, `source.on`, consent rules, `ready`, `run`, `session`, generic), * mapping `condition`, `fn`, and `validate` callbacks. In both cases the collector logs at error level: ``` // destination-side failure logger.scope('').error('Push failed', { error, event }); // collector-side boundary failure logger.error('push failed', { event, ingest, error }); logger.error('command failed', { command, data, error }); // pipeline-internal failures (counted) logger.error('mapping processing failed', { event, error }); logger.scope('source').error('source factory failed', { sourceId, error }); logger.scope('source').error('source init failed', { sourceId, error }); logger.scope('source').error('source on flush failed', { sourceId, type, error }); logger.scope('transformer:').error('transformer init failed', { transformer, error }); logger.scope('').error('destination init failed', { error }); // user-callback failures (logged, NOT counted) logger.error('mapping condition failed', { event, error }); logger.error('mapping fn failed', { event, error }); logger.error('mapping validate failed', { event, error }); logger.scope('on').error('on callback failed', { kind, error }); ``` A source whose `init()` throws stays with `config.init === false` instead of being marked initialized. Operators reading `source.config.init` see the source visibly stuck, not falsely healthy. Alarm on the ratio of `status.failed` to `status.in`. The log message text plus structured fields are enough to filter destination vs boundary failures downstream. **PII note.** Boundary error logs include the full failing event payload so operators can reproduce. If your event payloads carry sensitive data, configure redaction at the logger layer; do not parse and rewrite at every call site. ### Fatal errors[​](#fatal-errors "Direct link to Fatal errors") Throw `FatalError` (exported from `@walkeros/core`) for invariant violations or operator-initiated aborts that must crash the host process. Standard `Error` is absorbed by the boundary catch, logged, and counted. `FatalError` bypasses the catch and propagates, so a supervisor (CLI runner, Express server, container orchestrator) can terminate cleanly. ``` import { FatalError } from '@walkeros/core'; if (!config.apiKey) { throw new FatalError('apiKey missing — refusing to start'); } ``` Queue sizes and DLQ sizes can be read directly from destination instances, or from the status snapshot: ``` const dest = collector.destinations['meta']; const queueSize = dest.queuePush?.length || 0; const dlqSize = dest.dlq?.length || 0; // Or from status (point-in-time snapshots refreshed after each push pass): const metaStatus = collector.status.destinations['meta']; metaStatus.queuePushSize; // events waiting for consent metaStatus.dlqSize; // failed pushes retained for triage // Drop counts live on the process-wide `status.dropped` map, keyed by // stepId (use `stepId()` from `@walkeros/core`): import { stepId } from '@walkeros/core'; const metaDrops = collector.status.dropped[stepId('destination', 'meta')]; metaDrops?.queue ?? 0; // monotonic count of evicted consent-queued events metaDrops?.dlq ?? 0; // monotonic count of evicted DLQ entries ``` ## Buffer bounds[​](#buffer-bounds "Direct link to Buffer bounds") The collector keeps three internal buffers per process. Each is size-bounded with a configurable cap; on overflow the oldest entries are evicted (FIFO), the corresponding `dropped` counter is incremented, and a warning is logged once per overflow window. | Buffer | Purpose | Default cap | Config | | ----------------------- | -------------------------------------------------- | ----------- | ----------------------------- | | `collector.queue` | Replay buffer for late-registered destinations | 1000 | `Collector.Config.queueMax` | | `destination.queuePush` | Per-destination consent-denied buffer | 1000 | `Destination.Config.queueMax` | | `destination.dlq` | Per-destination dead-letter queue of failed pushes | 100 | `Destination.Config.dlqMax` | Each step has its own knob; no cascade. Set the collector cap on the collector, and per-destination caps on each destination: ``` const { collector } = await startFlow({ // Collector replay buffer cap queueMax: 5_000, destinations: { bigquery: { code: bigqueryDestination, config: { // Keep more failed rows for triage on this destination dlqMax: 1_000, }, }, }, }); ``` Operators alarm on the `dropped` counters to detect sustained overflow. Counters live on `collector.status.dropped`, keyed by stepId (build the key with `stepId()` from `@walkeros/core`): ``` import { stepId } from '@walkeros/core'; // Collector replay buffer drops: traffic bursts outrunning destination // registration. const collectorDrops = collector.status.dropped[stepId('collector')]?.queue ?? 0; // Destination drops by buffer: // - queue: consent-denied events evicted from the destination's queuePush // - dlq: failed-push entries evicted from the destination's DLQ const ga4Drops = collector.status.dropped[stepId('destination', 'ga4')]; ga4Drops?.queue ?? 0; ga4Drops?.dlq ?? 0; // sustained non-zero rate signals destination outage ``` *** ## See also[​](#see-also "Direct link to See also") * [**Operating modes**](/docs/getting-started/modes/.md): integrated vs bundled approaches * [**CLI documentation**](/docs/apps/cli.md): configure with JSON and build with CLI * [**Sources**](/docs/sources/.md): available event capture sources * [**Destinations**](/docs/destinations/.md): available event delivery destinations --- # Cache Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests. ## Configuration[​](#configuration "Direct link to Configuration") Add `cache` to any step in your flow config: ``` "sources": { "express": { "package": "@walkeros/server-source-express", "cache": { "rules": [ { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "key": ["ingest.method", "ingest.path"], "ttl": 300 } ] } } } ``` ### Cache Rule Properties[​](#cache-rule-properties "Direct link to Cache Rule Properties") | Property | Type | Description | | -------- | ----------------------- | -------------------------------------- | | `match` | `MatchExpression` | When to apply this cache rule | | `key` | `string[]` | Dot-paths to build the cache key from | | `ttl` | `number` | Time-to-live in seconds | | `update` | `Record` | Optional modifications applied on read | `match` is optional, omit it for an always-match rule. ### Cache Properties[​](#cache-properties "Direct link to Cache Properties") | Property | Type | Default | Description | | ----------- | ------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `stop` | `boolean` | `false` (`true` for sources) | Stop flow on cache HIT and skip remaining steps, returning the cached value | | `store` | `string` | Built-in memory | Reference to a store in `flow.stores` | | `namespace` | `string` | *(none)* | Optional key prefix. Omit to write keys directly to the store. Same store + same key + same namespace = same cache entry. | | `rules` | `CacheRule[]` | Required | At least one cache rule | ## Behavior by Step Type[​](#behavior-by-step-type "Direct link to Behavior by Step Type") ### Source Cache (stop by default)[​](#source-cache-stop-by-default "Direct link to Source Cache (stop by default)") Source cache intercepts requests before the pipeline runs. Sources default to `stop: true`. On cache HIT, the cached respond value is returned immediately and the pipeline is skipped. Set `stop: false` to skip only the source step and let the pipeline continue. ``` "sources": { "express": { "cache": { "rules": [ { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "key": ["ingest.method", "ingest.path"], "ttl": 300 } ] } } } ``` ### Transformer Cache (step-level)[​](#transformer-cache-step-level "Direct link to Transformer Cache (step-level)") Transformer cache skips the transformer's execution on HIT but continues the chain with the cached event. Useful for expensive computations like API enrichment. With `stop: true`, the remaining chain stops on HIT and downstream transformers and destinations are skipped. Halt scopes Pre-collector transformer caches with `stop: true` halt the entire pipeline on HIT, destinations do not see the event. Destination caches with `stop: true` halt only the per-destination delivery, sibling destinations still receive the event. ``` "transformers": { "enricher": { "package": "@walkeros/server-transformer-enricher", "cache": { "rules": [ { "key": ["event.name"], "ttl": 60 } ] } } } ``` ### Destination Cache (deduplication)[​](#destination-cache-deduplication "Direct link to Destination Cache (deduplication)") Destination cache skips the push on HIT, preventing duplicate events from reaching the destination. Other destinations still receive the event. With `stop: true`, the `before` transformer chain is also skipped on HIT. ``` "destinations": { "ga4": { "package": "@walkeros/web-destination-ga4", "cache": { "rules": [ { "key": ["event.name", "event.data.id"], "ttl": 300 } ] } } } ``` ## Key Resolution[​](#key-resolution "Direct link to Key Resolution") Cache keys use dot-path resolution from a structured context: * `ingest.method`: request method from source ingest * `ingest.path`: request path from source ingest * `event.name`: event name * `event.data.id`: nested event data field * `event.user.loggedIn`: user properties ## Update Rules[​](#update-rules "Direct link to Update Rules") Use `update` to modify the result on both HIT and MISS: ``` "cache": { "rules": [{ "key": ["ingest.method", "ingest.path"], "ttl": 300, "update": { "headers.X-Cache": { "key": "cache.status" }, "headers.Cache-Control": { "value": "public, max-age=300" } } }] } ``` `cache.status` resolves to `"HIT"` or `"MISS"`. Static values use `{ "value": "..." }`. ## Custom Store[​](#custom-store "Direct link to Custom Store") By default, cache uses a shared in-memory store. Reference a custom store for persistence: ``` { "stores": { "redis": { "package": "@walkeros/server-store-redis" } }, "sources": { "express": { "cache": { "store": "redis", "rules": [...] } } } } ``` Custom stores can be either synchronous (in-memory) or asynchronous (file system, Redis, HTTP, the cache wrapper). Cache lookups await the underlying `store.get` internally, so HIT/MISS semantics behave identically regardless of which backing you wire. ## Migration from prior v4 cache[​](#migration-from-prior-v4-cache "Direct link to Migration from prior v4 cache") If you're upgrading from an earlier v4 build, three breaking changes apply: * **`cache.full` is now `cache.stop`.** Search-and-replace `"full"` for `"stop"` in every `cache` block. Default behavior unchanged (sources default to `stop: true`, transformers and destinations default to `stop: false`). * **`cache.match: "*"` is now omitted.** Drop the `"match": "*"` entry from rules. A rule with no `match` matches every event, the behavior is identical to the previous wildcard. * **Implicit per-step namespacing is gone.** Previous builds prefixed cache keys with `s:`, `t:`, or `d:` based on step type, so a source and a destination writing the same key never collided. That prefix is no longer applied. If you share a store across step types and need isolation, set `cache.namespace` explicitly (for example `"namespace": "source"`). --- # Collector commands The collector provides a core event processing engine that manages destinations, consent, user data, and custom properties. Commands are executed through the `elb` function, which the collector also exposes as `collector.elb`. Unknown walker commands log a warning and return `{ ok: false }`. Commands like `walker init` are browser-specific and handled by the browser source before they reach the collector. note For browser-specific commands like DOM initialization and elbLayer communication, see [Browser Source Commands](/docs/sources/web/browser/commands.md). ## destination[​](#destination "Direct link to destination") Add destinations to the collector. The recommended approach is to configure destinations during initialization with `startFlow()`: ``` import { startFlow } from '@walkeros/collector'; import { destinationGtag } from '@walkeros/web-destination-gtag'; const { collector, elb } = await startFlow({ destinations: { gtag: { code: destinationGtag, config: { settings: { /* custom config */ }, }, }, }, }); ``` tip For dynamic scenarios requiring runtime destination addition, use `elb('walker destination')` command or `collector.addDestination()`. See destination-specific documentation for configuration options. ## consent[​](#consent "Direct link to consent") Manage consent states for the collector. Names can be defined arbitrarily, but common groups are functional, analytics, and marketing. Values are booleans, and once a value is set to `true` it's treated as consent being granted. ``` elb('walker consent', { marketing: true, analytics: true }); ``` Setting a consent state to `false` will immediately stop a destination from processing any events. Previously pushed events during the run are shared with existing destinations once consent is granted. info Learn more about [consent management](/docs/guides/consent/.md) in detail. ## on[​](#on "Direct link to on") Add event listeners to the collector. They get called when specific events occur like `run` or `consent` changes. ``` elb('walker on', { type, rules }); ``` `rules` depends on `type` and can also be an array for multiple listeners at once. Every callback receives `(data, context)` where `context` exposes `collector` and `logger`. ### Callback signatures[​](#callback-signatures "Direct link to Callback signatures") | Action | Callback signature | | ------- | -------------------------------------------------------- | | consent | `{ [key]: (consent, context) => void \| Promise }` | | ready | `(_, context) => void \| Promise` | | run | `(_, context) => void \| Promise` | | session | `(session, context) => void \| Promise` | | user | `(user, context) => void \| Promise` | | (other) | `(data, context) => void \| Promise` | The `context` object has the same shape for every action: | Field | Type | Description | | ----------- | -------------------- | ---------------------------------------- | | `collector` | `Collector.Instance` | Active collector, use for `push`/`queue` | | `logger` | `Logger.Instance` | Use for `info`/`warn`/`error`/`debug` | ### run[​](#run "Direct link to run") With each `run`, the on-event will be called. Use `context.collector` to access the running instance. ``` elb('walker on', { type: 'run', rules: function (_, context) { console.log('run with', { instance: context.collector }); }, }); ``` Every time the `run` command is called, the function will be executed: ``` // Setup collector with browser source const { collector } = await startFlow({ run: true }); // Output: run with { instance: { ... } } elb('walker run'); // Output: run with { instance: { ... } } ``` ### consent[​](#consent-1 "Direct link to consent") Every time the `consent` changes, the rules-matching function(s) will be called with `consent` as the first argument and `context` (carrying `collector` and `logger`) as the second. ``` function onConsent(consent, context) { console.log('consent with', { consent, instance: context.collector }); if (consent.marketing) context.collector.push('walker user', readFromStorage()); } elb('walker on', { type: 'consent', rules: { marketing: onConsent }, }); ``` The `onConsent` function will only be called when the `marketing` consent changes: ``` elb('walker consent', { functional: true }); // Won't trigger the onConsent function elb('walker consent', { marketing: true }); // Will trigger the onConsent function ``` ## user[​](#user "Direct link to user") Set user identification data for the collector. There are three levels: user (company's internal ID), device (longer-term identifier), and session (temporary identification). ``` elb('walker user', { id: 'us3r', device: 'c00k13', session: 's3ss10n' }); ``` User IDs are added to each event. ``` { "event": "entity action", "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" } // other properties omitted } ``` warning Use fully anonymized & arbitrary IDs by default and check your options with persistent user IDs with your data protection officer. tip Learn more about identification and [user stitching](/docs/guides/user-stitching.md) You can also set user identity declaratively from the DOM with the `data-elbuser` attribute. See the browser source's [HTML attributes](/docs/sources/web/browser/tagging/html-attributes.md#user). ## custom[​](#custom "Direct link to custom") Set custom properties that are added to each event processed by the collector. ``` elb('walker custom', { key: 'value' }); ``` ## globals[​](#globals "Direct link to globals") Set global properties that are added to each event processed by the collector. ``` elb('walker globals', { key: 'value' }); ``` ## State delivery[​](#state-delivery "Direct link to State delivery") State commands (`consent`, `user`, `globals`, `custom`) set values on the collector. The collector records every state change immediately, even when it arrives before `run`. Side-effecting delivery to subscribers (the `on` callbacks above, and source `on` handlers) happens at or after `run`: when the collector starts, it delivers the current state once to every subscriber that has not yet seen it. This gives three practical guarantees: * **Exactly-once.** Each subscriber is invoked once per state change. The collector tracks what each subscriber has already received, so re-running or re-registering never double-fires a reaction. * **Order-independent.** A consent-gated reaction fires correctly whether the state was set before or after `run`, and regardless of the order in which sources initialize. You don't have to set state and start the collector in a particular sequence. * **`require` is a timing hint, not a correctness dependency.** A source's `require` only delays when its first `on` delivery lands. A dependent source reacts to state correctly whether or not it declares `require`. This means sources don't need their own deduplication for state deliveries: the collector enforces exactly-once. ## hook[​](#hook "Direct link to hook") Hooks customize the default behavior of the collector. Available hooks include `Push`, `DestinationInit`, `DestinationPush`, `StoreGet`, `StoreSet`, and `StoreDelete`. Hooks allow for validation, manipulation, or cancellation of default behavior. Add hooks to the collector to customize or enhance default processing. ``` elb('walker hook', { name: '', fn: hookFn }); ``` ### Moments[​](#moments "Direct link to Moments") The overall function execution order is as follows: 1. prePush 2. preDestinationInit 3. postDestinationInit 4. preDestinationPush or preDestinationPushBatch 5. postDestinationPush or postDestinationPushBatch 6. postPush Others are: * preSessionStart * postSessionStart * preStoreGet / postStoreGet * preStoreSet / postStoreSet * preStoreDelete / postStoreDelete ### Function signatures[​](#function-signatures "Direct link to Function signatures") In general, `params` will be prefixed as a parameter, containing `fn` which is the original function and `result` for the post-hooks. Use the following function signatures: ``` // Push function prePush(params, event, data, options, context, nested) { return params.fn(event, data, options, context, nested); } function postPush(params, event, data, trigger, context, nested) { console.log('default return result', params.result); return; } // DestinationInit function preDestinationInit(params, config) { return params.fn(config); } function postDestinationInit(params, config) { console.log('default return result', params.result); return params.result; } // DestinationPush function preDestinationPush(params, event, config, mapping, runState) { console.log('default return result', params.result); return params.fn(event, config, mapping, runState); } function postDestinationPush(params) { // Custom code with a void return return; } // DestinationPushBatch function preDestinationPushBatch(params, event, config, mapping, runState) { console.log('default return result', params.result); return params.fn(event, config, mapping, runState); } function postDestinationPushBatch(params) { // Custom code with a void return return; } // StoreGet function preStoreGet(params, key) { return params.fn(key); } function postStoreGet(params, key) { return params.result; } // StoreSet function preStoreSet(params, key, value, ttl) { return params.fn(key, value, ttl); } function postStoreSet(params, key, value, ttl) { return params.result; } // StoreDelete function preStoreDelete(params, key) { return params.fn(key); } function postStoreDelete(params, key) { return params.result; } ``` ### Adding a hook[​](#adding-a-hook "Direct link to Adding a hook") Add hooks during collector initialization or via the `hook` command: ``` // Add hooks during initialization const { collector } = await startFlow({ hooks: { prePush: (params, ...args) => { window.elbTimer = Date.now(); return params.fn(...args); }, }, }); // Add hooks via command elb('walker hook', { name: 'postPush', fn: function (params, ...args) { console.log('walker exec time', Date.now() - window.elbTimer); }, }); elb('entity action'); // Output: // walker exec time 1 ``` --- # Logger walkerOS provides a centralized logging system that enables consistent logging across the collector, sources, and destinations. The logger supports four log levels and provides scoped logging for better debugging. ## Configuration[​](#configuration "Direct link to Configuration") Configure logging when initializing the collector: ``` import { startFlow } from '@walkeros/collector'; const { collector, elb } = await startFlow({ logger: { level: 'DEBUG', // 'ERROR' | 'WARN' | 'INFO' | 'DEBUG' }, }); ``` ### Log levels[​](#log-levels "Direct link to Log levels") | Level | Value | Description | | ------- | ----- | ---------------------------------------- | | `ERROR` | 0 | Only errors (default) | | `WARN` | 1 | Errors and warnings | | `INFO` | 2 | Errors, warnings, and informational | | `DEBUG` | 3 | All messages including debug information | ## Using the logger[​](#using-the-logger "Direct link to Using the logger") The logger is automatically available in destination and source contexts: ### In destinations[​](#in-destinations "Direct link to In destinations") ``` const myDestination = { type: 'my-destination', config: {}, push(event, { logger }) { logger.debug('Processing event', { name: event.name }); try { // Send event to service sendToService(event); logger.info('Event sent successfully'); } catch (error) { if (error.retryable) { logger.warn('Event send failed, will retry', error); } else { logger.error('Failed to send event', error); } } }, }; ``` ### In sources[​](#in-sources "Direct link to In sources") ``` const mySource = async (config, env) => { const { logger } = env; logger.info('Source initialized'); return { type: 'my-source', config, push: async (event) => { logger.debug('Received event', event); await env.push(event); }, }; }; ``` ## Logger API[​](#logger-api "Direct link to Logger API") ### Methods[​](#methods "Direct link to Methods") | Method | Description | | -------------------------- | ----------------------------------------------------- | | `error(message, context?)` | Log an error message | | `warn(message, context?)` | Log a warning (degraded state, config issues) | | `info(message, context?)` | Log an informational message | | `debug(message, context?)` | Log a debug message | | `throw(message, context?)` | Log an error and throw an exception (returns `never`) | | `json(data)` | Output structured JSON data | | `scope(name)` | Create a scoped logger with a prefixed scope | ### Error handling[​](#error-handling "Direct link to Error handling") The logger accepts `Error` objects directly and automatically extracts relevant information: ``` try { await riskyOperation(); } catch (error) { // Error properties (message, name, stack, cause) are extracted automatically logger.error(error); // Or with additional context logger.error('Operation failed', { error, operation: 'sync' }); } ``` ### Boundary error logs[​](#boundary-error-logs "Direct link to Boundary error logs") When an exception escapes the inner pipeline of `collector.push` or `collector.command`, the collector emits a structured error log and increments `collector.status.failed`. The messages are `'push failed'` and `'command failed'` respectively, with the full event/ingest or command/data context as structured fields. See [Status — What `failed` counts](/docs/collector/.md#what-failed-counts) for details and PII implications. Internal pipeline failures from mapping, source startup, transformer init, and destination init are scoped under `'source'`, `'transformer:'`, or the destination type. User-callback throws from `on` subscriptions are scoped under `'on'`. The full list of error verbs and scopes is documented in [Status — What `failed` counts](/docs/collector/.md#what-failed-counts). ### Throwing errors[​](#throwing-errors "Direct link to Throwing errors") Use `logger.throw()` to log an error and throw in a single call: ``` function validateConfig(config) { if (!config.apiKey) { // Logs the error AND throws - returns 'never' type logger.throw('API key is required'); } // TypeScript knows apiKey exists here due to 'never' return type return config.apiKey; } ``` ## Scoped loggers[​](#scoped-loggers "Direct link to Scoped loggers") Scoped loggers prefix log messages with context information, making it easier to trace logs to their origin: ``` // In a destination, the logger is already scoped: [google-gtag:myGtag] push(event, { logger }) { logger.info('Processing'); // Output: [google-gtag:myGtag] Processing // Create nested scope const ga4Logger = logger.scope('ga4'); ga4Logger.debug('Sending to GA4'); // Output: [google-gtag:myGtag:ga4] Sending to GA4 } ``` ## Custom handler[​](#custom-handler "Direct link to Custom handler") For advanced use cases like sending logs to external services, configure a custom handler: ``` import { startFlow } from '@walkeros/collector'; const { collector, elb } = await startFlow({ logger: { level: 'INFO', handler: (level, message, context, scope, originalHandler) => { // Send to external logging service loggingService.log({ level, message, context, scope: scope.join(':'), timestamp: Date.now(), }); // Optionally call the original console handler originalHandler(level, message, context, scope); }, }, }); ``` The handler receives: | Parameter | Type | Description | | ----------------- | ---------------- | ------------------------------------------------- | | `level` | `Level` | Log level enum (ERROR=0, WARN=1, INFO=2, DEBUG=3) | | `message` | `string` | The log message | | `context` | `LogContext` | Additional context object | | `scope` | `string[]` | Array of scope names | | `originalHandler` | `DefaultHandler` | The default console handler | ## Testing[​](#testing "Direct link to Testing") Use the mock logger utility for testing destinations and sources: ``` import { createMockLogger } from '@walkeros/core'; describe('MyDestination', () => { it('logs events correctly', async () => { const mockLogger = createMockLogger(); await myDestination.push(testEvent, { config: {}, logger: mockLogger, }); expect(mockLogger.info).toHaveBeenCalledWith( 'Event sent successfully' ); }); }); ``` The mock logger provides jest mock functions for all logger methods plus: * `scopedLoggers`: Array of scoped loggers created via `scope()` --- # State State is a built-in config property available on source, transformer, and destination steps. It replaces inline `$code:` for the two most common store operations: stash a value (`set`) and fetch a value (`get`). Both directions reuse the same [mapping](/docs/mapping/.md) engine, so any value you can express in a mapping you can stash or fetch. ## Configuration[​](#configuration "Direct link to Configuration") Add `state` to any step. It accepts a single operation or an array of operations: ``` "transformers": { "session": { "state": { "mode": "set", "key": "user.session", "value": "data.gclid" } } } ``` ### Properties[​](#properties "Direct link to Properties") | Property | Type | Description | | -------- | ---------------- | ---------------------------------------------------------------------- | | `mode` | `'get' \| 'set'` | Direction. `get` reads from the store, `set` writes to it. | | `store` | `string` | Optional store id from `flow.stores`. Defaults to the in-memory cache. | | `key` | `Mapping.Value` | Resolves against the event to the **store key**. | | `value` | `Mapping.Value` | The **event side** of the operation (see below). | ## key is the store side, value is the event side[​](#key-is-the-store-side-value-is-the-event-side "Direct link to key is the store side, value is the event side") The mental model is symmetric: `key` always names the slot in the store, `value` always refers to the event. `mode` decides the direction the data flows between them. The one asymmetry is what `value` means per mode: * **`set`** resolves `value` against the event to produce the **payload** to store. `value` can be any mapping value (a path, a constant, an `fn`, a `map`). * **`get`** treats `value` as a **write-target path** on the event. It must be a bare string path (or a `ValueConfig` with a `key`), because the fetched value is written there via that path. A pure constant, `fn`, `map`, `loop`, or `set` is rejected for a `get`, and the path may not contain `*`. ``` // set: value is the payload (read from the event, written to the store) { "mode": "set", "key": "user.id", "value": "data.token" } // get: value is the destination path (fetched value written onto the event) { "mode": "get", "key": "user.id", "value": "data.token" } ``` ## The default store[​](#the-default-store "Direct link to The default store") When `store` is omitted, state uses the same built-in in-memory store as [cache](/docs/collector/cache.md) (`__cache`). This store is an entry-capped LRU with a TTL sweep, so entries are **evictable**: treat the default store as best-effort, not durable. For persistence across restarts or instances, point `store` at a configured backing (Redis, S3, filesystem). State keys on the default store are prefixed with `state:` so they cannot collide with cache entries that share the same store. Explicitly named stores use the key as-is. An explicitly named store that is not declared in `flow.stores` is a validation error; state never silently falls back to the default store for a named miss. ## Execution order[​](#execution-order "Direct link to Execution order") `state` normalizes to an array and entries run sequentially. Where the operations slot into the pipeline depends on the step type: * **Source:** `before` chain → state → `collector.push`. All entries run before the collector receives the event, in array order. State is skipped when the source uses a user `env.push` override, which receives the raw event with no pipeline. * **Transformer:** `before` chain → `get` → step mapping → `set` → `next` dispatch. `get` entries enrich the event before the mapping runs; `set` entries run after it settles, on every emitting path (straight-through, `next` routing, and per fork of a fan-out). * **Destination:** `get` → mapping-to-payload push → `set` (after a successful send). A failed push skips the `set`. For a batched destination, an event that is enqueued (not yet delivered) does not trigger `set`; the batched `set`-on-flush path is not yet wired. State is **fail-open**: if a store call or value resolution throws, the error is logged and the event passes through unchanged. The chain always continues. ## Examples[​](#examples "Direct link to Examples") ### Stash a session value[​](#stash-a-session-value "Direct link to Stash a session value") A pre-collector transformer stashes the inbound `gclid` keyed by session id: ``` "transformers": { "stashGclid": { "state": { "mode": "set", "store": "sessions", "key": "user.session", "value": "data.gclid" } } } ``` ### Look up a stashed value[​](#look-up-a-stashed-value "Direct link to Look up a stashed value") A later step fetches the stored `gclid` back onto the event: ``` "transformers": { "restoreGclid": { "state": { "mode": "get", "store": "sessions", "key": "user.session", "value": "data.gclid" } } } ``` ### Fetch a product margin before sending[​](#fetch-a-product-margin-before-sending "Direct link to Fetch a product margin before sending") A destination reads a per-product margin from a store and writes it onto the event before mapping it to the vendor payload: ``` "destinations": { "warehouse": { "package": "@walkeros/server-destination-api", "state": { "mode": "get", "store": "catalog", "key": "data.product_id", "value": "data.margin" } } } ``` ## Related[​](#related "Direct link to Related") * [Cache](/docs/collector/cache.md) short-circuits the pipeline on a hit; state always reads or writes and continues. * [Stores](/docs/stores/.md) provide the backing that state reads from and writes to. * [Mapping](/docs/mapping/.md) defines the `key` and `value` resolution. --- # Comparisons In this section, we compare walkerOS with other popular tools and technologies to help you determine whether it's the right fit for your tracking and analytics needs. ## [📄️walkerOS tagging vs. dataLayer.push](/docs/comparisons/dataLayer.md) [Compare walkerOS HTML tagging with manual dataLayer.push calls.](/docs/comparisons/dataLayer.md) ## [📄️walkerOS vs. Google Tag Manager](/docs/comparisons/gtm.md) [Compare walkerOS and Google Tag Manager to understand which solution fits your needs.](/docs/comparisons/gtm.md) ## [📄️walkerOS vs. Jentis](/docs/comparisons/jentis.md) [Compare walkerOS and Jentis to understand which solution fits your needs.](/docs/comparisons/jentis.md) ## [📄️walkerOS vs. Jitsu](/docs/comparisons/jitsu.md) [Compare walkerOS and Jitsu to understand which solution fits your needs.](/docs/comparisons/jitsu.md) ## [📄️walkerOS vs. RudderStack](/docs/comparisons/rudderstack.md) [Compare walkerOS and RudderStack to understand which solution fits your needs.](/docs/comparisons/rudderstack.md) ## [📄️walkerOS vs. Segment](/docs/comparisons/segment.md) [Compare walkerOS and Twilio Segment to understand which solution fits your needs.](/docs/comparisons/segment.md) ## [📄️walkerOS vs. Snowplow](/docs/comparisons/snowplow.md) [Compare walkerOS and Snowplow to understand which solution fits your needs.](/docs/comparisons/snowplow.md) ## [📄️walkerOS vs. Stape](/docs/comparisons/stape.md) [Compare walkerOS and Stape to understand which solution fits your needs.](/docs/comparisons/stape.md) --- # walkerOS tagging vs. dataLayer.push Google's `dataLayer` has been the standard way to push data to Google Tag Manager. But it requires manual work for every event. Here's how walkerOS tagging compares. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | dataLayer.push | walkerOS tagging | | ---------------------- | ----------------------- | --------------------------------- | | Tagging approach | Manual JavaScript calls | HTML attributes (data-elb) | | Event state management | Manual updates required | Automatic, real-time | | Consent handling | Manual setup in GTM | Built-in | | Version control | Hard to track changes | In your codebase | | Component reusability | Copy-paste code | Tag once, use everywhere | | Metadata enrichment | Manual | Automatic (timing, session, etc.) | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Keep using dataLayer.push if[​](#keep-using-datalayerpush-if "Direct link to Keep using dataLayer.push if") * You have an existing GTM setup you don't want to change * Your tracking is minimal and rarely changes * Your team prefers JavaScript over HTML attributes ### Switch to walkerOS tagging if[​](#switch-to-walkeros-tagging-if "Direct link to Switch to walkerOS tagging if") * You want tracking integrated into your components * You're tired of manual event management * You need automatic consent handling * You want version-controlled, reviewable tracking * You're building with modern component frameworks ## Key differences[​](#key-differences "Direct link to Key differences") ### How tagging works[​](#how-tagging-works "Direct link to How tagging works") **dataLayer.push:** You write JavaScript to collect values, build the event object, and push it at the right time. Every event needs manual code. ``` dataLayer.push({ event: 'add_to_cart', ecommerce: { items: [ { item_id: 'sku123', item_name: 'Cool Hat', price: 29.99, quantity: 1 }, ], }, }); ``` You must gather all values yourself and ensure the push happens after DOM ready and consent is granted. **walkerOS tagging:** You add HTML attributes to your components. walkerOS reads the DOM, detects interactions, checks consent, and pushes events automatically. Code HTMLCSSJS Loading... Preview GlobalsContextEntityPropertyAction Events Loading... Mapping Loading... Result Loading... Tag the button once. walkerOS handles listening for clicks, collecting data from attributes, checking consent, and pushing to any destination. ### Component integration[​](#component-integration "Direct link to Component integration") **dataLayer.push:** Tracking is often hard-coded separately from components, or managed entirely in GTM. New features frequently launch without tracking because it's an afterthought. **walkerOS tagging:** Tracking lives in your HTML alongside your components. When you build a component, you tag it once. It works everywhere that component appears: different pages, different contexts, same tracking. ### Maintenance[​](#maintenance "Direct link to Maintenance") **dataLayer.push:** Every event needs individual updates. Change a product schema? Update every push call. Add a new field? Touch every file. **walkerOS tagging:** Global attributes apply everywhere automatically. Component-level tagging means one change updates all instances. Event metadata (timing, session, consent state) is added automatically. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You can use the [GTM destination](/docs/destinations/web/gtag/gtm.md) to send walkerOS events into the dataLayer for your existing GTM tags. This lets you adopt walkerOS tagging while keeping your current GTM setup working. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS tagging? Check out the [tagging guide](/docs/sources/web/browser/tagging/html-attributes.md) to see how HTML attributes replace manual push calls. --- # walkerOS vs. Google Tag Manager Google Tag Manager (GTM) has been the industry standard for tag management. But as privacy regulations tighten and development practices evolve, many teams are looking for alternatives. Here's how walkerOS compares. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | GTM | walkerOS | | ---------------------- | ---------------------- | ------------------------------ | | Open-source | No | Yes | | Self-hosted option | Yes (sGTM) | Yes | | Server-side runtime | Sandboxed environment | Full Node.js | | Config location | Google's UI | Your codebase | | Privacy/GDPR | Requires configuration | Built-in consent handling | | Version control | Limited UI history | Git-native | | Developer experience | GUI-based | Code-first | | Composable tagging | No | Yes (tag once, use everywhere) | | Learning curve | Low (GUI) | Medium (code) | | Works with Google tags | Yes | Yes | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose GTM if[​](#choose-gtm-if "Direct link to Choose GTM if") * You prefer GUI-based configuration over code * You're heavily invested in the Google ecosystem * You need quick setup without developer resources * Your team is already trained on GTM ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * Data privacy and ownership is a priority * You want version-controlled, reviewable tracking * You need self-hosted infrastructure * You're building with modern component frameworks (React, Vue, etc.) * You want tracking to be part of your development workflow ## Key differences[​](#key-differences "Direct link to Key differences") ### Data ownership & privacy[​](#data-ownership--privacy "Direct link to Data ownership & privacy") **GTM:** With client-side GTM, data flows through Google's servers. Server-side GTM (sGTM) lets you host your own endpoint, but your configuration still lives in Google's UI and the runtime is sandboxed. **walkerOS:** Self-hosted by default. Your data never touches third-party servers unless you explicitly send it there. Config lives in your codebase. Built-in consent management ensures you only track what you're allowed to. ### Developer experience[​](#developer-experience "Direct link to Developer experience") **GTM:** GUI-based configuration. Changes are made through a web interface, making code review difficult. Version history exists but isn't Git-integrated. **walkerOS:** Config-as-code approach. Tracking is defined in your codebase, reviewed in pull requests, and deployed with your application. If it works in staging, it works in production. ### Flexibility[​](#flexibility "Direct link to Flexibility") **GTM:** Optimized for Google products (GA4, Google Ads). Third-party integrations require custom templates or JavaScript. **walkerOS:** Vendor-agnostic. Send events to any destination: analytics tools, data warehouses, marketing platforms, or your own APIs. Switch providers without changing your tracking code. ### Cost[​](#cost "Direct link to Cost") **GTM:** Client-side GTM is free but data flows through Google. Server-side GTM requires hosting infrastructure. Both are free software, costs come from hosting. **walkerOS:** Free and open-source. Self-host on your own infrastructure. Both solutions have similar hosting costs for server-side use - the difference is runtime flexibility, not price. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You don't have to choose one or the other. walkerOS can push events to GTM via the [GTM destination](/docs/destinations/web/gtag/gtm.md), letting you: * Use walkerOS for structured, consent-aware event collection * Keep GTM if your team is already familiar with it * Gradually migrate away from GTM at your own pace ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to get up and running, or explore the [GTM destination](/docs/destinations/web/gtag/gtm.md) to see how they work together. --- # walkerOS vs. Jentis [Jentis](https://www.jentis.com/) is a managed server-side tracking platform from Austria, focused on privacy compliance and data recovery. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | Jentis | walkerOS | | -------------------- | ----------------------- | ------------------------------ | | Open-source | No | Yes | | Pricing | From €490/month | Free (paid support available) | | Self-hosted option | No (managed only) | Yes | | Server-side tracking | Yes | Yes | | Vendor lock-in | Yes | None | | Data ownership | Jentis servers (EU) | Your infrastructure | | Built-in connectors | 100+ managed | Build your own or use existing | | AI/Synthetic data | Yes ("Synthetic Users") | No | | Setup complexity | Low (fully managed) | Medium (self-hosted) | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose Jentis if[​](#choose-jentis-if "Direct link to Choose Jentis if") * You want a fully managed, hands-off solution * Budget isn't a constraint (starting at €490/month) * You prefer vendor support over self-management * You need their "Synthetic Users" AI feature for data modeling * You want 100+ pre-built connectors without building integrations ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You want full control and code transparency * You prefer open-source with no vendor lock-in * You want to self-host on your own infrastructure * When cost matters, walkerOS is free * You're developer-focused and want config-as-code workflows * You need flexibility to build custom destinations ## Key differences[​](#key-differences "Direct link to Key differences") ### Cost[​](#cost "Direct link to Cost") **Jentis:** Starts at €490/month for the Advanced plan. Premium and AI-powered tiers require custom pricing. Enterprise costs can scale significantly. **walkerOS:** The software is free and open-source. You pay for your own infrastructure costs. Optional paid services include implementation support (setup fees) and SLAs for teams that want help getting started or need guaranteed support. ### Hosting model[​](#hosting-model "Direct link to Hosting model") **Jentis:** Managed service only. Jentis hosts and maintains everything. You don't run your own servers, but you also can't. **walkerOS:** Self-hosted by default. You control where your data lives and how your infrastructure scales. Full ownership, full responsibility. ### Transparency[​](#transparency "Direct link to Transparency") **Jentis:** Proprietary and closed-source. You trust their implementation without being able to audit the code. **walkerOS:** Fully open-source. Audit the code, understand exactly what's happening, and contribute improvements. ### Flexibility[​](#flexibility "Direct link to Flexibility") **Jentis:** Pre-built connectors and managed platform. Great if their integrations cover your needs. **walkerOS:** Build anything, connect anywhere. Create custom destinations for any API, data warehouse, or internal system. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up server-side tracking on your own infrastructure. --- # walkerOS vs. Jitsu [Jitsu](https://jitsu.com/) is an open-source data collection platform founded in 2020 (Y Combinator S20), positioning itself as a Segment alternative. It focuses on streaming event data to data warehouses. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | Jitsu | walkerOS | | -------------------- | ------------------------------------------------------------------------- | ------------------------------------------- | | Open-source | Yes (MIT) | Yes (MIT) | | Company location | USA | Germany | | Data hosting | Self-host anywhere, US-hosting for Cloud version | Self-host anywhere | | Pricing | Free up to 200k events, then start at $99 per month for 2M events (Cloud) | Free, self-hosted (Paid support available) | | Consent handling | Config option (privacy mode) | First-class (consent in events) | | Event model | Schema-less, freeform naming | Structured entity-action | | Primary destinations | Data warehouses | Warehouses + ad platforms | | Composable tagging | No | Yes (tag once in component, use everywhere) | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose Jitsu if[​](#choose-jitsu-if "Direct link to Choose Jitsu if") * You're migrating from Segment and want API compatibility * You need included ClickHouse storage on the free tier * Data residency in the EU is not a requirement ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You need consent state in every event (GDPR-native, not just compliant) * You're routing to ad platforms (Meta, Google) and need match quality * You're a European company needing pure EU data hosting * You want structured event naming (entity-action prevents chaos at scale) ## Key differences[​](#key-differences "Direct link to Key differences") ### Consent handling[​](#consent-handling "Direct link to Consent handling") **Jitsu:** Offers a "privacy mode" that can be enabled via configuration. When active, it uses fingerprints instead of cookies and truncates IP addresses. However, consent is a configuration setting, not a property of the event itself. The consent state does not travel with events to destinations. **walkerOS:** Consent is a first-class citizen. Every event carries its consent state, so destinations always know what permissions were granted. This matters for GDPR accountability: you can prove exactly what consent existed when each event was collected. ### Data residency[​](#data-residency "Direct link to Data residency") **Jitsu:** US-based company with US-only infrastructure. According to their privacy policy, data is "stored and processed on servers in the United States" with no EU hosting option. For EU companies, this creates potential Schrems II compliance concerns when collecting personal data (IP addresses, user IDs, behavioral data). **walkerOS:** German company. The open-source software can be self-hosted on pure EU infrastructure, eliminating transatlantic data transfer concerns entirely. A commercial product is in development. ### Event model[​](#event-model "Direct link to Event model") **Jitsu:** Schema-less by design. Accepts any JSON, auto-creates warehouse columns, and uses freeform event naming (like Segment's "Product Viewed"). This is flexible but can lead to inconsistent naming across teams and messy warehouse schemas over time. **walkerOS:** Uses structured entity-action naming (`product view`, `order complete`). This convention enforces consistency: you always know what entity was involved and what action occurred. Optional contracts can validate events at collection time, catching issues before they reach your warehouse. ### Destination focus[​](#destination-focus "Direct link to Destination focus") **Jitsu:** Built primarily for streaming to data warehouses (BigQuery, Snowflake, Redshift, ClickHouse, PostgreSQL). The warehouse is the core destination. **walkerOS:** Destination-agnostic with strong support for ad platforms. Send events to warehouses AND to Meta/Google with high match quality for conversion optimization. This matters if you need both analytics and marketing use cases from the same event stream. ### Tagging approach[​](#tagging-approach "Direct link to Tagging approach") **Jitsu:** JavaScript SDK similar to Google Analytics. Add a tracking script, then call tracking functions in your code. Quick to start, but tracking code lives separate from your components. **walkerOS:** HTML attribute-based tagging with first-class React/Next.js support. Add `data-elb` attributes to components. Events are captured automatically based on user interactions. Tracking lives with your components, not in a separate tracking layer. ### Cost[​](#cost "Direct link to Cost") **Jitsu:** Free cloud tier up to 200k events/month with included ClickHouse. Business plans start at $99/month for 2M events. Self-hosted is free (MIT). **walkerOS:** The open-source software is free with no event limits. Currently self-hosted only - you pay for your own infrastructure. Optional paid services include implementation support and SLAs. ## The trade-off[​](#the-trade-off "Direct link to The trade-off") Jitsu optimizes for **speed of setup**: add a script tag, start collecting, auto-infer schemas, stream to warehouse. walkerOS optimizes for **quality of data over time**: structured events, consent-native, ad platform routing, EU-hosted. If you need to collect data fast and your destination is a warehouse, Jitsu's simplicity is compelling. If you need consent in every event, route to ad platforms, or require EU data residency, walkerOS is the better fit. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You can use walkerOS for event collection and send data to Jitsu as a destination. This gives you walkerOS's composable tagging, consent handling, and structured events on the frontend, with Jitsu's warehouse streaming on the backend. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up component-level tagging with consent-native event collection. --- # walkerOS vs. RudderStack [RudderStack](https://www.rudderstack.com/) is a warehouse-first customer data platform with both open-source and cloud offerings. It's often positioned as an open-source alternative to Segment. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | RudderStack | walkerOS | | ---------------------- | ----------------------- | ------------------------------ | | Open-source | Yes (with limitations) | Yes (MIT, no limitations) | | Pricing | From $750/month (cloud) | Free (paid support available) | | Free tier | 1M events/month | Unlimited (self-hosted) | | Self-hosted option | Yes | Yes | | Complexity | Medium to high | Low to medium | | Pre-built integrations | 200+ | Build your own or use existing | | Warehouse-first | Yes (core focus) | Warehouse is one destination | | Transformations | JavaScript/Python | JavaScript (mapping) | | Composable tagging | No | Yes (tag once, use everywhere) | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose RudderStack if[​](#choose-rudderstack-if "Direct link to Choose RudderStack if") * You need a warehouse-first CDP architecture * You want 200+ pre-built integrations * Your data team needs JavaScript/Python transformations * You prefer event-based pricing over MTU-based * You need advanced features like identity resolution ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You want something lightweight and quick to set up * You prefer truly open-source (MIT) without feature restrictions * You want composable, component-level tagging for web * When cost matters, walkerOS is free with no event limits * You need focused event collection without full CDP complexity ## Key differences[​](#key-differences "Direct link to Key differences") ### Open-source approach[​](#open-source-approach "Direct link to Open-source approach") **RudderStack:** Open-source core, but advanced features (Transformations, Live Events) require the cloud version or aren't available in Control Plane Lite. The open-source version has limitations. **walkerOS:** Fully open-source under MIT license. All features available, no restrictions. Use it however you want. ### Complexity and setup[​](#complexity-and-setup "Direct link to Complexity and setup") **RudderStack:** Requires Docker or Kubernetes for self-hosting. Separate data plane and control plane concepts. More infrastructure to manage for the full feature set. **walkerOS:** Simpler architecture. Get started quickly without complex infrastructure requirements. Designed for developers who want control without overhead. ### Cost[​](#cost "Direct link to Cost") **RudderStack:** Free cloud tier with 1M events/month. Paid plans start at $750/month with usage-based pricing (events, not MTUs). Self-hosted is "free" but requires infrastructure investment. **walkerOS:** The software is free and open-source with no event limits. You pay for your own infrastructure costs. Optional paid services include implementation support (setup fees) and SLAs for teams that want help getting started or need guaranteed support. ### Focus[​](#focus "Direct link to Focus") **RudderStack:** Warehouse-first CDP. Built around the idea that your data warehouse is the source of truth. Strong focus on data teams and analytics infrastructure. **walkerOS:** Event collection and routing. Focused on capturing events well and sending them where they need to go: warehouses, analytics tools, marketing platforms, or your own APIs. ### Integrations[​](#integrations "Direct link to Integrations") **RudderStack:** 200+ pre-built integrations. Great if you need broad coverage out of the box. **walkerOS:** Fewer pre-built destinations, but full flexibility to build custom ones. Better for teams with specific requirements or who prefer owning their integration code. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You can use walkerOS for event collection and send data to RudderStack as a destination. This gives you walkerOS's composable tagging and consent handling on the frontend, while leveraging RudderStack's warehouse integrations and transformations on the backend. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up lightweight event collection without CDP complexity. --- # walkerOS vs. Segment [Segment](https://segment.com/) (owned by Twilio) is a Customer Data Platform (CDP) that collects, unifies, and routes customer data. It's one of the most well-known names in the space. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | Segment | walkerOS | | ---------------------- | --------------------------- | ------------------------------ | | Open-source | Partially (some SDKs) | Yes (MIT) | | Pricing | From $120/month (Team plan) | Free (paid support available) | | Self-hosted option | No | Yes | | Vendor lock-in | Yes (Twilio ecosystem) | None | | Data ownership | Segment servers | Your infrastructure | | Pre-built integrations | 700+ | Build your own or use existing | | Identity resolution | Yes (Unify) | Basic (session/user stitching) | | Composable tagging | No | Yes (tag once, use everywhere) | | Target audience | Marketing & data teams | Developers | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose Segment if[​](#choose-segment-if "Direct link to Choose Segment if") * You need a full Customer Data Platform with identity resolution * You want 700+ pre-built integrations out of the box * Your team is marketing-focused and prefers GUI configuration * Budget allows for $120+/month (scales with MTUs) * You need enterprise features like audience building and journey orchestration ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You want full control and self-hosting capability * You prefer open-source with no vendor lock-in * You're developer-focused and want code-first workflows * When cost matters, walkerOS is free * You need event collection and routing without the full CDP overhead * You want composable, component-level tagging ## Key differences[​](#key-differences "Direct link to Key differences") ### Scope[​](#scope "Direct link to Scope") **Segment:** A full Customer Data Platform. Beyond event collection, it offers identity resolution, audience building, journey orchestration, and reverse ETL. It's designed for marketing and data teams who need unified customer profiles. **walkerOS:** Focused event collection and routing infrastructure. Collect events from sources, process them, send them to destinations. It does one thing well without the CDP overhead. ### Cost[​](#cost "Direct link to Cost") **Segment:** Free tier limited to 1,000 MTUs. Team plan starts around $120/month. Pricing scales with monthly tracked users (MTUs). Enterprise customers often negotiate 70-80% discounts on list prices, but costs can grow quickly at scale. **walkerOS:** The software is free and open-source. You pay for your own infrastructure costs. Optional paid services include implementation support (setup fees) and SLAs for teams that want help getting started or need guaranteed support. ### Integrations[​](#integrations "Direct link to Integrations") **Segment:** 700+ pre-built integrations. Connect to almost any marketing, analytics, or data tool without writing code. Great if your stack is covered. **walkerOS:** Fewer pre-built destinations, but full flexibility to build custom integrations. Send data anywhere via destinations you control. Better for teams with specific requirements or internal systems. ### Data ownership[​](#data-ownership "Direct link to Data ownership") **Segment:** Data flows through Segment's infrastructure. While they offer privacy features and compliance certifications, you're trusting a third party with your customer data. **walkerOS:** Self-hosted by default. Your data never leaves your infrastructure unless you explicitly send it somewhere. Full control over where data lives and how it's processed. ### Developer experience[​](#developer-experience "Direct link to Developer experience") **Segment:** SDK-based tracking with a GUI for configuration and debugging. Good documentation, but less control over the underlying behavior. **walkerOS:** Code-first, config-as-code approach. HTML attribute-based tagging for web. Version control your tracking setup, review in PRs, test before production. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You can use walkerOS for event collection and send data to Segment as a destination. This gives you walkerOS's composable tagging and consent handling on the frontend, while still feeding data into Segment's CDP for identity resolution and downstream integrations. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up lightweight event collection without CDP complexity. --- # walkerOS vs. Snowplow [Snowplow](https://snowplow.io/) is a behavioral data platform founded in 2012, offering both open-source and enterprise solutions for event data collection. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | Snowplow | walkerOS | | -------------------- | --------------------------- | ------------------------------ | | Open-source | Partially (SLULA license) | Yes (MIT) | | Pricing | From $800/month (BDP Cloud) | Free (paid support available) | | Self-hosted option | Yes | Yes | | Complexity | High (enterprise-grade) | Low to medium | | Setup time | Days to weeks | Hours | | Target audience | Data teams at scale | Developers and small teams | | Composable tagging | No | Yes (tag once, use everywhere) | | Data warehouse focus | Primary destination | One of many destinations | | Real-time streaming | Yes (enterprise) | Yes | | Trackers | 35+ SDKs | Web, server, flexible sources | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose Snowplow if[​](#choose-snowplow-if "Direct link to Choose Snowplow if") * You have a dedicated data engineering team * You need enterprise-grade data infrastructure at scale * Your primary goal is feeding data warehouses (Snowflake, BigQuery, Databricks) * You want 35+ pre-built trackers across platforms * Budget allows for $800+/month or self-hosting resources ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You want something lightweight and quick to set up * You're a developer or small team without dedicated data engineers * You need flexibility beyond just data warehouse destinations * You prefer truly open-source (MIT license) over restricted licenses * You want to start free and scale as needed ## Key differences[​](#key-differences "Direct link to Key differences") ### Complexity and setup[​](#complexity-and-setup "Direct link to Complexity and setup") **Snowplow:** Enterprise-grade infrastructure designed for large-scale data operations. Requires significant setup time and often dedicated data engineering resources. The open-source version needs self-hosting expertise; the managed BDP starts at $800/month. **walkerOS:** Lightweight and developer-friendly. Get started in hours, not weeks. Designed for teams that want tracking infrastructure without the complexity of enterprise data platforms. ### Licensing[​](#licensing "Direct link to Licensing") **Snowplow:** The open-source version uses the Snowplow Limited Use License Agreement (SLULA), which restricts commercial use. Full commercial use requires their enterprise offering. **walkerOS:** MIT licensed. Use it however you want: commercially, modify it, fork it. No restrictions. ### Cost[​](#cost "Direct link to Cost") **Snowplow:** BDP Cloud starts at $800/month. Self-hosted open-source is "free" but requires significant infrastructure and engineering investment. Enterprise tiers (Basecamp, Ascent, Summit) scale up with custom pricing. **walkerOS:** The software is free and open-source. You pay for your own infrastructure costs. Optional paid services include implementation support (setup fees) and SLAs for teams that want help getting started or need guaranteed support. ### Focus and philosophy[​](#focus-and-philosophy "Direct link to Focus and philosophy") **Snowplow:** Built as comprehensive customer data infrastructure (CDI) for feeding data warehouses and powering analytics at scale. Designed for data teams building sophisticated data products. **walkerOS:** Built as flexible event collection and routing. Send data anywhere: analytics tools, data warehouses, marketing platforms, or your own APIs. Designed for developers who want control without complexity. ### When they overlap[​](#when-they-overlap "Direct link to When they overlap") Both solutions can: * Collect behavioral event data * Self-host on your infrastructure * Send data to warehouses and other destinations * Handle consent management The difference is scale and complexity. Snowplow is a full data platform; walkerOS is focused event infrastructure. ## Can they work together?[​](#can-they-work-together "Direct link to Can they work together?") Yes! You can use walkerOS for event collection and send data to Snowplow as a destination. This gives you walkerOS's composable tagging and consent handling on the frontend, with Snowplow's data infrastructure on the backend. ### Using the Snowplow Destination[​](#using-the-snowplow-destination "Direct link to Using the Snowplow Destination") The `@walkeros/web-destination-snowplow` package provides full integration: ``` import { startFlow } from '@walkeros/collector'; import { destinationSnowplow } from '@walkeros/web-destination-snowplow'; const { elb } = await startFlow({ destinations: { snowplow: { destination: destinationSnowplow, config: { settings: { collectorUrl: 'https://collector.yourdomain.com', appId: 'my-app', userId: 'user.id', // Cross-session user stitching }, }, }, }, }); ``` **Features include:** * Full ecommerce tracking (product views, cart actions, transactions) * Cross-session user identity via `setUserId` * Anonymous tracking mode for privacy compliance * Page context with `setPageType` * Global contexts and custom schemas * Activity tracking (page pings) * Plugin support (NPM and URL-based) See the [Snowplow destination documentation](/docs/destinations/web/snowplow.md) for full configuration options. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up lightweight event collection without enterprise complexity. --- # walkerOS vs. Stape [Stape](https://stape.io/) is a server-side tracking platform focused on hosting server-side Google Tag Manager (sGTM). They simplify sGTM infrastructure with managed hosting and power-ups. Here's how it compares to walkerOS. ## Quick comparison[​](#quick-comparison "Direct link to Quick comparison") | Feature | Stape | walkerOS | | ------------------ | ---------------------- | ----------------------------- | | Open-source | No | Yes | | Pricing | From $20/month | Free (paid support available) | | Self-hosted option | No (managed only) | Yes | | Requires GTM | Yes (sGTM-based) | No (standalone) | | Vendor lock-in | GTM + Stape | None | | Data ownership | Stape servers | Your infrastructure | | Ad blocker bypass | Yes (Custom Loader) | Yes (first-party setup) | | Cookie extension | Yes (Cookie Keeper) | Yes (configurable) | | Setup complexity | Medium (GTM knowledge) | Medium (code-first) | ## Who should use what?[​](#who-should-use-what "Direct link to Who should use what?") ### Choose Stape if[​](#choose-stape-if "Direct link to Choose Stape if") * You're already using Google Tag Manager and want server-side * You prefer managed hosting over self-hosting * You want plug-and-play integrations for Shopify, WordPress, etc. * You need their specific power-ups (Cookie Keeper, Custom Loader) ### Choose walkerOS if[​](#choose-walkeros-if "Direct link to Choose walkerOS if") * You want full control without depending on GTM * You prefer open-source and no vendor lock-in * You want to self-host on your own infrastructure * You're building a code-first, version-controlled tracking setup * You want to avoid GTM complexity entirely ## Key differences[​](#key-differences "Direct link to Key differences") ### Architecture[​](#architecture "Direct link to Architecture") **Stape:** A hosting layer on top of server-side GTM. You still configure tracking in GTM's interface. Stape handles the infrastructure. This means you need GTM knowledge (containers, tags, triggers, custom domains). **walkerOS:** A standalone event collection system. No GTM required. Define tracking in code, collect events through the source, process with the collector, and send to any destination. ### Cost[​](#cost "Direct link to Cost") **Stape:** Starts at $20/month for basic sGTM hosting, scaling with traffic. High-volume sites can reach $200+/month. Compare to self-hosted sGTM on Google Cloud which starts around $120/month. **walkerOS:** The software is free and open-source. You pay for your own infrastructure costs. Optional paid services include implementation support (setup fees) and SLAs for teams that want help getting started or need guaranteed support. ### GTM dependency[​](#gtm-dependency "Direct link to GTM dependency") **Stape:** Requires Google Tag Manager. You're configuring tracking in GTM's interface, with Stape providing the server-side infrastructure. If GTM changes, your setup changes. **walkerOS:** GTM-independent. While walkerOS can send events to GTM via the [GTM destination](/docs/destinations/web/gtag/gtm.md), it doesn't require GTM to function. Your tracking logic lives in your codebase. ### Flexibility[​](#flexibility "Direct link to Flexibility") **Stape:** Focused on the Google ecosystem: GA4, Google Ads, plus Meta and TikTok gateways. Power-ups are pre-built by Stape. **walkerOS:** Send events anywhere. Build custom destinations for any API, data warehouse, or internal system. Full control over data transformation and routing. ## Getting started[​](#getting-started "Direct link to Getting started") Ready to try walkerOS? Check out the [quickstart guide](/docs/.md) to set up tracking without GTM dependencies. --- # Contributing walkerOS is open source and will remain open source. We believe companies should own their data infrastructure. True data ownership only comes when you control your data collection. No vendor lock-in, no surprises. You own your code, your data, your future. There are multiple ways to contribute, and we appreciate them all. ## Getting started[​](#getting-started "Direct link to Getting started") ### Devcontainer (recommended)[​](#devcontainer-recommended "Direct link to Devcontainer (recommended)") The easiest way to start developing is using the [devcontainer](https://code.visualstudio.com/docs/devcontainers/containers) setup. It automatically installs all dependencies, extensions, and AI-assisted development tooling. 1. Install the [Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) extension in VS Code 2. Open the command palette and select `Dev Containers: Reopen in Container` 3. Wait for the container to build. You're ready to code The `skills/` folder contains step-by-step guides that work with any AI assistant. ### Manual setup[​](#manual-setup "Direct link to Manual setup") If you prefer a local setup: ``` npm install # Install dependencies npm run build # Build all packages npm run dev # Start watch mode # Verification scripts (use the smallest scope that proves your change) npm run verify:touched -- core # One package: typecheck + lint + test npm run verify:affected # All packages affected since origin/main npm run test:smoke # Critical path + affected (PR-ready check) npm run test # Full suite (release / on demand) npm run lint # Full lint npm run typecheck # Full typecheck ``` The verification scripts compose with turbo cache and the `[origin/main]` filter. For day-to-day work, prefer `verify:touched` for the package you edited and `verify:affected` before opening a PR. Save the full root-level runs for release prep. ## Ways to contribute[​](#ways-to-contribute "Direct link to Ways to contribute") ### Code[​](#code "Direct link to Code") * Fix bugs or implement new features * Create new destinations or sources (see `skills/create-destination` and `skills/create-source` for guidance) * Improve existing packages ### Documentation[​](#documentation "Direct link to Documentation") The documentation lives in `website/docs/`. To run the docs locally: ``` cd website && npm run start ``` * Improve existing documentation * Add examples and use cases * Fix typos and clarify explanations ### Community[​](#community "Direct link to Community") * [Report bugs](https://github.com/elbwalker/walkerOS/issues) (issue templates available) * Suggest features * Answer questions from other users ## Development workflow[​](#development-workflow "Direct link to Development workflow") walkerOS uses a test-driven development approach with [Jest](https://jestjs.io/). Please run tests before submitting pull requests. ### Adding a changeset[​](#adding-a-changeset "Direct link to Adding a changeset") When your PR includes changes that should be released, add a changeset: ``` npx changeset ``` 1. Select the packages your PR affects 2. Choose the version bump type: * **patch:** bug fixes, minor improvements * **minor:** new features (backwards compatible) * **major:** breaking changes 3. Write a brief summary of the changes 4. Commit the generated `.changeset/*.md` file with your PR Changesets help generate accurate changelogs and version packages correctly. Not every PR needs a changeset. Skip it for documentation, CI changes, or internal refactoring that doesn't affect published packages. ### Package structure[​](#package-structure "Direct link to Package structure") The project is organized as a monorepo: ``` packages/ ├── core/ # Types, utilities, schemas ├── collector/ # Event processing engine ├── config/ # Shared tooling config ├── web/ # Browser: sources/, destinations/ └── server/ # Node.js: sources/, destinations/ apps/ ├── quickstart/ # Validated examples ├── walkerjs/ # Browser bundle └── demos/ # Demo applications ``` ## Resources[​](#resources "Direct link to Resources") **For deeper guidance:** * `skills/`: step-by-step guides for common tasks (creating destinations, sources, testing strategies) * `AGENT.md`: quick reference for contributors and AI assistants **Get in touch:** * [Open an issue](https://github.com/elbwalker/walkerOS/issues) * [Send an email](mailto:hello@elbwalker.com) * [Schedule a call](https://calendly.com/elb-alexander/30min) * [Connect on LinkedIn](https://www.linkedin.com/in/alexanderkirtzel/) --- # Core utilities Core utilities are a collection of platform-agnostic functions that can be used across all walkerOS environments. They provide standardized building blocks for data manipulation, validation, mapping, and more. ## Installation[​](#installation "Direct link to Installation") Import the core utilities directly from the `@walkeros/core` package: ``` import { assign, anonymizeIP, getMappingValue } from '@walkeros/core'; ``` ## Data manipulation[​](#data-manipulation "Direct link to Data manipulation") ### assign[​](#assign "Direct link to assign") `assign(target: T, source: U, options?): T & U` merges two objects with advanced merging capabilities. It has special behavior for arrays: when merging, it concatenates arrays from both objects, removing duplicates. ``` interface AssignOptions { merge?: boolean; // Merge array properties (default: true) shallow?: boolean; // Create shallow copy (default: true) extend?: boolean; // Extend with new properties (default: true) } const obj1 = { a: 1, b: [1, 2] }; const obj2 = { b: [2, 3], c: 3 }; assign(obj1, obj2); // Returns { a: 1, b: [1, 2, 3], c: 3 } assign(obj1, obj2, { merge: false }); // Returns { a: 1, b: [2, 3], c: 3 } ``` ### Path operations[​](#path-operations "Direct link to Path operations") #### getByPath[​](#getbypath "Direct link to getByPath") `getByPath(object: unknown, path: string, defaultValue?: unknown): unknown` accesses nested properties using dot notation. Supports wildcard `*` for array iteration. ``` getByPath({ data: { id: 'wow' } }, 'data.id'); // Returns "wow" getByPath({ nested: [1, 2, { id: 'cool' }] }, 'nested.*.id'); // Returns ['', '', 'cool'] getByPath({ arr: ['foo', 'bar'] }, 'arr.1'); // Returns "bar" ``` #### setByPath[​](#setbypath "Direct link to setByPath") `setByPath(object: WalkerOS.Event, path: string, value: unknown): WalkerOS.Event` sets nested values using dot notation, returning a new object with the updated value. ``` const updatedEvent = setByPath(event, 'data.id', 'new-value'); // Returns a new event with data.id set to 'new-value' ``` ### clone[​](#clone "Direct link to clone") `clone(original: T): T` creates a deep copy of objects/arrays with circular reference handling. ``` const original = { foo: true, arr: ['a', 'b'] }; const cloned = clone(original); original.foo = false; // cloned.foo remains true ``` ### castValue[​](#castvalue "Direct link to castValue") `castValue(value: unknown): WalkerOS.PropertyType` converts string values to appropriate types (number, boolean). ``` castValue('123'); // Returns 123 (number) castValue('true'); // Returns true (boolean) castValue('hello'); // Returns 'hello' (unchanged) ``` ## Privacy and security[​](#privacy-and-security "Direct link to Privacy and security") ### anonymizeIP[​](#anonymizeip "Direct link to anonymizeIP") `anonymizeIP(ip: string): string` anonymizes IPv4 addresses by setting the last octet to zero. ``` anonymizeIP('192.168.1.100'); // Returns '192.168.1.0' ``` ### Hashing[​](#hashing "Direct link to Hashing") `getId(length?: number, charset?: string): string` generates random identifier strings. Defaults to a 6-character alphanumeric id; pass `charset` to restrict the allowed characters. ``` getId(); // Returns random 6-char string like 'a1b2c3' getId(10); // Returns 10-character alphanumeric string getId(5, 'abcdefghijklmnopqrstuvwxyz'); // Lowercase letters only ``` ## Event processing[​](#event-processing "Direct link to Event processing") ### getMappingValue[​](#getmappingvalue "Direct link to getMappingValue") `getMappingValue(event: WalkerOS.Event, mapping: Mapping.Data, context?: Partial): Promise` extracts values from events using [mapping configurations](/docs/mapping/.md). ``` // Simple path mapping await getMappingValue(event, 'data.productId'); // Complex mapping with conditions and loops const mapping = { map: { orderId: 'data.id', products: { loop: [ 'nested', { condition: (entity) => entity.entity === 'product', map: { id: 'data.id', name: 'data.name' }, }, ], }, }, }; await getMappingValue(event, mapping); ``` ### getMappingEvent[​](#getmappingevent "Direct link to getMappingEvent") `getMappingEvent(event: WalkerOS.PartialEvent, mapping?: Mapping.Rules): Promise` finds the appropriate mapping rule for an event. ## Marketing and analytics[​](#marketing-and-analytics "Direct link to Marketing and analytics") ### getMarketingParameters[​](#getmarketingparameters "Direct link to getMarketingParameters") `getMarketingParameters(url: URL, custom?: MarketingParameters): WalkerOS.Properties` extracts UTM and click ID parameters from URLs. ``` getMarketingParameters( new URL('https://example.com/?utm_source=docs&gclid=123'), ); // Returns { source: "docs", gclid: "123", clickId: "gclid" } // With custom parameters getMarketingParameters(url, { utm_custom: 'custom', partner: 'partnerId' }); ``` ## Type validation[​](#type-validation "Direct link to Type validation") ### Type checkers[​](#type-checkers "Direct link to Type checkers") A comprehensive set of type checking functions: * `isString(value)`, `isNumber(value)`, `isBoolean(value)` * `isArray(value)`, `isObject(value)`, `isFunction(value)` * `isDefined(value)`, `isSameType(a, b)` * `isPropertyType(value)` - Checks if value is valid walkerOS property ### Property utilities[​](#property-utilities "Direct link to Property utilities") * `castToProperty(value)` - Casts to valid property type * `filterValues(object)` - Filters object to valid properties only * `isPropertyType(value)` - Type guard for property validation ## Request handling[​](#request-handling "Direct link to Request handling") ### requestToData[​](#requesttodata "Direct link to requestToData") `requestToData(parameter: unknown): WalkerOS.AnyObject | undefined` converts query strings to JavaScript objects with type casting. ``` requestToData('a=1&b=true&c=hello&arr[0]=x&arr[1]=y'); // Returns { a: 1, b: true, c: 'hello', arr: ['x', 'y'] } ``` ### requestToParameter[​](#requesttoparameter "Direct link to requestToParameter") `requestToParameter(data: WalkerOS.AnyObject): string` converts objects to URL-encoded query strings. ``` requestToParameter({ a: 1, b: true, arr: ['x', 'y'] }); // Returns 'a=1&b=true&arr[0]=x&arr[1]=y' ``` ## User agent parsing[​](#user-agent-parsing "Direct link to User agent parsing") ### parseUserAgent[​](#parseuseragent "Direct link to parseUserAgent") `parseUserAgent(userAgent?: string): WalkerOS.User` extracts browser, OS, and device information. ``` parseUserAgent(navigator.userAgent); // Returns { browser: 'Chrome', browserVersion: '91.0', os: 'Windows', ... } ``` Individual functions are also available: * `getBrowser(userAgent)` - Returns browser name * `getBrowserVersion(userAgent)` - Returns browser version * `getOS(userAgent)` - Returns operating system * `getOSVersion(userAgent)` - Returns OS version * `getDeviceType(userAgent)` - Returns 'Desktop', 'Tablet', or 'Mobile' ## Error handling[​](#error-handling "Direct link to Error handling") ### tryCatch[​](#trycatch "Direct link to tryCatch") `tryCatch(tryFn: Function, catchFn?: Function, finallyFn?: Function)` wraps functions with error handling. ``` const safeParse = tryCatch(JSON.parse, () => ({})); safeParse('{"valid": "json"}'); // Parses successfully safeParse('invalid'); // Returns {} instead of throwing ``` ### tryCatchAsync[​](#trycatchasync "Direct link to tryCatchAsync") `tryCatchAsync(tryFn: Function, catchFn?: Function, finallyFn?: Function)` for async operations. ``` const safeAsyncCall = tryCatchAsync( () => fetchUserData(), (error) => ({ error: 'Failed to load user' }), ); ``` ## Performance optimization[​](#performance-optimization "Direct link to Performance optimization") ### debounce[​](#debounce "Direct link to debounce") `debounce(fn: Function, wait?: number)` delays function execution until after the wait time. ``` const debouncedSearch = debounce(searchFunction, 300); // Only executes after 300ms of inactivity ``` ### throttle[​](#throttle "Direct link to throttle") `throttle(fn: Function, wait?: number)` limits function execution frequency. ``` const throttledScroll = throttle(scrollHandler, 100); // Executes at most every 100ms ``` ## Utilities[​](#utilities "Direct link to Utilities") ### trim[​](#trim "Direct link to trim") `trim(str: string): string` removes whitespace from string ends. ### throwError[​](#throwerror "Direct link to throwError") `throwError(message: string)` throws descriptive errors. ### onLog[​](#onlog "Direct link to onLog") `onLog(message: unknown, verbose?: boolean)` provides consistent logging. ``` onLog('Debug info', true); // Logs message onLog('Silent message'); // No output ``` For platform-specific utilities, see: * [Web Core](/docs/core/web.md) - Browser-specific functions * [Server Core](/docs/core/server.md) - Node.js server functions --- # Server Core Utilities [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/core) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/core)[ ](https://www.npmjs.com/package/@walkeros/server-core) [Package](https://www.npmjs.com/package/@walkeros/server-core) Server core utilities are Node.js-specific functions designed for server-side walkerOS implementations. These utilities handle server communication, cryptographic hashing, and other backend operations. ## Installation[​](#installation "Direct link to Installation") Import server utilities from the `@walkeros/server-core` package: ``` import { sendServer, getHashServer } from '@walkeros/server-core'; ``` ## Server communication[​](#server-communication "Direct link to Server communication") ### sendServer[​](#sendserver "Direct link to sendServer") `sendServer(url: string, data?: SendDataValue, options?: SendServerOptions): Promise` sends HTTP requests using Node.js built-in modules (`http`/`https`). ``` // Simple POST request const response = await sendServer('https://api.example.com/events', { name: 'page view', data: { url: '/home' }, }); // With custom options const response = await sendServer(url, data, { method: 'PUT', headers: { Authorization: 'Bearer token', 'Content-Type': 'application/json', }, timeout: 10000, // 10 seconds }); if (response.ok) { console.log('Data sent successfully:', response.data); } else { console.error('Send failed:', response.error); } ``` ### SendServerOptions[​](#sendserveroptions "Direct link to SendServerOptions") ``` interface SendServerOptions { headers?: Record; // Custom HTTP headers method?: string; // HTTP method (default: 'POST') timeout?: number; // Request timeout in milliseconds (default: 5000) } ``` ### SendResponse[​](#sendresponse "Direct link to SendResponse") ``` interface SendResponse { ok: boolean; // Indicates if the request was successful (2xx status) data?: unknown; // Parsed response data (if available) error?: string; // Error message (if request failed) } ``` ## Cryptographic operations[​](#cryptographic-operations "Direct link to Cryptographic operations") ### getHashServer[​](#gethashserver "Direct link to getHashServer") `getHashServer(str: string, length?: number, options?: { algorithm?: 'sha256' | 'md5' }): Promise` generates hashes using Node.js crypto module. The algorithm defaults to `sha256`; pass `{ algorithm: 'md5' }` for an MD5 digest. ``` // Generate full SHA-256 hash const fullHash = await getHashServer('user123@example.com'); // Returns full 64-character hash // Generate shortened hash for anonymization const userFingerprint = await getHashServer( userAgent + language + ipAddress + date.getDate(), 16, ); // Returns 16-character hash like '47e0bdd10f04ef13' // User identification while preserving privacy const anonymousId = await getHashServer(`${userEmail}${deviceId}${salt}`, 12); ``` This function is commonly used for: * **User Anonymization**: Creating privacy-safe user identifiers * **Fingerprinting**: Generating device/session fingerprints * **Data Deduplication**: Creating consistent identifiers * **Privacy Compliance**: Hashing PII for GDPR/CCPA compliance ## Usage examples[​](#usage-examples "Direct link to Usage examples") ### Event Processing Pipeline[​](#event-processing-pipeline "Direct link to Event Processing Pipeline") ``` import { sendServer, getHashServer } from '@walkeros/server-core'; async function processUserEvent(event, userInfo) { // Anonymize user identification const anonymousUserId = await getHashServer( `${userInfo.email}${userInfo.deviceId}`, 16, ); // Prepare event with anonymized data const processedEvent = { ...event, user: { ...event.user, id: anonymousUserId, }, }; // Send to analytics service const result = await sendServer( 'https://analytics.example.com/collect', processedEvent, { headers: { 'X-API-Key': process.env.ANALYTICS_API_KEY, }, timeout: 8000, }, ); return result; } ``` ### Privacy-Safe Session Tracking[​](#privacy-safe-session-tracking "Direct link to Privacy-Safe Session Tracking") ``` async function createSessionId(request) { const fingerprint = [ request.headers['user-agent'], request.ip.replace(/\.\d+$/, '.0'), // Anonymize IP new Date().toDateString(), // Daily rotation ].join('|'); return await getHashServer(fingerprint, 20); } ``` ## Error handling[​](#error-handling "Direct link to Error handling") Server utilities include comprehensive error handling: ``` try { const response = await sendServer(url, data, { timeout: 5000 }); if (response.ok) { // Success - response.data contains the result console.log('Success:', response.data); } else { // Request completed but with error status console.warn('Request failed:', response.error); } } catch (error) { // Network error, timeout, or other exception console.error('Network error:', error.message); } ``` ## Performance considerations[​](#performance-considerations "Direct link to Performance considerations") ### Timeout Configuration[​](#timeout-configuration "Direct link to Timeout Configuration") Configure appropriate timeouts based on your use case: ``` // Fast analytics endpoint await sendServer(url, data, { timeout: 2000 }); // Critical business data await sendServer(url, data, { timeout: 15000 }); ``` ### Batch Processing[​](#batch-processing "Direct link to Batch Processing") For high-volume scenarios, consider batching: ``` const events = [ /* ... multiple events ... */ ]; const response = await sendServer( '/api/events/batch', { events, timestamp: Date.now(), }, { timeout: 10000, }, ); ``` ### Connection Reuse[​](#connection-reuse "Direct link to Connection Reuse") The underlying Node.js HTTP agent automatically reuses connections for better performance with multiple requests to the same host. ## Security notes[​](#security-notes "Direct link to Security notes") * **HTTPS Only**: Use HTTPS URLs in production for encrypted transmission * **API Keys**: Store sensitive credentials in environment variables * **Timeout Limits**: Set reasonable timeouts to prevent hanging requests * **Hash Salting**: Use application-specific salts when hashing sensitive data ``` // Good security practices const apiKey = process.env.ANALYTICS_API_KEY; const saltedHash = await getHashServer(`${userData}${process.env.HASH_SALT}`); await sendServer('https://secure-api.example.com/events', data, { headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, timeout: 5000, }); ``` ## Integration with Core[​](#integration-with-core "Direct link to Integration with Core") Server utilities work seamlessly with [Core Utilities](/docs/core/.md): ``` import { getMappingValue, anonymizeIP } from '@walkeros/core'; import { sendServer, getHashServer } from '@walkeros/server-core'; async function processServerSideEvent(rawEvent, clientIP) { // Use core utilities for data processing const processedData = await getMappingValue(rawEvent, mappingConfig); const safeIP = anonymizeIP(clientIP); // Use server utilities for transmission const sessionId = await getHashServer(`${safeIP}${userAgent}`, 16); return await sendServer(endpoint, { ...processedData, sessionId, ip: safeIP, }); } ``` For platform-agnostic utilities, see [Core Utilities](/docs/core/.md). --- # Web Core Utilities [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/core) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/core)[ ](https://www.npmjs.com/package/@walkeros/web-core) [Package](https://www.npmjs.com/package/@walkeros/web-core) Web core utilities are browser-specific functions designed for client-side walkerOS implementations. These utilities handle DOM interactions, browser information, storage, sessions, and web-based communication. ## Installation[​](#installation "Direct link to Installation") Import web utilities from the `@walkeros/web-core` package: ``` import { getAttribute, sendWeb, sessionStart } from '@walkeros/web-core'; ``` ## DOM utilities[​](#dom-utilities "Direct link to DOM utilities") ### getAttribute[​](#getattribute "Direct link to getAttribute") `getAttribute(element: Element, name: string): string` retrieves attribute values from DOM elements with enhanced handling. ``` const element = document.querySelector('[data-elb="product"]'); const entityType = getAttribute(element, 'data-elb'); // Returns 'product' ``` ### Attribute Parsing[​](#attribute-parsing "Direct link to Attribute Parsing") #### splitAttribute[​](#splitattribute "Direct link to splitAttribute") `splitAttribute(str: string, separator?: string): string[]` splits attribute strings using specified separators. ``` splitAttribute('id:123,name:shirt', ','); // Returns ['id:123', 'name:shirt'] ``` #### splitKeyVal[​](#splitkeyval "Direct link to splitKeyVal") `splitKeyVal(str: string): [string, string]` splits key-value pairs from attribute strings. ``` splitKeyVal('id:123'); // Returns ['id', '123'] ``` #### parseInlineConfig[​](#parseinlineconfig "Direct link to parseInlineConfig") `parseInlineConfig(str: string): Record` parses inline configuration strings from HTML attributes. ``` parseInlineConfig('{"tracking": true, "debug": false}'); // Returns { tracking: true, debug: false } ``` ## Browser information[​](#browser-information "Direct link to Browser information") ### getLanguage[​](#getlanguage "Direct link to getLanguage") `getLanguage(navigatorRef: Navigator): string | undefined` extracts the user's preferred language. ``` getLanguage(navigator); // Returns 'en-US' or user's language ``` ### getTimezone[​](#gettimezone "Direct link to getTimezone") `getTimezone(): string | undefined` gets the user's timezone from the Intl API. ``` getTimezone(); // Returns 'America/New_York' or user's timezone ``` ### getScreenSize[​](#getscreensize "Direct link to getScreenSize") `getScreenSize(windowRef: Window): string` returns the window's screen dimensions. ``` getScreenSize(window); // Returns '1920x1080' or current screen size ``` ## Element visibility[​](#element-visibility "Direct link to Element visibility") ### isVisible[​](#isvisible "Direct link to isVisible") `isVisible(element: HTMLElement): boolean` checks whether an element is rendered and not painted over. It does not measure how much of the element is inside the viewport; that is the job of the `visible` and `impression` triggers. ``` const promoElement = document.getElementById('promotion'); if (isVisible(promoElement)) { // Element is rendered and nothing else is painted over it } ``` This function considers: * Computed `display`, `visibility`, and `opacity`, including transparent ancestors * Clipping against the viewport itself, not against ancestor overflow boxes; a parent with `overflow: hidden` is instead caught incidentally by the occlusion hit test below * Occlusion, via a hit test at the centre of the element's visible area, so something else painted on top of it counts as not visible ## Storage management[​](#storage-management "Direct link to Storage management") ### Storage Operations[​](#storage-operations "Direct link to Storage Operations") #### storageRead[​](#storageread "Direct link to storageRead") `storageRead(key: string, storage?: StorageType, env?: StorageEnv): WalkerOS.PropertyType` reads data from browser storage with automatic type conversion. ``` // Default uses localStorage const userId = storageRead('walker_user_id'); // Use sessionStorage const sessionData = storageRead('session_data', 'sessionStorage'); ``` #### storageWrite[​](#storagewrite "Direct link to storageWrite") `storageWrite(key: string, value: WalkerOS.PropertyType, maxAgeInMinutes?: number, storage?: StorageType, domain?: string, env?: StorageEnv): WalkerOS.PropertyType` writes data to storage with expiration and domain options. ``` // Store with 30-minute expiration storageWrite('user_preference', 'dark-mode', 30); // Store in sessionStorage storageWrite('temp_data', { id: 123 }, undefined, 'sessionStorage'); // Store with custom domain for cookies storageWrite('tracking_id', 'abc123', 1440, 'cookie', '.example.com'); ``` #### storageDelete[​](#storagedelete "Direct link to storageDelete") `storageDelete(key: string, storage?: StorageType, env?: StorageEnv)` removes data from storage. ``` storageDelete('expired_data'); storageDelete('session_temp', 'sessionStorage'); ``` ## Session management[​](#session-management "Direct link to Session management") ### sessionStart[​](#sessionstart "Direct link to sessionStart") `sessionStart(config?: SessionConfig): WalkerOS.SessionData | void` initializes and manages user sessions with automatic renewal and tracking. ``` // Start session with default config const session = sessionStart(); // Custom session configuration const session = sessionStart({ storage: true, domain: '.example.com', maxAge: 1440, // 24 hours sampling: 1.0, // 100% sampling }); ``` Session data includes: * `id` - Unique session identifier * `start` - Session start timestamp * `isNew` - Whether this is a new session * `count` - Number of events in session * `device` - Device identifier * `storage` - Whether storage is available ### Advanced Session Functions[​](#advanced-session-functions "Direct link to Advanced Session Functions") * `sessionStorage` - Session-specific storage operations * `sessionWindow` - Window/tab session management ## Web communication[​](#web-communication "Direct link to Web communication") ### sendWeb[​](#sendweb "Direct link to sendWeb") `sendWeb(url: string, data?: SendDataValue, options?: SendWebOptionsDynamic): SendWebReturn` sends data using various web transport methods. ``` // Default fetch transport await sendWeb('https://api.example.com/events', eventData); // Use specific transport await sendWeb(url, data, { transport: 'beacon' }); await sendWeb(url, data, { transport: 'xhr' }); // With custom headers await sendWeb(url, data, { headers: { Authorization: 'Bearer token' }, method: 'PUT', }); ``` ### Transport-Specific Functions[​](#transport-specific-functions "Direct link to Transport-Specific Functions") #### sendWebAsFetch[​](#sendwebasfetch "Direct link to sendWebAsFetch") `sendWebAsFetch(url: string, data?: SendDataValue, options?: SendWebOptionsFetch): Promise` uses the modern Fetch API with advanced options. ``` await sendWebAsFetch(url, data, { credentials: 'include', noCors: true, headers: { 'Content-Type': 'application/json' }, }); ``` #### sendWebAsBeacon[​](#sendwebasbeacon "Direct link to sendWebAsBeacon") `sendWebAsBeacon(url: string, data?: SendDataValue): SendResponse` uses the Beacon API for reliable data transmission, especially during page unload. ``` // Reliable sending during page unload window.addEventListener('beforeunload', () => { sendWebAsBeacon('/analytics/pageview', { duration: Date.now() - startTime }); }); ``` #### sendWebAsXhr[​](#sendwebasxhr "Direct link to sendWebAsXhr") `sendWebAsXhr(url: string, data?: SendDataValue, options?: SendWebOptions): SendResponse` uses XMLHttpRequest for synchronous communication. ``` // Synchronous request (blocks execution) const response = sendWebAsXhr(url, data, { method: 'POST' }); ``` ## Web hashing[​](#web-hashing "Direct link to Web hashing") ### getHashWeb[​](#gethashweb "Direct link to getHashWeb") `getHashWeb(str: string, length?: number): Promise` generates SHA-256 hashes using the Web Crypto API. ``` // Generate hash for fingerprinting const userFingerprint = await getHashWeb( navigator.userAgent + navigator.language + screen.width, 16, ); // Returns shortened hash like '47e0bdd10f04ef13' ``` ## Configuration types[​](#configuration-types "Direct link to Configuration types") ### SendWebOptions[​](#sendweboptions "Direct link to SendWebOptions") ``` interface SendWebOptions { headers?: Record; method?: string; // Default: 'POST' transport?: 'fetch' | 'beacon' | 'xhr'; // Default: 'fetch' } interface SendWebOptionsFetch extends SendWebOptions { credentials?: 'omit' | 'same-origin' | 'include'; noCors?: boolean; timeout?: number; // Abort the request after this many ms (default 10000) } ``` ### SessionConfig[​](#sessionconfig "Direct link to SessionConfig") ``` interface SessionConfig { storage?: boolean; // Enable storage persistence domain?: string; // Cookie domain maxAge?: number; // Session duration in minutes sampling?: number; // Sampling rate (0-1) } ``` ### StorageEnv[​](#storageenv "Direct link to StorageEnv") ``` interface StorageEnv { window?: Window & typeof globalThis; document?: Document; } ``` The optional `env` parameter on storage functions allows injecting `window` and `document` for testing and simulation. When omitted, the global `window` and `document` are used. ### StorageType[​](#storagetype "Direct link to StorageType") ``` type StorageType = 'localStorage' | 'sessionStorage' | 'cookie'; ``` ## Usage notes[​](#usage-notes "Direct link to Usage notes") * **Consent Required**: Browser information functions may require user consent depending on privacy regulations * **Storage Fallbacks**: Storage functions gracefully handle unavailable storage with fallbacks * **Transport Selection**: Choose transport based on use case: * `fetch` - Modern, flexible, supports responses * `beacon` - Reliable during page unload, small payloads * `xhr` - Synchronous when needed, broader browser support * **Performance**: Session and storage operations are optimized for minimal performance impact For platform-agnostic utilities, see [Core Utilities](/docs/core/.md). --- # Destinations Destinations are the endpoints where walkerOS sends your processed events. They transform standardized walkerOS events into the specific formats required by analytics platforms, marketing tools, data warehouses, and custom APIs. Each destination operates independently, so one failed destination won't affect others. ## Analytics tools[​](#analytics-tools "Direct link to Analytics tools") | Destination | Web | Server | | ---------------------------------------------------- | --- | --------------------------------------------- | | **[Amplitude](/docs/destinations/web/amplitude.md)** | Yes | [Yes](/docs/destinations/server/amplitude.md) | | **[Clarity](/docs/destinations/web/clarity.md)** | Yes | - | | **[GA4](/docs/destinations/web/gtag/ga4.md)** | Yes | - | | **[GTM](/docs/destinations/web/gtag/gtm.md)** | Yes | - | | **[Mixpanel](/docs/destinations/web/mixpanel.md)** | Yes | [Yes](/docs/destinations/server/mixpanel.md) | | **[Piano](/docs/destinations/web/piano.md)** | Yes | - | | **[Piwik PRO](/docs/destinations/web/piwikpro.md)** | Yes | - | | **[Plausible](/docs/destinations/web/plausible.md)** | Yes | - | | **[PostHog](/docs/destinations/web/posthog.md)** | Yes | [Yes](/docs/destinations/server/posthog.md) | | **[Segment](/docs/destinations/web/segment.md)** | Yes | [Yes](/docs/destinations/server/segment.md) | | **[Snowplow](/docs/destinations/web/snowplow.md)** | Yes | - | ## Marketing tools[​](#marketing-tools "Direct link to Marketing tools") | | Web | Server | | --------- | ---------------------------------------------------------- | ------------------------------------------------------------------- | | Google | [Google Ads](/docs/destinations/web/gtag/ads.md) | [Google Data Manager API](/docs/destinations/server/datamanager.md) | | LinkedIn | [LinkedIn Insight Tag](/docs/destinations/web/linkedin.md) | [LinkedIn Conversions API](/docs/destinations/server/linkedin.md) | | Meta | [Meta Pixel](/docs/destinations/web/meta-pixel.md) | [Meta Conversions API](/docs/destinations/server/meta-capi.md) | | Pinterest | [Pinterest Tag](/docs/destinations/web/pinterest.md) | [Pinterest Conversions API](/docs/destinations/server/pinterest.md) | | TikTok | [TikTok Pixel](/docs/destinations/web/tiktok.md) | [TikTok Events API](/docs/destinations/server/tiktok.md) | ## Data warehouses[​](#data-warehouses "Direct link to Data warehouses") | Destination | Web | Server | | ---------------------------------------------------- | ------------------------------------ | --------------------------------------- | | **[API](/docs/destinations/api/.md)** | [Yes](/docs/destinations/api/web.md) | [Yes](/docs/destinations/api/server.md) | | **[AWS Firehose](/docs/destinations/server/aws.md)** | - | Yes | | **[GCP BigQuery](/docs/destinations/server/gcp.md)** | - | Yes | ## Create your own[​](#create-your-own "Direct link to Create your own") Build a custom destination for any endpoint. See [Create your own destination](/docs/destinations/create-your-own.md) for a step-by-step guide. ## Configuration[​](#configuration "Direct link to Configuration") These fields are available on every destination, regardless of package. They wrap the package-specific `settings` field, which is documented on each destination's page. | Property | Type | Description | More | | ------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `consent` | `WalkerOS.Consent` | Required consent states to send events to this destination | | | `settings` | `Destination.Settings` | Implementation-specific configuration | | | `credentials` | `Destination.Credentials` | Optional credentials (destination-defined shape) | | | `data` | `any \| any` | Global data transformation applied to all events for this destination | | | `include` | `Array` | Event sections to flatten into context.data | | | `env` | `Destination.Env` | Environment dependencies (platform-specific) | | | `id` | `string` | Destination instance identifier (defaults to destination key) | | | `init` | `boolean` | Whether to initialize immediately | | | `loadScript` | `boolean` | Whether to load external script (for web destinations) | | | `mapping` | `Mapping.Rules` | Entity-action specific mapping rules for this destination | | | `policy` | `Mapping.Policy` | Pre-processing policy rules applied before event mapping | | | `queue` | `boolean` | Whether to queue events when consent is not granted | | | `require` | `Array` | Defer destination initialization until these collector events fire (e.g., \["consent"]) | | | `logger` | `Logger.Config` | | | | `setup` | `boolean \| object` | One-time setup options applied during destination registration (boolean enables defaults, object configures specifics) | | | `before` | `RouteWithoutMany` | Post-collector transformer chain applied before this destination receives the event. \`many\` is not valid here — use multiple destinations for post-collector fan-out. | | | `next` | `RouteWithoutMany` | Post-push transformer chain. Runs after destination push completes; push response is available at ingest.\_response. \`many\` is not valid here — use multiple destinations for post-collector fan-out. | | | `cache` | `EventCache.Config` | Cache configuration for deduplication; skip push on cache HIT | | | `state` | `State.Config \| State.Config[]` | Declarative store get/set operations applied around this destination | | | `disabled` | `boolean` | Completely skip this destination (no init, no push, no queue) | | | `mock` | `Destination.Mock` | | | | `queueMax` | `number` | Maximum consent-denied events retained in queuePush for this destination. FIFO drop on overflow. Default 1000. | | | `dlqMax` | `number` | Maximum failed-push entries retained in dlq for this destination. FIFO drop on overflow. Default 100. | | | `timeout` | `number` | Per-destination delivery timeout in ms (default 10000); a delivery that does not settle in this window is routed to the DLQ like a thrown push. | | | `batch` | `number \| object` | Enables batching for all of this destination's events into one shared default buffer; a mapping rule's own batch splits that entity-action into its own buffer and overrides per field. Bare number is the debounce wait window; object form supports wait (debounce ms), size (count cap, default 1000), age (max ms since first entry, default 30000). | | | `breaker` | `number \| object` | Per-destination circuit breaker (presence-gated). After threshold consecutive transport failures the breaker opens and events are skipped until cooldown ms elapse, then one probe is admitted; success closes it, failure re-opens it. Partial-batch row failures are breaker-neutral. Bare number is the threshold; object form supports threshold (default 5) and cooldown (default 30000ms). | | --- # API Destination Send walkerOS events to any HTTP(S) endpoint with customizable data transformation and flexible transport options. Where this fits API is a **destination** in the walkerOS flow: Sends events to any HTTP endpoint with customizable headers, data transformation, and multiple transport options. ## Environments[​](#environments "Direct link to Environments") | Environment | Package | Use Case | | ----------- | ---------------------------------- | ------------------------------ | | **Web** | `@walkeros/web-destination-api` | Browser-side with fetch/beacon | | **Server** | `@walkeros/server-destination-api` | Node.js with native HTTP | ## Features[​](#features "Direct link to Features") * **Flexible URL Configuration** - Send events to any HTTP(S) endpoint * **Custom Headers** - Add authentication tokens, API keys, or custom headers * **Data Transformation** - Transform event data before sending * **Multiple Transports** - Web: fetch, XHR, beacon; Server: native HTTP/HTTPS ## Quick Start[​](#quick-start "Direct link to Quick Start") Choose your environment: * [Web (Browser)](/docs/destinations/api/web.md) - fetch, XHR, beacon transports * [Server (Node.js)](/docs/destinations/api/server.md) - native HTTP/HTTPS ## Common Use Cases[​](#common-use-cases "Direct link to Common Use Cases") * Sending events to custom analytics backends * Integrating with internal APIs * Forwarding events to webhooks * Building custom data pipelines --- # Server API Destination [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/api) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/api)[ ](https://www.npmjs.com/package/@walkeros/server-destination-api) [Package](https://www.npmjs.com/package/@walkeros/server-destination-api) The server API destination sends events from Node.js to any HTTP(S) endpoint using native HTTP/HTTPS modules. It auto-selects HTTP or HTTPS from the URL, supports configurable timeouts (default 5000ms), custom transforms, and dependency injection for testing. Where this fits Server API is a **server destination** in the walkerOS flow: Sends events server-side to any HTTP endpoint using Node.js native modules with configurable request formatting and batching. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-api ``` ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------- | ------------------------ | ----------------------------------------------- | ---- | | `url*` | `string` | The API endpoint URL to send events to | | | `headers` | `Record` | Custom HTTP headers to include with requests | | | `method` | `string` | HTTP method to use (default: POST) | | | `timeout` | `number` | Request timeout in milliseconds (default: 5000) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Custom headers A form submission is sent to the API with custom request headers such as an API key for authentication. Event ``` { "name": "form submit", "data": { "type": "contact", "email": "user@example.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "b7b368bff87feb27", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000801, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "data": "data" } ``` Out ``` sendServer("https://api.example.com/events", "{\"type\":\"contact\",\"email\":\"user@example.com\"}", { "headers": { "X-API-Key": "YOUR_API_KEY" } }) ``` ### Custom payload An order event is reshaped via a data mapping into a custom JSON payload with renamed fields for the API. Event ``` { "name": "order complete", "data": { "id": "ORD-500", "total": 199.99, "currency": "USD" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "buyer-42" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "babbe9274bf1fc95", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000802, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "data": { "map": { "order_id": "data.id", "amount": "data.total", "currency": "data.currency", "customer_id": "user.id", "event_type": "name" } } } ``` Out ``` sendServer("https://api.example.com/events", "{\"order_id\":\"ORD-500\",\"amount\":199.99,\"currency\":\"USD\",\"customer_id\":\"buyer-42\",\"event_type\":\"order complete\"}", {}) ``` ### Forward event data A page view is POSTed to the configured endpoint with the event data serialized as the JSON body. Event ``` { "name": "page view", "data": { "title": "Docs", "url": "https://example.com/docs" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "5d4020d98395d1e5", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000800, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "data": "data" } ``` Out ``` sendServer("https://api.example.com/events", "{\"title\":\"Docs\",\"url\":\"https://example.com/docs\"}", {}) ``` ## Usage[​](#usage "Direct link to Usage") ### Basic usage[​](#basic-usage "Direct link to Basic usage") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', }, }, }, }, }); ``` ### With headers and timeout[​](#with-headers-and-timeout "Direct link to With headers and timeout") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer your-token', }, timeout: 5000, }, }, }, }, }); ``` ### With custom transform[​](#with-custom-transform "Direct link to With custom transform") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', transform: (event, config, mapping) => { return JSON.stringify({ timestamp: Date.now(), event_name: event.name, properties: event.data, }); }, }, }, }, }, }); ``` ### With batching[​](#with-batching "Direct link to With batching") Set `config.batch` to send every event in one shared batch instead of one request per event. A bare number is the debounce `wait` window; an object tunes `wait`, `size`, and `age`. No `'* *'` wildcard mapping rule is needed. ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { api: { code: destinationAPI, config: { batch: { wait: 1000, size: 100 }, settings: { url: 'https://api.example.com/events', }, }, }, }, }); ``` ## Use cases[​](#use-cases "Direct link to Use cases") ### Webhook integration[​](#webhook-integration "Direct link to Webhook integration") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { webhook: { code: destinationAPI, config: { settings: { url: 'https://hooks.example.com/webhook', headers: { 'X-Webhook-Secret': process.env.WEBHOOK_SECRET, }, }, }, }, }, }); ``` ### Custom analytics backend[​](#custom-analytics-backend "Direct link to Custom analytics backend") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { analytics: { code: destinationAPI, config: { settings: { url: 'https://analytics.internal.com/ingest', method: 'PUT', headers: { 'X-API-Key': process.env.ANALYTICS_API_KEY, }, timeout: 10000, }, }, }, }, }); ``` ### With event mapping[​](#with-event-mapping "Direct link to With event mapping") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/server-destination-api'; await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', }, mapping: { page: { view: { data: { map: { pageUrl: 'data.path', pageTitle: 'data.title', }, }, }, }, }, }, }, }, }); ``` ## Related[​](#related "Direct link to Related") * [Web API Destination](/docs/destinations/api/web.md) - Browser-side variant * [Mapping](/docs/mapping/.md) - Transform events before sending --- # Web API Destination [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/api) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/api)[ ](https://www.npmjs.com/package/@walkeros/web-destination-api) [Package](https://www.npmjs.com/package/@walkeros/web-destination-api) The web API destination allows you to send events from the browser to any HTTP endpoint with customizable data transformation and transport methods. Where this fits Web API is a **web destination** in the walkerOS flow: Sends events from the browser to any HTTP endpoint using fetch, XHR, or beacon transport with customizable request formatting. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-api ``` ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | ------------------------------ | ------------------------------------------------ | ---- | | `url*` | `string` | The HTTP endpoint URL to send events to | | | `headers` | `Record` | Additional HTTP headers to include with requests | | | `method` | `string` | HTTP method for the request | | | `transform` | `function` | Function to transform event data before sending | | | `transport` | `'fetch' \| 'xhr' \| 'beacon'` | Transport method for sending requests | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Custom payload An order event is reshaped via a data mapping into a custom JSON body for the API endpoint. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "dd9c79f85b10cf8e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000502, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "data": { "map": { "order_id": "data.id", "amount": "data.total", "tax": "data.taxes", "shipping_cost": "data.shipping", "currency": "data.currency", "event_name": "name", "user_id": "user.id" } } } ``` Out ``` sendWeb("https://api.example.com/events", "{\"order_id\":\"0rd3r1d\",\"amount\":555,\"tax\":73.76,\"shipping_cost\":5.22,\"currency\":\"EUR\",\"event_name\":\"order complete\",\"user_id\":\"us3r\"}", { "headers": { "traceparent": "00-0a1b2c3d4e5f60718293a4b5c6d7e8f9-dd9c79f85b10cf8e-01" }, "transport": "fetch" }) ``` ### Entity action A generic entity action event is forwarded to the configured API endpoint with the mapped data JSON body. Event ``` { "name": "entity action", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "510a2a5afe9e407d", "trigger": "test", "entity": "entity", "action": "action", "timestamp": 1700000500, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "data": "data" } ``` Out ``` sendWeb("https://api.example.com/events", "{\"string\":\"foo\",\"number\":1,\"boolean\":true,\"array\":[0,\"text\",false]}", { "headers": { "traceparent": "00-0a1b2c3d4e5f60718293a4b5c6d7e8f9-510a2a5afe9e407d-01" }, "transport": "fetch" }) ``` ### Page view A page view is POSTed to the configured API endpoint with the event data section as the JSON body. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "7fffddf8586aa655", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000501, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "data": "data" } ``` Out ``` sendWeb("https://api.example.com/events", "{\"domain\":\"www.example.com\",\"title\":\"walkerOS documentation\",\"referrer\":\"https://www.walkeros.io/\",\"search\":\"?foo=bar\",\"hash\":\"#hash\",\"id\":\"/docs/\"}", { "headers": { "traceparent": "00-0a1b2c3d4e5f60718293a4b5c6d7e8f9-7fffddf8586aa655-01" }, "transport": "fetch" }) ``` ## Modes[​](#modes "Direct link to Modes") * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/web-destination-api'; const { collector, elb } = await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "https://api.example.com/events" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Usage[​](#usage "Direct link to Usage") ### Basic usage[​](#basic-usage "Direct link to Basic usage") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/web-destination-api'; const { collector, elb } = await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', method: 'POST', headers: { 'Content-Type': 'application/json', Authorization: 'Bearer your-token', }, }, }, }, }, }); ``` ### Advanced usage with transform[​](#advanced-usage-with-transform "Direct link to Advanced usage with transform") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/web-destination-api'; const { collector, elb } = await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', transport: 'fetch', transform: (event, config, mapping) => { // Custom transformation logic return JSON.stringify({ timestamp: Date.now(), event_name: `${event.entity}_${event.action}`, properties: event.data, context: event.context, }); }, }, }, }, }, }); ``` ## Use cases[​](#use-cases "Direct link to Use cases") ### Sending to analytics API[​](#sending-to-analytics-api "Direct link to Sending to analytics API") ``` import { startFlow } from '@walkeros/collector'; import { destinationAPI } from '@walkeros/web-destination-api'; const { collector, elb } = await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://analytics.example.com/track', method: 'POST', headers: { 'Content-Type': 'application/json', 'X-API-Key': 'your-api-key', }, transform: (event) => { return JSON.stringify({ event_type: `${event.entity}_${event.action}`, user_id: event.user?.id, session_id: event.user?.session, properties: event.data, timestamp: event.timing, }); }, }, }, }, }, }); ``` ### Using beacon transport[​](#using-beacon-transport "Direct link to Using beacon transport") For critical events that need to be sent even when the page is unloading: ``` // Add to existing startFlow config const { collector, elb } = await startFlow({ destinations: { criticalApi: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/critical-events', transport: 'beacon', // Reliable for page unload scenarios }, }, }, }, }); ``` ### Custom data mapping[​](#custom-data-mapping "Direct link to Custom data mapping") Use mapping rules to control which events are sent: ``` // Add to existing startFlow config const { collector, elb } = await startFlow({ destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://api.example.com/events', }, mapping: { entity: { action: { data: 'data', }, }, }, }, }, }, }); ``` ## Transport methods[​](#transport-methods "Direct link to Transport methods") * **fetch** (default): Modern, promise-based HTTP requests * **xhr**: Traditional XMLHttpRequest for older browser compatibility * **beacon**: Uses Navigator.sendBeacon() for reliable data transmission during page unload --- # Code Destination The code destination is a built-in, platform-agnostic destination that executes custom JavaScript code strings. It provides a lightweight alternative to tag managers like GTM, allowing you to run arbitrary code in response to events without external dependencies. Where this fits Code is a **built-in destination** in the walkerOS flow: Executes custom JavaScript code strings on events, loads external scripts dynamically, and acts as a lightweight tag manager replacement. ## Configuration[​](#configuration "Direct link to Configuration") Use `code: true` to enable the built-in code destination: ``` import { startFlow } from '@walkeros/collector'; const { collector, elb } = await startFlow({ destinations: { analytics: { code: true, config: { settings: { init: "context.logger.info('Analytics ready')", push: "context.logger.debug('Event:', event.name)", }, }, }, }, }); ``` ## Script Loading[​](#script-loading "Direct link to Script Loading") Load external scripts during initialization - useful for tag management: ``` import { startFlow } from '@walkeros/collector'; const { elb } = await startFlow({ destinations: { gtm: { code: true, config: { consent: { marketing: true }, // Scripts only load when consent granted settings: { scripts: [ 'https://www.googletagmanager.com/gtag/js?id=G-XXXXX', 'https://connect.facebook.net/en_US/fbevents.js', ], init: "window.dataLayer = window.dataLayer || [];", push: "dataLayer.push({ event: event.name, ...event.data });", }, }, }, }, }); ``` Scripts are injected in parallel with `async="true"`. The collector only handles injection - loading, errors, and execution are managed by the browser. For advanced loading patterns (sequential loading, onload callbacks), use custom `init` code. ## Configuration reference[​](#configuration-reference "Direct link to Configuration reference") ### Settings[​](#settings "Direct link to Settings") | Property | Type | Description | | ----------- | ---------- | ------------------------------------------------- | | `scripts` | `string[]` | URLs of external scripts to inject on init | | `init` | `string` | Code to run once when the destination initializes | | `on` | `string` | Code to run on lifecycle events (consent, etc.) | | `push` | `string` | Default code to run for each event | | `pushBatch` | `string` | Default code to run for batched events | ### Mapping[​](#mapping "Direct link to Mapping") Event-specific code can override settings via mapping: | Property | Type | Description | | ----------- | -------- | ----------------------------------- | | `push` | `string` | Code to run for this specific event | | `pushBatch` | `string` | Code to run for batched events | ## Context variables[​](#context-variables "Direct link to Context variables") Each code string has access to specific variables: ### init[​](#init "Direct link to init") * `context.collector` - The collector instance * `context.config` - Destination configuration * `context.env` - Environment variables * `context.logger` - Scoped logger instance ### push[​](#push "Direct link to push") * `event` - The WalkerOS event object * `context.collector` - The collector instance * `context.config` - Destination configuration * `context.data` - Transformed event data (from mapping) * `context.env` - Environment variables * `context.logger` - Scoped logger instance * `context.mapping` - The event mapping rule ### pushBatch[​](#pushbatch "Direct link to pushBatch") * `batch.key` - The batch key (event name) * `batch.events` - Array of events in the batch * `batch.data` - Array of transformed data * `batch.entries` - Per-event entries, each carrying `{ event, ingest?, respond?, rule?, data? }`. Read this when you need the per-event `ingest` or `respond` (for HTTP responses, per-event request IDs, etc.). `batch.events` and `batch.data` are derived views kept for backward compatibility. * `context.collector` - The collector instance * `context.config` - Destination configuration * `context.env` - Environment variables * `context.logger` - Scoped logger instance * `context.mapping` - The event mapping rule (representative entry's rule) #### Batch scheduling[​](#batch-scheduling "Direct link to Batch scheduling") Set `config.batch` to batch **all** of a destination's events into one shared buffer. No `'* *'` wildcard mapping rule is needed. A bare number is the debounce `wait` window; an object tunes `wait`, `size`, and `age`: ``` { config: { batch: { wait: 1000, size: 1000, age: 30000 }, // ... } } ``` * `wait` (ms) - Debounce window. The timer resets on every push. * `size` - Hard count cap. Flushes immediately at this many events. Default `1000` when batching is enabled. * `age` (ms) - Hard age cap since the first entry of the current window. Forces a flush even when pushes keep arriving. Default `30000`. Without `size`/`age` the batch can grow unbounded under sustained load (the debounce timer keeps resetting). Defaults are conservative; raise them only when you understand your destination's batch-size limits. **Batch all vs. batch selectively.** With `config.batch` set, every event of the destination joins the shared default buffer, including events matched by data-only mapping rules. A mapping rule's own `batch` splits that entity-action into its own buffer and overrides `config.batch` per field (`rule ?? config ?? default`). To batch only specific events, omit `config.batch` and set `batch` on those rules: ``` { config: { // no config.batch: batching stays off for everything else mapping: { order: { complete: { batch: { wait: 1000, size: 100 } } }, }, }, } ``` Pending batches flush automatically on shutdown, so buffered events are not lost. #### Failure handling[​](#failure-handling "Direct link to Failure handling") If `pushBatch` throws (or returns a rejected Promise), the entire batch is routed to the destination's `dlq` (dead-letter queue) and `status.destinations[id].failed` is incremented by the batch size. Per-item retry logic is the destination SDK's responsibility (BigQuery, Kafka, HubSpot each have their own backoff semantics). The collector never drops batch failures silently. Batched delivery is asynchronous, so the originating `elb()` call's result cannot reflect a later batch-flush outcome; failures appear in the destination's `failed` count, dead-letter buffer, and logs. ### on[​](#on "Direct link to on") * `type` - The event type (`'consent'`, `'ready'`, etc.) * `context.collector` - The collector instance * `context.config` - Destination configuration * `context.data` - Event-specific data * `context.env` - Environment variables * `context.logger` - Scoped logger instance ## Examples[​](#examples "Direct link to Examples") ### Basic logging[​](#basic-logging "Direct link to Basic logging") ``` const { elb } = await startFlow({ destinations: { logger: { code: true, config: { settings: { push: "console.log('Event:', event.name, event.data)", }, }, }, }, }); ``` ### API calls[​](#api-calls "Direct link to API calls") ``` const { elb } = await startFlow({ destinations: { api: { code: true, config: { settings: { push: ` fetch('https://api.example.com/track', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ name: event.name, data: event.data, timestamp: event.timestamp }) }) `, }, }, }, }, }); ``` ### Consent handling[​](#consent-handling "Direct link to Consent handling") ``` const { elb } = await startFlow({ destinations: { tracking: { code: true, config: { settings: { on: ` if (type === 'consent' && context.data?.marketing) { context.logger.info('Marketing consent granted'); // Initialize third-party scripts } `, push: "context.logger.debug(event.name)", }, }, }, }, }); ``` ### Event-specific overrides[​](#event-specific-overrides "Direct link to Event-specific overrides") Use mapping to override the default push code for specific events: ``` const { elb } = await startFlow({ destinations: { analytics: { code: true, config: { settings: { // Default handler for all events push: "context.logger.debug('Event:', event.name)", }, mapping: { product: { view: { // Custom handler for product view events push: ` fetch('/api/product-view', { method: 'POST', body: JSON.stringify({ productId: event.data.id, name: event.data.name }) }) `, }, }, order: { complete: { // Custom handler for purchase events push: ` window.dataLayer?.push({ event: 'purchase', value: event.data.total, transaction_id: event.data.id }) `, }, }, }, }, }, }, }); ``` ### Batched events[​](#batched-events "Direct link to Batched events") ``` const { elb } = await startFlow({ destinations: { batchApi: { code: true, config: { settings: { pushBatch: ` fetch('/api/batch', { method: 'POST', body: JSON.stringify({ events: batch.events.map(e => ({ name: e.name, data: e.data })) }) }) `, }, batch: 1000, // Batch all events with 1 second debounce }, }, }, }); ``` ## Error handling[​](#error-handling "Direct link to Error handling") All code execution is wrapped in try-catch blocks. Errors are logged using the destination's scoped logger and don't affect other destinations or event processing. ``` // Errors are caught and logged automatically settings: { push: ` // This error will be logged but won't crash the app throw new Error('Something went wrong'); `, } ``` ## Security considerations[​](#security-considerations "Direct link to Security considerations") The code destination uses `new Function()` to execute code strings. This is similar to `eval()` and should only be used with trusted code. Never execute user-provided code strings directly. For production environments, consider: * Only using code strings defined in your source code * Validating and sanitizing any dynamic configuration * Using Content Security Policy headers where appropriate --- # Create Your Own Destination This guide provides the essentials for building a custom walkerOS destination. ## What is a destination?[​](#what-is-a-destination "Direct link to What is a destination?") A destination is a function that receives events from walkerOS and sends them to an external service, such as an analytics platform, an API, or a database. ## The destination interface[​](#the-destination-interface "Direct link to The destination interface") A destination is an object that implements the `Destination` interface. The most important property is the `push` function, which is called for every event. ``` interface Destination { config: {}; push: PushFn; type?: string; init?: InitFn; on?( event: 'consent' | 'session' | 'ready' | 'run', context?: unknown, ): void | Promise; } ``` ## The `push` function[​](#the-push-function "Direct link to the-push-function") The `push` function is where you'll implement the logic to send the event to your desired service. It receives the `event` and a `context` object. ``` type PushFn = ( event: WalkerOS.Event, context: Destination.PushContext, ) => void; ``` The push context contains: * **`config`**: Destination configuration with settings * **`env`**: Environment with window, document, etc. * **`logger`**: Logger instance * **`id`**: Unique destination identifier * **`data`**: Pre-computed data from mapping * **`rule`**: The matching mapping rule for this event * **`ingest`**: Optional request metadata from source ## Example: A simple webhook destination[​](#example-a-simple-webhook-destination "Direct link to Example: A simple webhook destination") Here is an example of a simple destination that sends events to a webhook URL. ``` import type { Destination } from '@walkeros/core'; // 1. Define your settings interface interface WebhookSettings { url: string; } // 2. Create the destination object export const destinationWebhook: Destination = { type: 'webhook', config: {}, push(event, { config }) { const { settings } = config; // 3. Access your settings if (!settings?.url) return; // 4. Send the event fetch(settings.url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(event), }).catch(console.error); }, }; ``` ## Schema validation (optional)[​](#schema-validation-optional "Direct link to Schema validation (optional)") Destinations can export Zod schemas to provide runtime validation and TypeScript IDE support for configuration options. Export a `schemas` namespace containing `SettingsSchema` and `MappingSchema` to enable validation, autocomplete, and JSON Schema generation for tools like MCP and Explorer. See the [Destination Schemas Guide](https://github.com/elbwalker/walkerOS/blob/main/DESTINATION_SCHEMAS.md) for implementation details, or reference existing destinations like [Meta Pixel](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/meta) for complete examples. ## The `on` method (Optional)[​](#the-on-method-optional "Direct link to the-on-method-optional") The optional `on` method allows your destination to respond to collector lifecycle events. This is useful for handling consent changes, session management, or cleanup tasks. ### Available Events[​](#available-events "Direct link to Available Events") * **`consent`** - Called when user consent changes, with consent state as context * **`session`** - Called when a new session starts, with session data as context * **`ready`** - Called when the collector is ready to process events * **`run`** - Called when the collector starts or resumes processing ### Example: Consent-Aware Destination[​](#example-consent-aware-destination "Direct link to Example: Consent-Aware Destination") ``` export const destinationWithConsent: Destination = { type: 'webhook-consent', config: {}, on(event, context) { if (event === 'consent') { console.log('Consent updated:', context); // React to consent changes - maybe clear cookies if consent withdrawn } }, push(event, { config }) { console.log('Event:', event); }, }; ``` ## Setup lifecycle (optional)[​](#setup-lifecycle-optional "Direct link to Setup lifecycle (optional)") Components may implement an optional `setup()` lifecycle to provision external resources (BigQuery datasets, Pub/Sub topics, SQLite tables, webhook registrations) before the runtime ever processes events. Setup is **operator-time**: it runs only when an operator explicitly invokes `walkeros setup .`. The runtime never auto-invokes it. This separation isolates one-shot infrastructure provisioning from the high-volume event hot path. Production flows can ship with restricted runtime credentials (write-only) while operators run setup with elevated credentials (create) once per environment. The same lifecycle applies to sources, destinations, and stores. Transformers are pure functions and have no setup. ### Behavior[​](#behavior "Direct link to Behavior") * Triggered only by `walkeros setup .` (per-component, explicit). Never by `push`, `simulate`, `deploy`, or the runtime. * Setup is opt-in via `config.setup` in the flow config. The CLI invocation requires it to be `true` or an object; `false` or omitted means the operator's setup invocation is a narrated skip. * Idempotency is the package's responsibility. Re-running setup against a fully provisioned environment is a safe no-op. * Structured JSON output is opt-in via `--json` (e.g. `walkeros setup . --json`). In normal mode the CLI narrates the action and never splices a JSON envelope into the output, even if `setup()` returns a value. The opt-in setup behavior and idempotency are independent of `--json`. ### Example: a destination with setup[​](#example-a-destination-with-setup "Direct link to Example: a destination with setup") ``` import type { Destination } from '@walkeros/core'; interface BigQuerySettings { projectId: string; datasetId: string; tableId: string; } export const destinationBigquery: Destination = { type: 'bigquery', config: {}, async setup({ config }) { const { settings } = config; if (!settings) return; // Create dataset and table if missing. Idempotent: re-running is a no-op. // Return any structured result the operator may want to inspect. return { datasetCreated: settings.datasetId, tableCreated: settings.tableId }; }, push(event, { config }) { // Hot path: only writes rows, never creates resources. }, }; ``` The corresponding flow config opts in via `config.setup`: ``` { "destinations": { "bigquery": { "package": "@walkeros/server-destination-bigquery", "config": { "setup": true, "settings": { "projectId": "my-proj", "datasetId": "events", "tableId": "raw" } } } } } ``` Then the operator runs: ``` walkeros setup destination.bigquery ``` ### See also[​](#see-also "Direct link to See also") * [walkeros setup CLI command](/docs/apps/cli.md#setup-command) for the operator-facing command, options, and exit codes. * [Create your own source](/docs/sources/create-your-own.md) for source setup. * [Stores](/docs/getting-started/stores.md) for store setup and lifecycle ordering. ## Conditional activation with `require`[​](#conditional-activation-with-require "Direct link to conditional-activation-with-require") Destinations can use `require` to delay registration until specific events fire. This prevents SDK loading and network requests before conditions are met: ``` await startFlow({ destinations: { ga4: { code: ga4Destination, config: { require: ['session'], consent: { marketing: true }, }, }, }, }); ``` `require` and `consent` compose: `require` gates initialization before session information is available, `consent` gates every push (filter events by consent state). ## Using your destination[​](#using-your-destination "Direct link to Using your destination") To use your custom destination, add it to the `destinations` object in your collector configuration. ``` import { startFlow } from '@walkeros/collector'; import { destinationWebhook } from './destinationWebhook'; const { elb } = await startFlow({ destinations: { myWebhook: { destination: destinationWebhook, config: { settings: { url: 'https://api.example.com/events', }, }, }, }, }); ``` ## Advanced example: Session management[​](#advanced-example-session-management "Direct link to Advanced example: Session management") Here's a more advanced example that demonstrates session handling and cleanup: ``` export const destinationWithSession: Destination = { type: 'webhook-session', config: {}, on(event, context) { switch (event) { case 'session': // New session started console.log('New session:', context); // Could initialize session-specific tracking break; case 'consent': // Handle consent changes const consent = context as { marketing?: boolean; analytics?: boolean }; if (!consent?.marketing) { // Clear marketing-related data if consent withdrawn console.log('Marketing consent withdrawn, clearing data'); } break; case 'ready': // Collector is ready console.log('Starting destination services'); break; case 'run': // Collector resumed processing console.log('Collector resumed, processing queued events'); break; } }, push(event, { config }) { // Regular event processing const { settings } = config; if (!settings?.url) return; fetch(settings.url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(event), }).catch(console.error); }, }; ``` ## TypeScript integration[​](#typescript-integration "Direct link to TypeScript integration") To get full TypeScript support for your destination's configuration, you can extend the `WalkerOS.Destinations` interface. ``` // types.ts import type { Destination } from '@walkeros/core'; import type { WebhookSettings } from './destinationWebhook'; declare global { namespace WalkerOS { interface Destinations { webhook: Destination.Config; } } } ``` ## Environment dependencies (testing)[​](#environment-dependencies-testing "Direct link to Environment dependencies (testing)") The `env` parameter enables dependency injection for external APIs and SDKs. This allows you to test your destination logic without making actual API calls or requiring real browser globals. **Use Cases:** * Mock external SDKs (Google Analytics, Facebook Pixel, AWS SDK) * Test without network requests * Simulate different API responses * Run tests in any environment (Node.js, browser, CI) ### Defining an Environment[​](#defining-an-environment "Direct link to Defining an Environment") Define the external dependencies your destination needs: ``` // types.ts - Web destination import type { DestinationWeb } from '@walkeros/web-core'; export interface Env extends DestinationWeb.Env { window: { gtag: (command: string, ...args: unknown[]) => void; }; } // types.ts - Server destination import type { DestinationServer } from '@walkeros/server-core'; import type { BigQuery } from '@google-cloud/bigquery'; export interface Env extends DestinationServer.Env { BigQuery?: typeof BigQuery; } ``` ### Using Environment in Your Destination[​](#using-environment-in-your-destination "Direct link to Using Environment in Your Destination") Use the 3rd generic parameter for type safety, then access env in `init` or `push`: ``` import type { DestinationWeb } from '@walkeros/web-core'; import { getEnv } from '@walkeros/web-core'; interface Settings { /* ... */ } interface Mapping { /* ... */ } interface Env extends DestinationWeb.Env { window: { customAPI: (event: string) => void }; } // Add Env as 3rd generic parameter for proper typing export const destination: DestinationWeb.Destination = { type: 'custom', config: {}, async init({ config, env }) { // Initialize SDK using env, falls back to real APIs const { window } = getEnv(env); window.customAPI('init'); return config; }, push(event, { config, env }) { const { window } = getEnv(env); window.customAPI(event.name); }, }; ``` ### Creating Test Environments[​](#creating-test-environments "Direct link to Creating Test Environments") Create reusable mock environments in an `examples/env.ts` file: ``` // examples/env.ts import type { Env } from '../types'; export const push: Env = { window: { customAPI: jest.fn(), }, }; ``` Export from your examples index: ``` // examples/index.ts export * as env from './env'; ``` ### Using in Tests[​](#using-in-tests "Direct link to Using in Tests") ``` import { clone } from '@walkeros/core'; import type { Destination, Collector } from '@walkeros/core'; import { createMockLogger } from '@walkeros/core'; import { examples } from './index'; // Helper to create push context for testing function createPushContext( overrides: Partial> = {}, ): Destination.PushContext { return { config: {}, env: examples.env.push, logger: createMockLogger(), id: 'test-destination', collector: {} as Collector.Instance, data: {}, rule: undefined, ...overrides, }; } describe('My Destination', () => { it('calls custom API', async () => { // Clone the example env to avoid mutations const testEnv = clone(examples.env.push); const context = createPushContext({ env: testEnv }); await destination.push(event, context); expect(testEnv.window.customAPI).toHaveBeenCalledWith('page view'); }); }); ``` **Key Points:** * **Production**: No `env` needed, uses real APIs (window, fetch, SDKs) * **Testing**: Provide `env` with mocks for isolated testing * **Type Safety**: 3rd generic parameter gives full autocomplete * **Fallback**: `getEnv(env)` automatically uses real APIs if env not provided * **Reusable**: Store mock environments in `examples/env.ts` for consistency * **Context helper**: Use `createPushContext()` to standardize test context with `id` and `rule` ## Package convention[​](#package-convention "Direct link to Package convention") Every walkerOS package includes machine-readable metadata for tooling and discovery. ### `walkerOS` field in package.json[​](#walkeros-field-in-packagejson "Direct link to walkeros-field-in-packagejson") ``` { "walkerOS": { "type": "destination", "platform": "web" }, "keywords": ["walkeros", "walkeros-destination"] } ``` | Field | Required | Description | | ---------- | -------- | ------------------------------------------ | | `walkerOS` | Yes | Object with `type` and `platform` metadata | ### Build-time generation[​](#build-time-generation "Direct link to Build-time generation") Use `buildDev()` from the shared tsup config to auto-generate `walkerOS.json`: ``` import { buildDev } from '@walkeros/config/tsup'; ``` This file contains your package's JSON Schemas and examples, enabling MCP tools and the CLI to validate configurations without installing your package. ### Optional: Hints[​](#optional-hints "Direct link to Optional: Hints") Packages can export a `hints` record from `src/dev.ts` to provide lightweight, actionable context beyond schemas and examples, such as authentication methods, storage behavior, or troubleshooting tips. Hints are serialized into `walkerOS.json` and surfaced via MCP tools. See the `walkeros-create-destination` skill for details. ## Publishing checklist[​](#publishing-checklist "Direct link to Publishing checklist") * [ ] `walkerOS` field in package.json * Keywords include `walkeros` and `walkeros-destination` * [ ] `buildDev()` in tsup.config.ts * [ ] `dist/walkerOS.json` generated on build * [ ] `npm run test` passes * [ ] `npm run lint` passes --- # Amplitude [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/amplitude) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/amplitude)[ ](https://www.npmjs.com/package/@walkeros/server-destination-amplitude) [Package](https://www.npmjs.com/package/@walkeros/server-destination-amplitude) Server-side event delivery to [Amplitude](https://amplitude.com/) for product analytics, using the official [`@amplitude/analytics-node`](https://www.npmjs.com/package/@amplitude/analytics-node) SDK. Supports `track`, `identify`, `revenue`, `setGroup`, `groupIdentify`, and `flush` calls with per-event identity via EventOptions. Where this fits Amplitude is a **server destination** in the walkerOS flow: Sends events server-side to Amplitude's HTTP API, providing reliable delivery with identity resolution, revenue tracking, and group analytics without browser-side SDK overhead. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-amplitude ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationAmplitude } from '@walkeros/server-destination-amplitude'; await startFlow({ destinations: { amplitude: { code: destinationAmplitude, config: { settings: { apiKey: 'YOUR_AMPLITUDE_API_KEY', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "amplitude": { "package": "@walkeros/server-destination-amplitude", "import": "destinationAmplitude", "config": { "settings": { "apiKey": "YOUR_AMPLITUDE_API_KEY" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------------------ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `apiKey*` | `string` | Your Amplitude project API key. Find it in your Amplitude project settings under "General" -> "API Keys". | | | `serverZone` | `'US' \| 'EU'` | Amplitude data residency zone. Use EU for European data residency. Default: US. | | | `flushIntervalMillis` | `integer` | How often (in ms) to flush the event queue. Default: 10000. | | | `flushQueueSize` | `integer` | Max queued events before a flush. Default: 200. | | | `flushMaxRetries` | `integer` | Max retries on failed flush. Default: 12. | | | `useBatch` | `boolean` | Use the Amplitude batch endpoint for higher rate limits. Recommended for high-volume server flows. Default: false. | | | `minIdLength` | `integer` | Minimum length for user\_id and device\_id fields. | | | `serverUrl` | `string` | Custom server URL for proxies or self-hosted endpoints. | | | `optOut` | `boolean` | Initial opt-out state. When true, no events are sent. Default: false. | | | `enableRequestBodyCompression` | `boolean` | Enable gzip compression for request payloads. Default: false. | | | `identify` | `any` | walkerOS mapping value resolving to per-event identity. Keys: user\_id, device\_id, session\_id, set, setOnce, add, append, prepend, preInsert, postInsert, remove, unset, clearAll. | | | `eventOptions` | `any` | walkerOS mapping value resolving to per-event EventOptions. Keys: time, insert\_id, ip, city, country, region, language, platform, os\_name, os\_version, device\_brand, device\_model, app\_version, user\_agent. | | | `include` | `Array` | walkerOS event sections to include as event\_properties (like \['data', 'globals']). | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | --------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to an object with any of: user\_id, device\_id, session\_id, set, setOnce, add, append, prepend, preInsert, postInsert, remove, unset, clearAll. | | | `revenue` | `any` | Revenue mapping. Resolves to a single object or (via loop) an array, each with: productId, price, quantity, revenueType, currency, revenue, receipt, receiptSig, eventProperties. One amplitude.revenue() call fires per item. | | | `group` | `any` | Group assignment. Resolves to { type, name } -> amplitude.setGroup(type, name, eventOptions). | | | `groupIdentify` | `any` | Group properties. Resolves to { type, name, set?, setOnce?, ... } -> amplitude.groupIdentify(type, name, identify, eventOptions). | | | `eventOptions` | `any` | Per-rule EventOptions override. Resolves to { time?, insert\_id?, ip?, ... }. Overrides destination-level eventOptions for this rule. | | | `include` | `Array` | Per-rule include override. Replaces destination-level include for this rule. | | ## Examples ### Consent granted A walker consent command with analytics granted opts back into Amplitude tracking via setOptOut(false). Event ``` { "analytics": true } ``` Out ``` amplitude.setOptOut(false) ``` ### Consent revoked After analytics consent is granted (Amplitude loads), revoking it opts out of tracking via setOptOut(true). Event ``` { "analytics": false } ``` Out ``` amplitude.setOptOut(false); amplitude.setOptOut(true) ``` ### Default event A walkerOS event forwarded as an Amplitude track call with the event name and empty properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "541a72afa937f518", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("product view", {}) ``` ### Identify per event Destination-level identify resolves user\_id, device\_id, and session\_id into the Amplitude EventOptions on every call. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "bc7bbcd9ce130da6", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("page view", {}, { "user_id": "us3r", "device_id": "c00k13", "session_id": 394324160 }) ``` ### Include data section Destination-level include flattens the event data section into prefixed event\_properties on every track call. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "23221e9219da4448", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("product view", { "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420 }) ``` ### Event options Destination-level eventOptions map walker fields into Amplitude per-event metadata such as time and insert\_id. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "1700000110abcdef", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000110, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("page view", {}, { "time": 1700000110, "insert_id": "1700000110abcdef" }) ``` ### Group assignment A company update assigns the user to a group and sets group properties via setGroup plus groupIdentify. Event ``` { "name": "company update", "data": { "company": "Acme", "industry": "tech", "employee_count": 50, "founded_year": 2020 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "65204d8f52e7c91b", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "group": { "map": { "type": { "value": "company" }, "name": "data.company" } }, "groupIdentify": { "map": { "type": { "value": "company" }, "name": "data.company", "set": { "map": { "industry": "data.industry", "size": "data.employee_count" } }, "setOnce": { "map": { "founded": "data.founded_year" } } } } } } ``` Out ``` amplitude.setGroup("company", "Acme"); amplitude.groupIdentify("company", "Acme", { "set": { "industry": "tech", "size": 50 }, "setOnce": { "founded": 2020 } }) ``` ### Multi-product order An order with multiple nested products fires one amplitude.revenue per product plus a single track for the order. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "b161462477119c00", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "globals" ], "settings": { "revenue": { "loop": [ "nested", { "condition": { "$code": "e=>{const t=e;return\"number\"==typeof t?.data?.price}" }, "map": { "productId": "data.id", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 }, "revenueType": { "value": "purchase" }, "currency": { "key": "data.currency", "value": "EUR" } } } ] } } } ``` Out ``` amplitude.revenue({ "productId": "ers", "price": 420, "quantity": 1, "revenueType": "purchase", "currency": "EUR" }); amplitude.revenue({ "productId": "cc", "price": 42, "quantity": 1, "revenueType": "purchase", "currency": "EUR" }); amplitude.track("order complete", { "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "globals_pagegroup": "shop" }) ``` ### Subscription revenue A subscription renewal fires a single amplitude.revenue call with productId, price, and a currency fallback. Event ``` { "name": "subscription renew", "data": { "plan_id": "plan-pro", "amount": 9.99 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "7b89f10fe4fb29c0", "trigger": "test", "entity": "subscription", "action": "renew", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "revenue": { "map": { "productId": "data.plan_id", "price": "data.amount", "revenueType": { "value": "renewal" }, "currency": { "key": "data.currency", "value": "EUR" } } } } } ``` Out ``` amplitude.revenue({ "productId": "plan-pro", "price": 9.99, "revenueType": "renewal", "currency": "EUR" }) ``` ### User login identify A user login maps to amplitude.identify with set, setOnce, and add operations while the default track call is skipped. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "plan": "premium", "company": "Acme", "email": "user@acme.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "1df95c21e51376a4", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "user_id": "data.user_id", "set": { "map": { "plan": "data.plan", "company": "data.company", "email": "data.email" } }, "setOnce": { "map": { "first_login": "timestamp" } }, "add": { "map": { "login_count": { "value": 1 } } } } } } } ``` Out ``` amplitude.identify({ "set": { "plan": "premium", "company": "Acme", "email": "user@acme.com" }, "setOnce": { "first_login": 1700000105 }, "add": { "login_count": 1 } }, { "user_id": "new-user-123" }) ``` --- # AWS Firehose [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/aws) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/aws)[ ](https://www.npmjs.com/package/@walkeros/server-destination-aws) [Package](https://www.npmjs.com/package/@walkeros/server-destination-aws) Looking for AWS SNS? Topic publishing with FIFO ordering, KMS encryption, and idempotent provisioning lives on the dedicated [AWS SNS destination page](/docs/destinations/server/sns.md). The AWS destination package provides server-side integration for streaming events from walkerOS to Amazon Kinesis Data Firehose for real-time analytics, data lakes, and downstream processing. Where this fits AWS Firehose is a **server destination** in the walkerOS flow: Streams events to Amazon Kinesis Data Firehose for real-time analytics, S3 data lakes, and downstream AWS services. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-aws ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationFirehose } from '@walkeros/server-destination-aws'; await startFlow({ destinations: { firehose: { code: destinationFirehose, config: { settings: { firehose: { streamName: 'your-firehose-stream-name', region: 'eu-central-1', config: { credentials: { accessKeyId: 'your-access-key-id', secretAccessKey: 'your-secret-access-key', }, }, }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "firehose": { "package": "@walkeros/server-destination-aws", "import": "destinationFirehose", "config": { "settings": { "firehose": { "streamName": "your-firehose-stream-name", "region": "eu-central-1", "config": { "credentials": { "accessKeyId": "your-access-key-id", "secretAccessKey": "your-secret-access-key" } } } } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | ---------- | ------------------------------------------------- | ---- | | `firehose` | `firehose` | AWS Firehose configuration settings | | | `streamName*` | `string` | Name of the Kinesis Data Firehose delivery stream | | | `client` | `any` | Pre-configured AWS Firehose client instance | | | `region` | `string` | AWS region for the Firehose service | | | `config` | `any` | AWS SDK client configuration options | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Page view record A page view is sent to Kinesis Firehose as a single record with the full event JSON-stringified into the Data buffer. Event ``` { "name": "page view", "data": { "title": "Home", "url": "https://example.com/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700001000", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700001000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` client.send({ "DeliveryStreamName": "walkeros-events", "Records": [ { "Data": { "0": 123, "1": 34, "2": 110, "3": 97, "4": 109, "5": 101, "6": 34, "7": 58, "8": 34, "9": 112, "10": 97, "11": 103, "12": 101, "13": 32, "14": 118, "15": 105, "16": 101, "17": 119, "18": 34, "19": 44, "20": 34, "21": 100, "22": 97, "23": 116, "24": 97, "25": 34, "26": 58, "27": 123, "28": 34, "29": 116, "30": 105, "31": 116, "32": 108, "33": 101, "34": 34, "35": 58, "36": 34, "37": 72, "38": 111, "39": 109, "40": 101, "41": 34, "42": 44, "43": 34, "44": 117, "45": 114, "46": 108, "47": 34, "48": 58, "49": 34, "50": 104, "51": 116, "52": 116, "53": 112, "54": 115, "55": 58, "56": 47, "57": 47, "58": 101, "59": 120, "60": 97, "61": 109, "62": 112, "63": 108, "64": 101, "65": 46, "66": 99, "67": 111, "68": 109, "69": 47, "70": 34, "71": 125, "72": 44, "73": 34, "74": 99, "75": 111, "76": 110, "77": 116, "78": 101, "79": 120, "80": 116, "81": 34, "82": 58, "83": 123, "84": 34, "85": 100, "86": 101, "87": 118, "88": 34, "89": 58, "90": 91, "91": 34, "92": 116, "93": 101, "94": 115, "95": 116, "96": 34, "97": 44, "98": 49, "99": 93, "100": 125, "101": 44, "102": 34, "103": 103, "104": 108, "105": 111, "106": 98, "107": 97, "108": 108, "109": 115, "110": 34, "111": 58, "112": 123, "113": 34, "114": 112, "115": 97, "116": 103, "117": 101, "118": 103, "119": 114, "120": 111, "121": 117, "122": 112, "123": 34, "124": 58, "125": 34, "126": 100, "127": 111, "128": 99, "129": 115, "130": 34, "131": 125, "132": 44, "133": 34, "134": 99, "135": 117, "136": 115, "137": 116, "138": 111, "139": 109, "140": 34, "141": 58, "142": 123, "143": 34, "144": 99, "145": 111, "146": 109, "147": 112, "148": 108, "149": 101, "150": 116, "151": 101, "152": 108, "153": 121, "154": 34, "155": 58, "156": 34, "157": 114, "158": 97, "159": 110, "160": 100, "161": 111, "162": 109, "163": 34, "164": 125, "165": 44, "166": 34, "167": 117, "168": 115, "169": 101, "170": 114, "171": 34, "172": 58, "173": 123, "174": 34, "175": 105, "176": 100, "177": 34, "178": 58, "179": 34, "180": 117, "181": 115, "182": 51, "183": 114, "184": 34, "185": 44, "186": 34, "187": 100, "188": 101, "189": 118, "190": 105, "191": 99, "192": 101, "193": 34, "194": 58, "195": 34, "196": 99, "197": 48, "198": 48, "199": 107, "200": 49, "201": 51, "202": 34, "203": 44, "204": 34, "205": 115, "206": 101, "207": 115, "208": 115, "209": 105, "210": 111, "211": 110, "212": 34, "213": 58, "214": 34, "215": 115, "216": 51, "217": 115, "218": 115, "219": 49, "220": 48, "221": 110, "222": 34, "223": 125, "224": 44, "225": 34, "226": 110, "227": 101, "228": 115, "229": 116, "230": 101, "231": 100, "232": 34, "233": 58, "234": 91, "235": 123, "236": 34, "237": 101, "238": 110, "239": 116, "240": 105, "241": 116, "242": 121, "243": 34, "244": 58, "245": 34, "246": 99, "247": 104, "248": 105, "249": 108, "250": 100, "251": 34, "252": 44, "253": 34, "254": 100, "255": 97, "256": 116, "257": 97, "258": 34, "259": 58, "260": 123, "261": 34, "262": 105, "263": 115, "264": 34, "265": 58, "266": 34, "267": 115, "268": 117, "269": 98, "270": 111, "271": 114, "272": 100, "273": 105, "274": 110, "275": 97, "276": 116, "277": 101, "278": 100, "279": 34, "280": 125, "281": 125, "282": 93, "283": 44, "284": 34, "285": 99, "286": 111, "287": 110, "288": 115, "289": 101, "290": 110, "291": 116, "292": 34, "293": 58, "294": 123, "295": 34, "296": 102, "297": 117, "298": 110, "299": 99, "300": 116, "301": 105, "302": 111, "303": 110, "304": 97, "305": 108, "306": 34, "307": 58, "308": 116, "309": 114, "310": 117, "311": 101, "312": 125, "313": 44, "314": 34, "315": 105, "316": 100, "317": 34, "318": 58, "319": 34, "320": 101, "321": 118, "322": 45, "323": 49, "324": 55, "325": 48, "326": 48, "327": 48, "328": 48, "329": 49, "330": 48, "331": 48, "332": 48, "333": 34, "334": 44, "335": 34, "336": 116, "337": 114, "338": 105, "339": 103, "340": 103, "341": 101, "342": 114, "343": 34, "344": 58, "345": 34, "346": 108, "347": 111, "348": 97, "349": 100, "350": 34, "351": 44, "352": 34, "353": 101, "354": 110, "355": 116, "356": 105, "357": 116, "358": 121, "359": 34, "360": 58, "361": 34, "362": 112, "363": 97, "364": 103, "365": 101, "366": 34, "367": 44, "368": 34, "369": 97, "370": 99, "371": 116, "372": 105, "373": 111, "374": 110, "375": 34, "376": 58, "377": 34, "378": 118, "379": 105, "380": 101, "381": 119, "382": 34, "383": 44, "384": 34, "385": 116, "386": 105, "387": 109, "388": 101, "389": 115, "390": 116, "391": 97, "392": 109, "393": 112, "394": 34, "395": 58, "396": 49, "397": 55, "398": 48, "399": 48, "400": 48, "401": 48, "402": 49, "403": 48, "404": 48, "405": 48, "406": 44, "407": 34, "408": 116, "409": 105, "410": 109, "411": 105, "412": 110, "413": 103, "414": 34, "415": 58, "416": 51, "417": 46, "418": 49, "419": 52, "420": 44, "421": 34, "422": 115, "423": 111, "424": 117, "425": 114, "426": 99, "427": 101, "428": 34, "429": 58, "430": 123, "431": 34, "432": 99, "433": 111, "434": 117, "435": 110, "436": 116, "437": 34, "438": 58, "439": 49, "440": 44, "441": 34, "442": 116, "443": 114, "444": 97, "445": 99, "446": 101, "447": 34, "448": 58, "449": 34, "450": 48, "451": 97, "452": 49, "453": 98, "454": 50, "455": 99, "456": 51, "457": 100, "458": 52, "459": 101, "460": 53, "461": 102, "462": 54, "463": 48, "464": 55, "465": 49, "466": 56, "467": 50, "468": 57, "469": 51, "470": 97, "471": 52, "472": 98, "473": 53, "474": 99, "475": 54, "476": 100, "477": 55, "478": 101, "479": 56, "480": 102, "481": 57, "482": 34, "483": 44, "484": 34, "485": 116, "486": 121, "487": 112, "488": 101, "489": 34, "490": 58, "491": 34, "492": 101, "493": 120, "494": 112, "495": 114, "496": 101, "497": 115, "498": 115, "499": 34, "500": 44, "501": 34, "502": 112, "503": 108, "504": 97, "505": 116, "506": 102, "507": 111, "508": 114, "509": 109, "510": 34, "511": 58, "512": 34, "513": 115, "514": 101, "515": 114, "516": 118, "517": 101, "518": 114, "519": 34, "520": 125, "521": 125 } } ] }) ``` ### Order record An order complete event is serialized and delivered to Firehose as a batch record for downstream storage. Event ``` { "name": "order complete", "data": { "id": "ORD-400", "total": 99.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700001001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700001001, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` client.send({ "DeliveryStreamName": "walkeros-events", "Records": [ { "Data": { "0": 123, "1": 34, "2": 110, "3": 97, "4": 109, "5": 101, "6": 34, "7": 58, "8": 34, "9": 111, "10": 114, "11": 100, "12": 101, "13": 114, "14": 32, "15": 99, "16": 111, "17": 109, "18": 112, "19": 108, "20": 101, "21": 116, "22": 101, "23": 34, "24": 44, "25": 34, "26": 100, "27": 97, "28": 116, "29": 97, "30": 34, "31": 58, "32": 123, "33": 34, "34": 105, "35": 100, "36": 34, "37": 58, "38": 34, "39": 79, "40": 82, "41": 68, "42": 45, "43": 52, "44": 48, "45": 48, "46": 34, "47": 44, "48": 34, "49": 116, "50": 111, "51": 116, "52": 97, "53": 108, "54": 34, "55": 58, "56": 57, "57": 57, "58": 46, "59": 57, "60": 57, "61": 44, "62": 34, "63": 99, "64": 117, "65": 114, "66": 114, "67": 101, "68": 110, "69": 99, "70": 121, "71": 34, "72": 58, "73": 34, "74": 69, "75": 85, "76": 82, "77": 34, "78": 125, "79": 44, "80": 34, "81": 99, "82": 111, "83": 110, "84": 116, "85": 101, "86": 120, "87": 116, "88": 34, "89": 58, "90": 123, "91": 34, "92": 115, "93": 104, "94": 111, "95": 112, "96": 112, "97": 105, "98": 110, "99": 103, "100": 34, "101": 58, "102": 91, "103": 34, "104": 99, "105": 111, "106": 109, "107": 112, "108": 108, "109": 101, "110": 116, "111": 101, "112": 34, "113": 44, "114": 48, "115": 93, "116": 125, "117": 44, "118": 34, "119": 103, "120": 108, "121": 111, "122": 98, "123": 97, "124": 108, "125": 115, "126": 34, "127": 58, "128": 123, "129": 34, "130": 112, "131": 97, "132": 103, "133": 101, "134": 103, "135": 114, "136": 111, "137": 117, "138": 112, "139": 34, "140": 58, "141": 34, "142": 115, "143": 104, "144": 111, "145": 112, "146": 34, "147": 125, "148": 44, "149": 34, "150": 99, "151": 117, "152": 115, "153": 116, "154": 111, "155": 109, "156": 34, "157": 58, "158": 123, "159": 34, "160": 99, "161": 111, "162": 109, "163": 112, "164": 108, "165": 101, "166": 116, "167": 101, "168": 108, "169": 121, "170": 34, "171": 58, "172": 34, "173": 114, "174": 97, "175": 110, "176": 100, "177": 111, "178": 109, "179": 34, "180": 125, "181": 44, "182": 34, "183": 117, "184": 115, "185": 101, "186": 114, "187": 34, "188": 58, "189": 123, "190": 34, "191": 105, "192": 100, "193": 34, "194": 58, "195": 34, "196": 117, "197": 115, "198": 51, "199": 114, "200": 34, "201": 44, "202": 34, "203": 100, "204": 101, "205": 118, "206": 105, "207": 99, "208": 101, "209": 34, "210": 58, "211": 34, "212": 99, "213": 48, "214": 48, "215": 107, "216": 49, "217": 51, "218": 34, "219": 44, "220": 34, "221": 115, "222": 101, "223": 115, "224": 115, "225": 105, "226": 111, "227": 110, "228": 34, "229": 58, "230": 34, "231": 115, "232": 51, "233": 115, "234": 115, "235": 49, "236": 48, "237": 110, "238": 34, "239": 125, "240": 44, "241": 34, "242": 110, "243": 101, "244": 115, "245": 116, "246": 101, "247": 100, "248": 34, "249": 58, "250": 91, "251": 123, "252": 34, "253": 101, "254": 110, "255": 116, "256": 105, "257": 116, "258": 121, "259": 34, "260": 58, "261": 34, "262": 112, "263": 114, "264": 111, "265": 100, "266": 117, "267": 99, "268": 116, "269": 34, "270": 44, "271": 34, "272": 100, "273": 97, "274": 116, "275": 97, "276": 34, "277": 58, "278": 123, "279": 34, "280": 105, "281": 100, "282": 34, "283": 58, "284": 34, "285": 101, "286": 114, "287": 115, "288": 34, "289": 44, "290": 34, "291": 110, "292": 97, "293": 109, "294": 101, "295": 34, "296": 58, "297": 34, "298": 69, "299": 118, "300": 101, "301": 114, "302": 121, "303": 100, "304": 97, "305": 121, "306": 32, "307": 82, "308": 117, "309": 99, "310": 107, "311": 32, "312": 83, "313": 110, "314": 97, "315": 99, "316": 107, "317": 34, "318": 44, "319": 34, "320": 99, "321": 111, "322": 108, "323": 111, "324": 114, "325": 34, "326": 58, "327": 34, "328": 98, "329": 108, "330": 97, "331": 99, "332": 107, "333": 34, "334": 44, "335": 34, "336": 115, "337": 105, "338": 122, "339": 101, "340": 34, "341": 58, "342": 34, "343": 108, "344": 34, "345": 44, "346": 34, "347": 112, "348": 114, "349": 105, "350": 99, "351": 101, "352": 34, "353": 58, "354": 52, "355": 50, "356": 48, "357": 125, "358": 44, "359": 34, "360": 99, "361": 111, "362": 110, "363": 116, "364": 101, "365": 120, "366": 116, "367": 34, "368": 58, "369": 123, "370": 34, "371": 115, "372": 104, "373": 111, "374": 112, "375": 112, "376": 105, "377": 110, "378": 103, "379": 34, "380": 58, "381": 91, "382": 34, "383": 99, "384": 111, "385": 109, "386": 112, "387": 108, "388": 101, "389": 116, "390": 101, "391": 34, "392": 44, "393": 48, "394": 93, "395": 125, "396": 44, "397": 34, "398": 110, "399": 101, "400": 115, "401": 116, "402": 101, "403": 100, "404": 34, "405": 58, "406": 91, "407": 93, "408": 125, "409": 44, "410": 123, "411": 34, "412": 101, "413": 110, "414": 116, "415": 105, "416": 116, "417": 121, "418": 34, "419": 58, "420": 34, "421": 112, "422": 114, "423": 111, "424": 100, "425": 117, "426": 99, "427": 116, "428": 34, "429": 44, "430": 34, "431": 100, "432": 97, "433": 116, "434": 97, "435": 34, "436": 58, "437": 123, "438": 34, "439": 105, "440": 100, "441": 34, "442": 58, "443": 34, "444": 99, "445": 99, "446": 34, "447": 44, "448": 34, "449": 110, "450": 97, "451": 109, "452": 101, "453": 34, "454": 58, "455": 34, "456": 67, "457": 111, "458": 111, "459": 108, "460": 32, "461": 67, "462": 97, "463": 112, "464": 34, "465": 44, "466": 34, "467": 115, "468": 105, "469": 122, "470": 101, "471": 34, "472": 58, "473": 34, "474": 111, "475": 110, "476": 101, "477": 32, "478": 115, "479": 105, "480": 122, "481": 101, "482": 34, "483": 44, "484": 34, "485": 112, "486": 114, "487": 105, "488": 99, "489": 101, "490": 34, "491": 58, "492": 52, "493": 50, "494": 125, "495": 44, "496": 34, "497": 99, "498": 111, "499": 110, "500": 116, "501": 101, "502": 120, "503": 116, "504": 34, "505": 58, "506": 123, "507": 34, "508": 115, "509": 104, "510": 111, "511": 112, "512": 112, "513": 105, "514": 110, "515": 103, "516": 34, "517": 58, "518": 91, "519": 34, "520": 99, "521": 111, "522": 109, "523": 112, "524": 108, "525": 101, "526": 116, "527": 101, "528": 34, "529": 44, "530": 48, "531": 93, "532": 125, "533": 44, "534": 34, "535": 110, "536": 101, "537": 115, "538": 116, "539": 101, "540": 100, "541": 34, "542": 58, "543": 91, "544": 93, "545": 125, "546": 44, "547": 123, "548": 34, "549": 101, "550": 110, "551": 116, "552": 105, "553": 116, "554": 121, "555": 34, "556": 58, "557": 34, "558": 103, "559": 105, "560": 102, "561": 116, "562": 34, "563": 44, "564": 34, "565": 100, "566": 97, "567": 116, "568": 97, "569": 34, "570": 58, "571": 123, "572": 34, "573": 110, "574": 97, "575": 109, "576": 101, "577": 34, "578": 58, "579": 34, "580": 83, "581": 117, "582": 114, "583": 112, "584": 114, "585": 105, "586": 115, "587": 101, "588": 34, "589": 125, "590": 44, "591": 34, "592": 99, "593": 111, "594": 110, "595": 116, "596": 101, "597": 120, "598": 116, "599": 34, "600": 58, "601": 123, "602": 34, "603": 115, "604": 104, "605": 111, "606": 112, "607": 112, "608": 105, "609": 110, "610": 103, "611": 34, "612": 58, "613": 91, "614": 34, "615": 99, "616": 111, "617": 109, "618": 112, "619": 108, "620": 101, "621": 116, "622": 101, "623": 34, "624": 44, "625": 48, "626": 93, "627": 125, "628": 44, "629": 34, "630": 110, "631": 101, "632": 115, "633": 116, "634": 101, "635": 100, "636": 34, "637": 58, "638": 91, "639": 93, "640": 125, "641": 93, "642": 44, "643": 34, "644": 99, "645": 111, "646": 110, "647": 115, "648": 101, "649": 110, "650": 116, "651": 34, "652": 58, "653": 123, "654": 34, "655": 102, "656": 117, "657": 110, "658": 99, "659": 116, "660": 105, "661": 111, "662": 110, "663": 97, "664": 108, "665": 34, "666": 58, "667": 116, "668": 114, "669": 117, "670": 101, "671": 125, "672": 44, "673": 34, "674": 105, "675": 100, "676": 34, "677": 58, "678": 34, "679": 101, "680": 118, "681": 45, "682": 49, "683": 55, "684": 48, "685": 48, "686": 48, "687": 48, "688": 49, "689": 48, "690": 48, "691": 49, "692": 34, "693": 44, "694": 34, "695": 116, "696": 114, "697": 105, "698": 103, "699": 103, "700": 101, "701": 114, "702": 34, "703": 58, "704": 34, "705": 108, "706": 111, "707": 97, "708": 100, "709": 34, "710": 44, "711": 34, "712": 101, "713": 110, "714": 116, "715": 105, "716": 116, "717": 121, "718": 34, "719": 58, "720": 34, "721": 111, "722": 114, "723": 100, "724": 101, "725": 114, "726": 34, "727": 44, "728": 34, "729": 97, "730": 99, "731": 116, "732": 105, "733": 111, "734": 110, "735": 34, "736": 58, "737": 34, "738": 99, "739": 111, "740": 109, "741": 112, "742": 108, "743": 101, "744": 116, "745": 101, "746": 34, "747": 44, "748": 34, "749": 116, "750": 105, "751": 109, "752": 101, "753": 115, "754": 116, "755": 97, "756": 109, "757": 112, "758": 34, "759": 58, "760": 49, "761": 55, "762": 48, "763": 48, "764": 48, "765": 48, "766": 49, "767": 48, "768": 48, "769": 49, "770": 44, "771": 34, "772": 116, "773": 105, "774": 109, "775": 105, "776": 110, "777": 103, "778": 34, "779": 58, "780": 51, "781": 46, "782": 49, "783": 52, "784": 44, "785": 34, "786": 115, "787": 111, "788": 117, "789": 114, "790": 99, "791": 101, "792": 34, "793": 58, "794": 123, "795": 34, "796": 99, "797": 111, "798": 117, "799": 110, "800": 116, "801": 34, "802": 58, "803": 49, "804": 44, "805": 34, "806": 116, "807": 114, "808": 97, "809": 99, "810": 101, "811": 34, "812": 58, "813": 34, "814": 48, "815": 97, "816": 49, "817": 98, "818": 50, "819": 99, "820": 51, "821": 100, "822": 52, "823": 101, "824": 53, "825": 102, "826": 54, "827": 48, "828": 55, "829": 49, "830": 56, "831": 50, "832": 57, "833": 51, "834": 97, "835": 52, "836": 98, "837": 53, "838": 99, "839": 54, "840": 100, "841": 55, "842": 101, "843": 56, "844": 102, "845": 57, "846": 34, "847": 44, "848": 34, "849": 116, "850": 121, "851": 112, "852": 101, "853": 34, "854": 58, "855": 34, "856": 101, "857": 120, "858": 112, "859": 114, "860": 101, "861": 115, "862": 115, "863": 34, "864": 44, "865": 34, "866": 112, "867": 108, "868": 97, "869": 116, "870": 102, "871": 111, "872": 114, "873": 109, "874": 34, "875": 58, "876": 34, "877": 115, "878": 101, "879": 114, "880": 118, "881": 101, "882": 114, "883": 34, "884": 125, "885": 125 } } ] }) ``` ### User signup record A user signup event including user fields is streamed to Firehose as a JSON record. Event ``` { "name": "user signup", "data": { "plan": "pro", "source": "landing-page" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "usr-789", "email": "new@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700001002", "trigger": "test", "entity": "user", "action": "signup", "timestamp": 1700001002, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` client.send({ "DeliveryStreamName": "walkeros-events", "Records": [ { "Data": { "0": 123, "1": 34, "2": 110, "3": 97, "4": 109, "5": 101, "6": 34, "7": 58, "8": 34, "9": 117, "10": 115, "11": 101, "12": 114, "13": 32, "14": 115, "15": 105, "16": 103, "17": 110, "18": 117, "19": 112, "20": 34, "21": 44, "22": 34, "23": 100, "24": 97, "25": 116, "26": 97, "27": 34, "28": 58, "29": 123, "30": 34, "31": 112, "32": 108, "33": 97, "34": 110, "35": 34, "36": 58, "37": 34, "38": 112, "39": 114, "40": 111, "41": 34, "42": 44, "43": 34, "44": 115, "45": 111, "46": 117, "47": 114, "48": 99, "49": 101, "50": 34, "51": 58, "52": 34, "53": 108, "54": 97, "55": 110, "56": 100, "57": 105, "58": 110, "59": 103, "60": 45, "61": 112, "62": 97, "63": 103, "64": 101, "65": 34, "66": 125, "67": 44, "68": 34, "69": 99, "70": 111, "71": 110, "72": 116, "73": 101, "74": 120, "75": 116, "76": 34, "77": 58, "78": 123, "79": 34, "80": 100, "81": 101, "82": 118, "83": 34, "84": 58, "85": 91, "86": 34, "87": 116, "88": 101, "89": 115, "90": 116, "91": 34, "92": 44, "93": 49, "94": 93, "95": 125, "96": 44, "97": 34, "98": 103, "99": 108, "100": 111, "101": 98, "102": 97, "103": 108, "104": 115, "105": 34, "106": 58, "107": 123, "108": 34, "109": 108, "110": 97, "111": 110, "112": 103, "113": 34, "114": 58, "115": 34, "116": 101, "117": 108, "118": 98, "119": 34, "120": 125, "121": 44, "122": 34, "123": 99, "124": 117, "125": 115, "126": 116, "127": 111, "128": 109, "129": 34, "130": 58, "131": 123, "132": 34, "133": 99, "134": 111, "135": 109, "136": 112, "137": 108, "138": 101, "139": 116, "140": 101, "141": 108, "142": 121, "143": 34, "144": 58, "145": 34, "146": 114, "147": 97, "148": 110, "149": 100, "150": 111, "151": 109, "152": 34, "153": 125, "154": 44, "155": 34, "156": 117, "157": 115, "158": 101, "159": 114, "160": 34, "161": 58, "162": 123, "163": 34, "164": 105, "165": 100, "166": 34, "167": 58, "168": 34, "169": 117, "170": 115, "171": 114, "172": 45, "173": 55, "174": 56, "175": 57, "176": 34, "177": 44, "178": 34, "179": 101, "180": 109, "181": 97, "182": 105, "183": 108, "184": 34, "185": 58, "186": 34, "187": 110, "188": 101, "189": 119, "190": 64, "191": 101, "192": 120, "193": 97, "194": 109, "195": 112, "196": 108, "197": 101, "198": 46, "199": 99, "200": 111, "201": 109, "202": 34, "203": 125, "204": 44, "205": 34, "206": 110, "207": 101, "208": 115, "209": 116, "210": 101, "211": 100, "212": 34, "213": 58, "214": 91, "215": 123, "216": 34, "217": 101, "218": 110, "219": 116, "220": 105, "221": 116, "222": 121, "223": 34, "224": 58, "225": 34, "226": 99, "227": 104, "228": 105, "229": 108, "230": 100, "231": 34, "232": 44, "233": 34, "234": 100, "235": 97, "236": 116, "237": 97, "238": 34, "239": 58, "240": 123, "241": 34, "242": 105, "243": 115, "244": 34, "245": 58, "246": 34, "247": 115, "248": 117, "249": 98, "250": 111, "251": 114, "252": 100, "253": 105, "254": 110, "255": 97, "256": 116, "257": 101, "258": 100, "259": 34, "260": 125, "261": 125, "262": 93, "263": 44, "264": 34, "265": 99, "266": 111, "267": 110, "268": 115, "269": 101, "270": 110, "271": 116, "272": 34, "273": 58, "274": 123, "275": 34, "276": 102, "277": 117, "278": 110, "279": 99, "280": 116, "281": 105, "282": 111, "283": 110, "284": 97, "285": 108, "286": 34, "287": 58, "288": 116, "289": 114, "290": 117, "291": 101, "292": 125, "293": 44, "294": 34, "295": 105, "296": 100, "297": 34, "298": 58, "299": 34, "300": 101, "301": 118, "302": 45, "303": 49, "304": 55, "305": 48, "306": 48, "307": 48, "308": 48, "309": 49, "310": 48, "311": 48, "312": 50, "313": 34, "314": 44, "315": 34, "316": 116, "317": 114, "318": 105, "319": 103, "320": 103, "321": 101, "322": 114, "323": 34, "324": 58, "325": 34, "326": 116, "327": 101, "328": 115, "329": 116, "330": 34, "331": 44, "332": 34, "333": 101, "334": 110, "335": 116, "336": 105, "337": 116, "338": 121, "339": 34, "340": 58, "341": 34, "342": 117, "343": 115, "344": 101, "345": 114, "346": 34, "347": 44, "348": 34, "349": 97, "350": 99, "351": 116, "352": 105, "353": 111, "354": 110, "355": 34, "356": 58, "357": 34, "358": 115, "359": 105, "360": 103, "361": 110, "362": 117, "363": 112, "364": 34, "365": 44, "366": 34, "367": 116, "368": 105, "369": 109, "370": 101, "371": 115, "372": 116, "373": 97, "374": 109, "375": 112, "376": 34, "377": 58, "378": 49, "379": 55, "380": 48, "381": 48, "382": 48, "383": 48, "384": 49, "385": 48, "386": 48, "387": 50, "388": 44, "389": 34, "390": 116, "391": 105, "392": 109, "393": 105, "394": 110, "395": 103, "396": 34, "397": 58, "398": 51, "399": 46, "400": 49, "401": 52, "402": 44, "403": 34, "404": 115, "405": 111, "406": 117, "407": 114, "408": 99, "409": 101, "410": 34, "411": 58, "412": 123, "413": 34, "414": 99, "415": 111, "416": 117, "417": 110, "418": 116, "419": 34, "420": 58, "421": 49, "422": 44, "423": 34, "424": 116, "425": 114, "426": 97, "427": 99, "428": 101, "429": 34, "430": 58, "431": 34, "432": 48, "433": 97, "434": 49, "435": 98, "436": 50, "437": 99, "438": 51, "439": 100, "440": 52, "441": 101, "442": 53, "443": 102, "444": 54, "445": 48, "446": 55, "447": 49, "448": 56, "449": 50, "450": 57, "451": 51, "452": 97, "453": 52, "454": 98, "455": 53, "456": 99, "457": 54, "458": 100, "459": 55, "460": 101, "461": 56, "462": 102, "463": 57, "464": 34, "465": 44, "466": 34, "467": 116, "468": 121, "469": 112, "470": 101, "471": 34, "472": 58, "473": 34, "474": 101, "475": 120, "476": 112, "477": 114, "478": 101, "479": 115, "480": 115, "481": 34, "482": 44, "483": 34, "484": 112, "485": 108, "486": 97, "487": 116, "488": 102, "489": 111, "490": 114, "491": 109, "492": 34, "493": 58, "494": 34, "495": 115, "496": 101, "497": 114, "498": 118, "499": 101, "500": 114, "501": 34, "502": 125, "503": 125 } } ] }) ``` --- # Bing UET Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/bing) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/bing)[ ](https://www.npmjs.com/package/@walkeros/server-destination-bing) [Package](https://www.npmjs.com/package/@walkeros/server-destination-bing) Beta Server-side event delivery to Microsoft Advertising's Universal Event Tracking (UET) Conversions API for enhanced conversion tracking in Microsoft Ads campaigns, bypassing browser limitations and improving match quality. Where this fits Bing UET CAPI is a **server destination** in the walkerOS flow: Sends events server-side to Microsoft's UET Conversions API using raw HTTP (no SDK). Identity fields (\`em\`, \`ph\`) are SHA-256 hashed before sending with Microsoft-specific email normalization. Events include a stable \`eventId\` for deduplication with the UET JavaScript tag. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-bing ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationBing } from '@walkeros/server-destination-bing'; await startFlow({ destinations: { bing: { code: destinationBing, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', tagId: 'YOUR_UET_TAG_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "bing": { "package": "@walkeros/server-destination-bing", "import": "destinationBing", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "tagId": "YOUR_UET_TAG_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------------------- | ------------------------ | --------------------------------------------------------------------------- | ---- | | `accessToken*` | `string` | Long-lived Bing UET CAPI access token from Microsoft Advertising | | | `tagId*` | `string` | Microsoft Advertising UET tag ID | | | `url` | `string` | Custom Bing UET CAPI base URL (default https\://capi.uet.microsoft.com/v1/) | | | `doNotHash` | `Array` | User data fields to skip hashing | | | `user_data` | `Record` | Mapping for user data fields | | | `dataProvider` | `string` | Identifier of the data source (default "walkerOS") | | | `continueOnValidationError` | `boolean` | When true, Microsoft continues to ingest events despite validation errors | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ----------- | ------------------------ | ---------------------------------------------------------------------------- | ---- | | `eventType` | `'pageLoad' \| 'custom'` | Override event type: "pageLoad" for page views, "custom" (default) otherwise | | ## Examples ### Add to cart A product add is sent to Bing UET as an add\_to\_cart event with value, currency, and item details. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "color": "blue" }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "quantity": 1 } } ], "consent": { "functional": true }, "id": "b1c2d3e4f5a60004", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000903000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products/running-shoes" } } ``` Mapping ``` { "name": "add_to_cart", "data": { "map": { "customData": { "map": { "value": "data.price", "currency": { "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>b(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "name": "data.name", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } } } ``` Out ``` sendServer("https://capi.uet.microsoft.com/v1/UET-12345/events", "{\"data\":[{\"eventType\":\"custom\",\"eventId\":\"b1c2d3e4f5a60004\",\"eventTime\":1700000903,\"adStorageConsent\":\"G\",\"userData\":{},\"eventName\":\"add_to_cart\",\"eventSourceUrl\":\"https://shop.example.com/products/running-shoes\",\"customData\":{\"value\":89.99,\"currency\":\"EUR\",\"items\":[{\"id\":\"SKU-B2\",\"name\":\"Running Shoes\",\"price\":89.99,\"quantity\":1}]}}],\"dataProvider\":\"walkerOS\"}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json" } }) ``` ### Lead A newsletter form submission is sent to Bing UET as a lead event with the SHA-256 hashed email. Event ``` { "name": "form submit", "data": { "type": "newsletter" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "b1c2d3e4f5a60003", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/contact" } } ``` Mapping ``` { "name": "lead", "data": { "map": { "userData": { "map": { "em": "user.email" } } } } } ``` Out ``` sendServer("https://capi.uet.microsoft.com/v1/UET-12345/events", "{\"data\":[{\"eventType\":\"custom\",\"eventId\":\"b1c2d3e4f5a60003\",\"eventTime\":1700000902,\"adStorageConsent\":\"G\",\"userData\":{\"em\":\"b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514\"},\"eventName\":\"lead\",\"eventSourceUrl\":\"https://example.com/contact\"}],\"dataProvider\":\"walkerOS\"}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json" } }) ``` ### Page view A page view is sent to the Bing UET CAPI with eventType pageLoad and the source URL. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "b1c2d3e4f5a60002", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/docs/" } } ``` Mapping ``` { "settings": { "eventType": "pageLoad" } } ``` Out ``` sendServer("https://capi.uet.microsoft.com/v1/UET-12345/events", "{\"data\":[{\"eventType\":\"pageLoad\",\"eventId\":\"b1c2d3e4f5a60002\",\"eventTime\":1700000901,\"adStorageConsent\":\"G\",\"userData\":{},\"eventSourceUrl\":\"https://example.com/docs/\"}],\"dataProvider\":\"walkerOS\"}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json" } }) ``` ### Purchase A completed order is sent to the Bing UET CAPI with transaction id, value, items, and hashed user data. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Widget Pro", "price": 124.99, "quantity": 2 } } ], "consent": { "functional": true }, "id": "b1c2d3e4f5a60001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/checkout/complete" } } ``` Mapping ``` { "name": "purchase", "data": { "map": { "customData": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "transactionId": "data.id", "pageType": { "value": "purchase" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>b(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "name": "data.name", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } }, "userData": { "map": { "externalId": "user.id" } } } } } ``` Out ``` sendServer("https://capi.uet.microsoft.com/v1/UET-12345/events", "{\"data\":[{\"eventType\":\"custom\",\"eventId\":\"b1c2d3e4f5a60001\",\"eventTime\":1700000900,\"adStorageConsent\":\"G\",\"userData\":{\"externalId\":\"user-123\"},\"eventName\":\"purchase\",\"eventSourceUrl\":\"https://shop.example.com/checkout/complete\",\"customData\":{\"value\":249.99,\"currency\":\"EUR\",\"transactionId\":\"ORD-300\",\"pageType\":\"purchase\",\"items\":[{\"id\":\"SKU-A1\",\"name\":\"Widget Pro\",\"price\":124.99,\"quantity\":2}]}}],\"dataProvider\":\"walkerOS\"}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json" } }) ``` ## Event types[​](#event-types "Direct link to Event types") Bing UET distinguishes two event types. Set the type via `mapping.settings.eventType`: | `eventType` | When to use | | ----------- | ----------------------------------------------------------- | | `pageLoad` | Page view events (no `eventName` required) | | `custom` | All other conversion events (default, requires `eventName`) | ``` "mapping": { "page": { "view": { "settings": { "eventType": "pageLoad" } } }, "order": { "complete": { "name": "purchase", "data": { "map": { "customData": { "map": { "value": "data.total", "currency": "data.currency", "transactionId": "data.id", "pageType": { "value": "purchase" } } } } } } } } ``` ### customData structure[​](#customdata-structure "Direct link to customData structure") Event properties must be nested under `customData`. Common fields include `value`, `currency`, `transactionId`, `items`, `itemIds`, `pageType`, `eventCategory`, `eventLabel`, `eventValue`, `searchTerm`. ## Hashing[​](#hashing "Direct link to Hashing") Only `em` (email) and `ph` (phone) are SHA-256 hashed before sending. All other identity fields pass through as-is: `anonymousId`, `externalId`, `msclkid`, `clientIpAddress`, `clientUserAgent`, `idfa`, `gaid`. ### Microsoft-specific email normalization[​](#microsoft-specific-email-normalization "Direct link to Microsoft-specific email normalization") Before hashing, emails are normalized to match Microsoft's canonicalization: 1. Trim whitespace 2. Lowercase the entire address 3. Remove dots from the user portion (`a.b.c@example.com` becomes `abc@example.com`) 4. Strip `+alias` suffix (`user+promo@example.com` becomes `user@example.com`) Pass raw values and hashing is handled for you. If a value is already hashed or you want to skip hashing for a specific field, use `doNotHash`: ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "tagId": "YOUR_UET_TAG_ID", "doNotHash": ["em"] } ``` ## Deduplication[​](#deduplication "Direct link to Deduplication") Each event is sent with `eventId` set to the walkerOS event `id`. If you also run the UET JavaScript tag in the browser, Microsoft deduplicates server and browser events that share the same `eventId` and `eventName`, so conversions are not double-counted. ## Consent[​](#consent "Direct link to Consent") All events sent to this destination include `adStorageConsent: "G"` (granted). Gate the destination via walkerOS consent rules at the collector level rather than sending denied events. ## Links[​](#links "Direct link to Links") * [Microsoft UET Conversions API documentation](https://learn.microsoft.com/en-us/advertising/guides/universal-event-tracking-capi) * [Microsoft Advertising CAPI reference](https://help.ads.microsoft.com/) --- # Criteo Events API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/criteo) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/criteo)[ ](https://www.npmjs.com/package/@walkeros/server-destination-criteo) [Package](https://www.npmjs.com/package/@walkeros/server-destination-criteo) Beta Server-side event delivery to Criteo's Events API (S2S v0) for reliable retargeting, bypassing browser limitations and providing consistent user matching for ad optimization. Where this fits Criteo Events API is a **server destination** in the walkerOS flow: Sends events server-side to Criteo using raw HTTP (no SDK). Authentication is in-payload (\`account\` = Partner ID, \`id.mapping\_key\` = Caller ID). Emails are hashed into MD5, SHA-256, and SHA-256(MD5) before sending. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-criteo ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationCriteo } from '@walkeros/server-destination-criteo'; await startFlow({ destinations: { criteo: { code: destinationCriteo, config: { settings: { partnerId: 'YOUR_PARTNER_ID', callerId: 'YOUR_CALLER_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "criteo": { "package": "@walkeros/server-destination-criteo", "import": "destinationCriteo", "config": { "settings": { "partnerId": "YOUR_PARTNER_ID", "callerId": "YOUR_CALLER_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------ | ------------------------ | --------------------------------------------------------------------------------------- | ---- | | `partnerId*` | `string` | Criteo Partner ID (numeric string, provided by Criteo) | | | `callerId*` | `string` | Caller ID for user mapping (provided by Criteo) | | | `siteType` | `'d' \| 'm' \| 't'` | Site type: d (desktop), m (mobile web), t (tablet) | | | `country` | `string` | ISO 3166-1 alpha-2 country code | | | `language` | `string` | 2-letter language code | | | `url` | `string` | Custom Events API endpoint (default https\://widget.criteo.com/m/event?version=s2s\_v0) | | | `user_data` | `Record` | Mapping for identity fields | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add becomes a Criteo addToCart event with the item id, price, and quantity. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "quantity": 1 } } ], "consent": { "functional": true }, "id": "56b75647d0f8bdc4", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products/running-shoes" } } ``` Mapping ``` { "name": "addToCart", "data": { "map": { "item": { "loop": [ "nested", { "condition": { "$code": "e=>w(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` sendServer("https://widget.criteo.com/m/event?version=s2s_v0", "{\"version\":\"walkeros_criteo_1.0.0\",\"site_type\":\"d\",\"account\":\"PARTNER_ID\",\"id\":{\"mapping_key\":\"CALLER_ID\"},\"events\":[{\"event\":\"addToCart\",\"timestamp\":\"2023-11-14T22:28:21.000Z\",\"item\":[{\"id\":\"SKU-B2\",\"price\":89.99,\"quantity\":1}]}],\"full_url\":\"https://shop.example.com/products/running-shoes\"}") ``` ### Page view A page view becomes a Criteo viewHome event used for home page impression tracking. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a4294bcdcdd51674", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000903000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/" } } ``` Mapping ``` { "name": "viewHome" } ``` Out ``` sendServer("https://widget.criteo.com/m/event?version=s2s_v0", "{\"version\":\"walkeros_criteo_1.0.0\",\"site_type\":\"d\",\"account\":\"PARTNER_ID\",\"id\":{\"mapping_key\":\"CALLER_ID\"},\"events\":[{\"event\":\"viewHome\",\"timestamp\":\"2023-11-14T22:28:23.000Z\"}],\"full_url\":\"https://example.com/\"}") ``` ### Purchase A completed order is posted to the Criteo Events API as a trackTransaction event with items. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Widget Pro", "price": 124.99, "quantity": 2 } } ], "consent": { "functional": true }, "id": "c0e9b6c2d37be3f9", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/checkout/complete", "referrer": "https://shop.example.com/cart" } } ``` Mapping ``` { "name": "trackTransaction", "data": { "map": { "id": "data.id", "item": { "loop": [ "nested", { "condition": { "$code": "e=>w(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` sendServer("https://widget.criteo.com/m/event?version=s2s_v0", "{\"version\":\"walkeros_criteo_1.0.0\",\"site_type\":\"d\",\"account\":\"PARTNER_ID\",\"id\":{\"mapping_key\":\"CALLER_ID\"},\"events\":[{\"event\":\"trackTransaction\",\"timestamp\":\"2023-11-14T22:28:20.000Z\",\"id\":\"ORD-300\",\"item\":[{\"id\":\"SKU-A1\",\"price\":124.99,\"quantity\":2}]}],\"full_url\":\"https://shop.example.com/checkout/complete\",\"previous_url\":\"https://shop.example.com/cart\"}") ``` ### View item A product view becomes a Criteo viewItem event with the viewed product id. Event ``` { "name": "product view", "data": { "id": "SKU-C3", "name": "Coffee Maker" }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "SKU-C3" } } ], "consent": { "functional": true }, "id": "fb475670cdee3ec9", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products/coffee-maker" } } ``` Mapping ``` { "name": "viewItem", "data": { "map": { "item": { "loop": [ "nested", { "condition": { "$code": "e=>w(e)&&\"product\"===e.entity" }, "map": { "id": "data.id" } } ] } } } } ``` Out ``` sendServer("https://widget.criteo.com/m/event?version=s2s_v0", "{\"version\":\"walkeros_criteo_1.0.0\",\"site_type\":\"d\",\"account\":\"PARTNER_ID\",\"id\":{\"mapping_key\":\"CALLER_ID\"},\"events\":[{\"event\":\"viewItem\",\"timestamp\":\"2023-11-14T22:28:22.000Z\",\"item\":[{\"id\":\"SKU-C3\"}]}],\"full_url\":\"https://shop.example.com/products/coffee-maker\"}") ``` ## Event mapping[​](#event-mapping "Direct link to Event mapping") Criteo uses its own standard event names. Set the Criteo event name via each rule's `name` field. | walkerOS Event | Criteo Event Name | | ------------------ | ----------------------- | | `page view` | `viewHome` / `viewPage` | | `product view` | `viewItem` | | `product list` | `viewList` | | `product add` | `addToCart` | | `cart view` | `viewBasket` | | `checkout start` | `beginCheckout` | | `checkout payment` | `addPaymentInfo` | | `order complete` | `trackTransaction` | | `user login` | `login` | ``` "mapping": { "order": { "complete": { "name": "trackTransaction", "data": { "map": { "id": "data.id", "item": { "loop": [ "nested", { "map": { "id": "data.id", "price": "data.price", "quantity": "data.quantity" } } ] } } } } } } ``` ## User identity[​](#user-identity "Direct link to User identity") Criteo matches users via three signals. Wire them through `user_data` in settings: | Field | Description | | --------------------- | ---------------------------------------------------- | | `mapped_user_id` | Criteo GUM ID | | `email` | Raw email. Hashed into MD5, SHA-256 and SHA-256(MD5) | | `retailer_visitor_id` | Stable retailer visitor identifier | ``` "settings": { "partnerId": "YOUR_PARTNER_ID", "callerId": "YOUR_CALLER_ID", "user_data": { "mapped_user_id": "user.id", "email": "user.email", "retailer_visitor_id": "user.device" } } ``` ## Email hashing[​](#email-hashing "Direct link to Email hashing") Raw emails are lowercased and trimmed, then hashed into all three formats Criteo expects: * `md5`: MD5 hex digest * `sha256`: SHA-256 hex digest * `sha256_md5`: SHA-256 hex of the MD5 hex string Pre-hashed values (detected by hex length: 32 chars for MD5, 64 for SHA-256) are passed through without re-hashing. ## Deduplication[​](#deduplication "Direct link to Deduplication") Criteo Events API supports deduplication with Criteo OneTag by sharing a `deduplication_page_view_id` across channels. Wire it via `data.map.deduplication_page_view_id` in the event mapping. ## Links[​](#links "Direct link to Links") * [Criteo Events API Guide](https://guides.criteotilt.com/events-api/) --- # Customer.io [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/customerio) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/customerio)[ ](https://www.npmjs.com/package/@walkeros/server-destination-customerio) [Package](https://www.npmjs.com/package/@walkeros/server-destination-customerio) Beta Server-side event delivery to [Customer.io](https://customer.io/) via the official [`customerio-node`](https://www.npmjs.com/package/customerio-node) SDK. Tracks events with `TrackClient.track()`, manages identities with `identify()`, and supports the full lifecycle: page views, device registration, suppress/unsuppress, profile merging, and transactional messaging via `APIClient.sendEmail()` / `sendPush()`. Where this fits Customer.io is a **server destination** in the walkerOS flow: Receives events server-side from the collector, resolves a \`customerId\` (or falls back to \`anonymousId\`), optionally fires \`identify()\`, then forwards the event to Customer.io for campaigns, broadcasts, and transactional messaging. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-customerio ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationCustomerIo } from '@walkeros/server-destination-customerio'; await startFlow({ destinations: { customerio: { code: destinationCustomerIo, config: { settings: { siteId: 'YOUR_SITE_ID', apiKey: 'YOUR_API_KEY', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "customerio": { "package": "@walkeros/server-destination-customerio", "config": { "settings": { "siteId": "YOUR_SITE_ID", "apiKey": "YOUR_API_KEY" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `siteId*` | `string` | Customer.io Site ID. Find it in Settings > Workspace Settings > API Credentials. | | | `apiKey*` | `string` | Customer.io API Key. Find it in Settings > Workspace Settings > API Credentials. | | | `appApiKey` | `string` | App API Key for transactional messaging (sendEmail/sendPush). Find it in Settings > Workspace Settings > API Credentials > App API Keys. | | | `region` | `'us' \| 'eu'` | Data center region. Must match where your Customer.io workspace was created. Default: us. | | | `timeout` | `integer` | HTTP request timeout in milliseconds. Default: 10000. | | | `customerId` | `string` | walkerOS mapping value path to resolve customerId from each event (like user.id). | | | `anonymousId` | `string` | walkerOS mapping value path to resolve anonymousId from each event (like user.session). | | | `identify` | `any` | Destination-level identity mapping. Resolves to { email?, first\_name?, ... } attributes. Fires identify() on first push and re-fires when values change. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identify attributes. Resolves to { email?, first\_name?, ... }. Use with silent: true on login/identify events. | | | `page` | `any` | Per-event page view. Resolves to { url, ... }. Calls trackPageView(). Use with silent: true. | | | `destroy` | `boolean` | Permanently delete person from Customer.io. Set true on delete events with silent: true. | | | `suppress` | `boolean` | Suppress person (stop messaging without deleting data). Set true with silent: true. | | | `unsuppress` | `boolean` | Unsuppress person (resume messaging). Set true with silent: true. | | | `addDevice` | `any` | Register push device. Resolves to { deviceId, platform, data? }. Use with silent: true. | | | `deleteDevice` | `any` | Remove push device. Resolves to { deviceId, platform }. Use with silent: true. | | | `merge` | `any` | Merge duplicate profiles. Resolves to { primaryType, primaryId, secondaryType, secondaryId }. Use with silent: true. | | | `sendEmail` | `any` | Send transactional email. Resolves to { to, transactional\_message\_id, message\_data?, identifiers? }. Requires appApiKey. Use with silent: true. | | | `sendPush` | `any` | Send transactional push. Resolves to { transactional\_message\_id, message\_data?, identifiers? }. Requires appApiKey. Use with silent: true. | | ## Examples ### Anonymous track When no customer id is resolved the event is sent via trackAnonymous keyed by the session id. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "33151d63f4735f75", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` trackClient.trackAnonymous("s3ss10n", { "name": "product view", "data": {}, "timestamp": 1700000 }) ``` ### Default track A walkerOS event is forwarded to Customer.io as a track call keyed by the user id. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "31249707f7f2ffa0", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` trackClient.track("us3r", { "name": "product view", "data": {}, "timestamp": 1700000 }) ``` ### Destination identify Destination-level identify fires a Customer.io identify call once on the first push, before the track, attaching user attributes. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n", "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "fbef372ecfe82ffc", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` trackClient.identify("us3r", { "email": "user@example.com" }); trackClient.track("us3r", { "name": "page view", "data": {}, "timestamp": 1700000 }) ``` ### Destroy person A user delete event permanently removes the person from Customer.io via trackClient.destroy. Event ``` { "name": "user delete", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a449dc1cd842a239", "trigger": "test", "entity": "user", "action": "delete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "destroy": true } } ``` Out ``` trackClient.destroy("us3r") ``` ### Mapped properties A data mapping transforms the event payload into Customer.io track properties for an order. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "total": 555, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "5dfd9e5f3d1867ab", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "data": { "map": { "order_id": "data.id", "value": "data.total", "currency": "data.currency" } } } ``` Out ``` trackClient.track("us3r", { "name": "purchase", "data": { "order_id": "0rd3r1d", "value": 555, "currency": "EUR" }, "timestamp": 1700000 }) ``` ### Rename event A mapping rule renames the walker event to a Customer.io-specific event name such as purchase. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "4a309708c9106f3b", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase" } ``` Out ``` trackClient.track("us3r", { "name": "purchase", "data": {}, "timestamp": 1700000 }) ``` ### Page view A page view fires trackPageView with the URL and referrer instead of a generic track call. Event ``` { "name": "page view", "data": { "url": "https://example.com/pricing", "referrer": "https://google.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "6bd93952a2dbb8d1", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "page": { "map": { "url": "data.url", "referrer": "data.referrer" } } } } ``` Out ``` trackClient.trackPageView("us3r", "https://example.com/pricing", { "referrer": "https://google.com" }) ``` ### Suppress person A user suppress event stops messaging for the person without deleting their profile data. Event ``` { "name": "user suppress", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ca4f4216ca8ae418", "trigger": "test", "entity": "user", "action": "suppress", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "suppress": true } } ``` Out ``` trackClient.suppress("us3r") ``` ### Unsuppress person A user unsuppress event resumes messaging for a previously suppressed Customer.io profile. Event ``` { "name": "user unsuppress", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "230ec6319b1ea93a", "trigger": "test", "entity": "user", "action": "unsuppress", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "unsuppress": true } } ``` Out ``` trackClient.unsuppress("us3r") ``` ### User login identify A user login triggers only a Customer.io identify call with profile attributes, skipping the track. Event ``` { "name": "user login", "data": { "email": "user@acme.com", "first_name": "Jane", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0edec210bb8c0b57", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "email": "data.email", "first_name": "data.first_name", "plan": "data.plan" } } } } ``` Out ``` trackClient.identify("us3r", { "email": "user@acme.com", "first_name": "Jane", "plan": "premium" }) ``` Identity is resolved automatically from each event: `customerId` defaults to `user.id` and `anonymousId` defaults to `user.session`. Customer.io requires a `customerId` or `anonymousId` per event. When `customerId` is missing, the destination automatically falls back to `trackAnonymous()` so anonymous visitor data is preserved. ## Customer lifecycle[​](#customer-lifecycle "Direct link to Customer lifecycle") Per-rule mapping settings control which lifecycle methods are called in addition to the default `track()`: | Mapping Setting | SDK Call | Use For | | --------------- | ------------------ | -------------------------------- | | `identify` | `identify()` | Login, signup, profile updates | | `page` | `trackPageView()` | Page view events | | `destroy` | `destroy()` | GDPR delete / user removal | | `suppress` | `suppress()` | Stop messaging (keep data) | | `unsuppress` | `unsuppress()` | Resume messaging | | `addDevice` | `addDevice()` | Register push notification token | | `deleteDevice` | `deleteDevice()` | Remove push token | | `merge` | `mergeCustomers()` | Consolidate duplicate profiles | Use `skip: true` on the rule to fire only the lifecycle call without tracking a second `track()` event. ## Transactional messaging[​](#transactional-messaging "Direct link to Transactional messaging") Set `settings.appApiKey` to enable `sendEmail` / `sendPush` via Customer.io's App API. These mappings resolve to the full request body expected by the SDK. --- # Google Data Manager API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/datamanager) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/datamanager)[ ](https://www.npmjs.com/package/@walkeros/server-destination-datamanager) [Package](https://www.npmjs.com/package/@walkeros/server-destination-datamanager) The Google Data Manager API destination package provides server-side integration for sending conversion events and audience data to Google's advertising ecosystem through a single unified API endpoint. Where this fits Google Data Manager API is a **server destination** in the walkerOS flow: Sends conversion events server-side to Google Ads, Display & Video 360, and GA4 through a single unified API. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-datamanager ``` ``` import { startFlow } from '@walkeros/collector'; import { destinationDataManager } from '@walkeros/server-destination-datamanager'; await startFlow({ destinations: { datamanager: { code: destinationDataManager, config: { // Service account credentials credentials: { client_email: process.env.GOOGLE_CLIENT_EMAIL, private_key: process.env.GOOGLE_PRIVATE_KEY, }, settings: { destinations: [ { operatingAccount: { accountId: '123-456-7890', accountType: 'GOOGLE_ADS', }, productDestinationId: 'AW-CONVERSION-123', }, ], }, }, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `credentials` | `credentials` | Service account credentials (client\_email + private\_key). Recommended for serverless environments. (deprecated: use config.credentials) | | | `client_email*` | `string` | Service account email | | | `private_key*` | `string` | Service account private key (PEM format) | | | `keyFilename` | `string` | Path to service account JSON file. For local development or environments with filesystem access. | | | `scopes` | `Array` | OAuth scopes for Data Manager API. Defaults to datamanager scope. | | | `destinations*` | `Array` | Array of destination accounts and conversion actions/user lists (max 10) | | | `eventSource` | `'WEB' \| 'APP' \| 'IN_STORE' \| '` | Event source for all events. Defaults to WEB. Values: WEB, APP, IN\_STORE, PHONE, OTHER | | | `batchSize` | `integer` | Maximum number of events to batch before sending (max 2000, like 100) | | | `batchInterval` | `integer` | Time in milliseconds to wait before auto-flushing batch | | | `validateOnly` | `boolean` | If true, validate request without ingestion (testing mode) | | | `url` | `string` | Override API endpoint for testing | | | `consent` | `consent` | Request-level consent for all events | | | `adUserData` | `'CONSENT_GRANTED' \| 'CONSENT_D` | Consent for data collection and use | | | `adPersonalization` | `'CONSENT_GRANTED' \| 'CONSENT_D` | Consent for ad personalization | | | `testEventCode` | `string` | Test event code for debugging | | | `logLevel` | `'debug' \| 'info' \| 'warn' \| 'e` | Log level for debugging (debug shows all API calls) | | | `userData` | `Record` | Guided helper: User data mapping for all events | | | `userId` | `any` | Guided helper: First-party user ID for all events | | | `clientId` | `any` | Guided helper: GA4 client ID for all events | | | `appInstanceId` | `any` | Guided helper: GA4 app instance ID (Firebase) for all events | | | `sessionAttributes` | `any` | Guided helper: Privacy-safe attribution for all events | | | `consentAdUserData` | `string \| boolean` | Consent mapping: Field name from event.consent (like 'marketing') or static boolean value | | | `consentAdPersonalization` | `string \| boolean` | Consent mapping: Field name from event.consent (like 'targeting') or static boolean value | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Page view A page view is sent to Data Manager as a page\_view event tied to the walker user id. Event ``` { "name": "page view", "data": { "title": "Pricing", "url": "https://example.com/pricing" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "visitor-55" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000902000", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "page_view", "data": { "map": { "transactionId": "id", "eventName": { "value": "page_view" }, "userId": "user.id" } } } ``` Out ``` fetch("https://datamanager.googleapis.com/v1/events:ingest", { "method": "POST", "headers": { "Authorization": "Bearer ya29.c.test_token", "Content-Type": "application/json" }, "body": "{\"events\":[{\"eventTimestamp\":\"2023-11-14T22:28:22.000Z\",\"transactionId\":\"ev-1700000902000\",\"userId\":\"visitor-55\",\"eventName\":\"page_view\",\"eventSource\":\"WEB\"}],\"destinations\":[{\"operatingAccount\":{\"accountId\":\"123-456-7890\",\"accountType\":\"GOOGLE_ADS\"},\"productDestinationId\":\"AW-CONVERSION-123\"}],\"encoding\":\"HEX\"}" }) ``` ### Lead A demo request form submission is sent to Data Manager as a generate\_lead conversion with a hashed email. Event ``` { "name": "form submit", "data": { "type": "demo-request" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "prospect@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000901000", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "generate_lead", "data": { "map": { "transactionId": "id", "eventName": { "value": "generate_lead" }, "conversionValue": { "value": 10 }, "currency": { "value": "USD" }, "email": "user.email" } } } ``` Out ``` fetch("https://datamanager.googleapis.com/v1/events:ingest", { "method": "POST", "headers": { "Authorization": "Bearer ya29.c.test_token", "Content-Type": "application/json" }, "body": "{\"events\":[{\"eventTimestamp\":\"2023-11-14T22:28:21.000Z\",\"transactionId\":\"ev-1700000901000\",\"userData\":{\"userIdentifiers\":[{\"emailAddress\":\"395ec5f334be0ab5b28568a1e7f6ed5ea80e443fb1ce3d803340586a3df46642\"}]},\"conversionValue\":10,\"currency\":\"USD\",\"eventName\":\"generate_lead\",\"eventSource\":\"WEB\"}],\"destinations\":[{\"operatingAccount\":{\"accountId\":\"123-456-7890\",\"accountType\":\"GOOGLE_ADS\"},\"productDestinationId\":\"AW-CONVERSION-123\"}],\"encoding\":\"HEX\"}" }) ``` ### Purchase A completed order is posted to Google Data Manager as a purchase conversion with hashed user identifiers. Event ``` { "name": "order complete", "data": { "id": "ORD-600", "total": 149.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-abc", "email": "buyer@example.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000900000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "purchase", "data": { "map": { "transactionId": "data.id", "conversionValue": "data.total", "currency": { "key": "data.currency", "value": "USD" }, "eventName": { "value": "purchase" }, "userId": "user.id", "email": "user.email" } } } ``` Out ``` fetch("https://datamanager.googleapis.com/v1/events:ingest", { "method": "POST", "headers": { "Authorization": "Bearer ya29.c.test_token", "Content-Type": "application/json" }, "body": "{\"events\":[{\"eventTimestamp\":\"2023-11-14T22:28:20.000Z\",\"transactionId\":\"ORD-600\",\"userId\":\"user-abc\",\"userData\":{\"userIdentifiers\":[{\"emailAddress\":\"6a6c26195c3682faa816966af789717c3bfa834eee6c599d667d2b3429c27cfd\"}]},\"conversionValue\":149.99,\"currency\":\"EUR\",\"eventName\":\"purchase\",\"eventSource\":\"WEB\"}],\"destinations\":[{\"operatingAccount\":{\"accountId\":\"123-456-7890\",\"accountType\":\"GOOGLE_ADS\"},\"productDestinationId\":\"AW-CONVERSION-123\"}],\"encoding\":\"HEX\"}" }) ``` ## Guided mapping helpers[​](#guided-mapping-helpers "Direct link to Guided mapping helpers") Define common fields once in Settings instead of repeating them in every event mapping: ``` await startFlow({ destinations: { datamanager: { code: destinationDataManager, config: { credentials: { /* ... */ }, settings: { destinations: [...], // Guided helpers (apply to all events) userData: { email: 'user.mail', phone: 'data.phone', }, userId: 'user.id', clientId: 'user.device', // GA4 web stream appInstanceId: 'user.appInstanceId', // GA4 app stream (Firebase) sessionAttributes: 'context.sessionAttributes', // Consent mapping consentAdUserData: 'marketing', consentAdPersonalization: 'personalization', }, }, }, }, }); ``` **Consent mapping**: Map your consent field names (string) or use static values (boolean). Event mappings always override Settings helpers. ## Store sales (IN\_STORE)[​](#store-sales-in_store "Direct link to Store sales (IN_STORE)") For physical store conversions, set `eventSource: 'IN_STORE'` on the destination and map a `storeId` in the event mapping. The destination wraps it into the API's required `eventLocation.storeId` shape. ``` mapping: { order: { complete: { name: 'purchase', data: { map: { transactionId: 'data.id', conversionValue: 'data.total', currency: 'data.currency', storeId: 'data.storeId', email: 'user.email', // hashed PII still applies }, }, }, }, } ``` ## Hash encoding[​](#hash-encoding "Direct link to Hash encoding") Every request is sent with `encoding: 'HEX'`. Google requires the field whenever the payload contains hashed `userData`. The value is pinned because all hashing in this destination produces lowercase hex SHA-256 digests; exposing encoding as a setting would risk silently mismatched identifiers. --- # File [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/file) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/file)[ ](https://www.npmjs.com/package/@walkeros/server-destination-file) [Package](https://www.npmjs.com/package/@walkeros/server-destination-file) Beta Local filesystem sink for walkerOS server flows. Appends events to a file as JSON Lines (JSONL), tab-separated values (TSV), or comma-separated values (CSV). Useful for debug logging, audit trails, replay sources, and lightweight local persistence without standing up a database or external service. Where this fits File is a **server destination** in the walkerOS flow: Receives events server-side from the collector, serialises each event, and appends a line to a local file. No network calls, no SDKs, just \`fs.createWriteStream\` with \`flag: 'a'\`. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-file ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationFile } from '@walkeros/server-destination-file'; await startFlow({ destinations: { log: { code: destinationFile, config: { settings: { filename: 'events.jsonl', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "log": { "package": "@walkeros/server-destination-file", "config": { "settings": { "filename": "events.jsonl" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ---- | | `filename*` | `string \| object` | Output filename. Static string or Mapping.Value (e.g. { fn: "$code:..." } for daily rotation, { key: "data.tenant" } for sharding). | | | `format` | `'jsonl' \| 'tsv' \| 'csv'` | Serialisation format. Defaults to jsonl. | | | `fields` | `Array` | Event paths used as columns for tsv/csv formats. Object values are JSON-stringified. Required when format is tsv or csv. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### CSV with object cell An event is written as a CSV row with the data object JSON-stringified and properly quoted for embedded commas and quotes. Event ``` { "name": "page view", "data": { "title": "Hello, \"World\"", "count": 3 }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "7a2409297e527619", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000000000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fs.writeFile("events.csv", "1700000000000,page view,\"{\"\"title\"\":\"\"Hello, \\\"\"World\\\"\"\"\",\"\"count\"\":3}\"\n") ``` ### Daily rotation A mapping function derives a date-stamped filename from the event timestamp to rotate JSONL files daily. Event ``` { "name": "order complete", "data": { "id": "ORD-1" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "52b85f3da912e709", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1776256496000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fs.writeFile("events-2026-04-15.jsonl", "jsonl:event") ``` ### JSONL append An event is appended as a single JSON line to a static filename using default JSONL formatting. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "3a5bb7d0285865dd", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000000000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fs.writeFile("events.jsonl", "jsonl:event") ``` ### Tenant sharding The filename is resolved from an event field so events are partitioned into per-tenant JSONL files. Event ``` { "name": "custom event", "data": { "tenant": "acme" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "79c3b631ee0d0f40", "trigger": "test", "entity": "custom", "action": "event", "timestamp": 1700000000000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fs.writeFile("acme", "jsonl:event") ``` ### TSV log An event is written as a tab-separated line selecting specific fields for a compact access-log style file. Event ``` { "name": "page view", "data": { "title": "Docs" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "session": "sess-1" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0132b1fad3bdea76", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000000000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/docs", "referrer": "https://example.com/" } } ``` Out ``` fs.writeFile("storage/mblog.txt", "1700000000000\tsess-1\tpage view\thttps://example.com/docs\tDocs\thttps://example.com/\n") ``` The destination opens one long-lived `WriteStream` per resolved filename during `init()` (for static filenames) or on first matching event (for dynamic filenames). On flow hot-swap or server shutdown, `destroy()` closes all cached streams. ## Filename templating[​](#filename-templating "Direct link to Filename templating") `filename` accepts either a static string or a standard walkerOS `Mapping.Value`. Common patterns: ### Tenant sharding[​](#tenant-sharding "Direct link to Tenant sharding") ``` { "filename": { "key": "data.tenant" } } ``` Each event lands in a file named after the tenant value (e.g. `acme`, `venti`). Pair with a static suffix via `fn:` if you need a `.jsonl` extension. ### Daily rotation[​](#daily-rotation "Direct link to Daily rotation") ``` { "filename": { "fn": "$code:`events-${new Date(value.timestamp).toISOString().slice(0,10)}.jsonl`" } } ``` Inside the `$code:` function, `value` is the event being processed. One file per UTC day, created automatically. No `logrotate`, no cron. ## Formats[​](#formats "Direct link to Formats") ### JSONL (default)[​](#jsonl-default "Direct link to JSONL (default)") One JSON object per line. The entire event is serialised with `JSON.stringify`. Easy to ingest with `jq`, DuckDB (`read_json_auto`), ClickHouse `JSONEachRow`, BigQuery external tables, and Athena. ### TSV / CSV[​](#tsv--csv "Direct link to TSV / CSV") Specify `fields: string[]` listing the event paths to extract as columns. Object values are JSON-stringified into a single cell. CSV output follows RFC 4180 quoting. ``` { "filename": "events.csv", "format": "csv", "fields": ["timestamp", "name", "data"] } ``` ## Limits[​](#limits "Direct link to Limits") * One file handle is opened per resolved filename and kept open until `destroy()`. Sharding by high-cardinality keys (e.g. `user.session`) can exhaust the OS file descriptor table. Be deliberate about cardinality. * External rotation (e.g. `logrotate`) leaves the cached handle pointing at the rotated inode. Use the date-token pattern above instead. * No batching. Each event is written individually; Node's stream layer buffers under the hood. * Write errors log a warning and drop the event. They never fail the flow. --- # Google BigQuery [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/gcp) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/gcp)[ ](https://www.npmjs.com/package/@walkeros/server-destination-gcp) [Package](https://www.npmjs.com/package/@walkeros/server-destination-gcp) Server-side event streaming to [Google BigQuery](https://cloud.google.com/bigquery) via the Storage Write API for low-latency analytics, machine learning workloads, and data warehousing. The `@walkeros/server-destination-gcp` package also ships [`destinationPubSub`](/docs/destinations/server/pubsub.md) for publishing events to Pub/Sub topics; this page covers BigQuery only. Where this fits GCP BigQuery is a **server destination** in the walkerOS flow: Streams events to Google BigQuery for data warehousing, analytics dashboards, and machine learning workloads. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-gcp ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationBigQuery } from '@walkeros/server-destination-gcp'; await startFlow({ destinations: { bigquery: { code: destinationBigQuery, config: { settings: { projectId: 'YOUR_PROJECT_ID', datasetId: 'YOUR_DATASET_ID', tableId: 'YOUR_TABLE_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "bigquery": { "package": "@walkeros/server-destination-gcp", "import": "destinationBigQuery", "config": { "settings": { "projectId": "YOUR_PROJECT_ID", "datasetId": "YOUR_DATASET_ID", "tableId": "YOUR_TABLE_ID" } } } } ``` [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------ | -------- | ------------------------------------------------ | ---- | | `client` | `any` | Google Cloud BigQuery client instance | | | `projectId*` | `string` | Google Cloud Project ID | | | `datasetId` | `string` | BigQuery dataset ID where events will be stored | | | `tableId` | `string` | BigQuery table ID for event storage | | | `location` | `string` | Geographic location for the BigQuery dataset | | | `bigquery` | `any` | Additional BigQuery client configuration options | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Page view A page view is appended as one row through the BigQuery Storage Write API JSONWriter. Nested objects/arrays in data, source, etc. are JSON-stringified by eventToRow. Event ``` { "name": "page view", "data": { "title": "Documentation", "url": "https://example.com/docs" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "bb3abfa263d51eeb", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700001100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` appendRows([ { "name": "page view", "data": "{\"title\":\"Documentation\",\"url\":\"https://example.com/docs\"}", "context": "{\"dev\":[\"test\",1]}", "globals": "{\"pagegroup\":\"docs\"}", "custom": "{\"completely\":\"random\"}", "user": "{\"id\":\"us3r\",\"device\":\"c00k13\",\"session\":\"s3ss10n\"}", "nested": "[{\"entity\":\"child\",\"data\":{\"is\":\"subordinated\"}}]", "consent": "{\"functional\":true}", "id": "bb3abfa263d51eeb", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700001100000, "timing": 3.14, "source": "{\"count\":1,\"trace\":\"0a1b2c3d4e5f60718293a4b5c6d7e8f9\",\"type\":\"express\",\"platform\":\"server\"}" } ]) ``` ### Purchase An order event is appended as a single row through JSONWriter.appendRows. The entire nested data object (including arrays like items) is JSON-stringified into the data column via eventToRow(). Event ``` { "name": "order complete", "data": { "id": "ORD-500", "total": 199.99, "items": [ { "sku": "SKU-1", "qty": 2 } ] }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1a51af55c31952fd", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700001101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` appendRows([ { "name": "order complete", "data": "{\"id\":\"ORD-500\",\"total\":199.99,\"items\":[{\"sku\":\"SKU-1\",\"qty\":2}]}", "context": "{\"shopping\":[\"complete\",0]}", "globals": "{\"pagegroup\":\"shop\"}", "custom": "{\"completely\":\"random\"}", "user": "{\"id\":\"us3r\",\"device\":\"c00k13\",\"session\":\"s3ss10n\"}", "nested": "[{\"entity\":\"product\",\"data\":{\"id\":\"ers\",\"name\":\"Everyday Ruck Snack\",\"color\":\"black\",\"size\":\"l\",\"price\":420},\"context\":{\"shopping\":[\"complete\",0]},\"nested\":[]},{\"entity\":\"product\",\"data\":{\"id\":\"cc\",\"name\":\"Cool Cap\",\"size\":\"one size\",\"price\":42},\"context\":{\"shopping\":[\"complete\",0]},\"nested\":[]},{\"entity\":\"gift\",\"data\":{\"name\":\"Surprise\"},\"context\":{\"shopping\":[\"complete\",0]},\"nested\":[]}]", "consent": "{\"functional\":true}", "id": "1a51af55c31952fd", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700001101000, "timing": 3.14, "source": "{\"count\":1,\"trace\":\"0a1b2c3d4e5f60718293a4b5c6d7e8f9\",\"type\":\"express\",\"platform\":\"server\"}" } ]) ``` ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [Google Cloud account](https://cloud.google.com/) with billing enabled * [gcloud CLI](https://cloud.google.com/sdk/docs/install) installed and authenticated (includes `bq` command) ## Setup lifecycle[​](#setup-lifecycle "Direct link to Setup lifecycle") Provision the dataset and table once per environment with the CLI: ``` walkeros setup destination.bigquery ``` Output: a narrated `setup: ok destination.bigquery` line. Add `--json` to also emit a structured envelope reporting `{ datasetCreated, tableCreated }` for `jq` piping. The command is idempotent, safe to re-run. `config.setup` controls provisioning: * omitted or `false`: narrated skip, no provisioning. Operator runs setup explicitly to provision. * `true`: provision with the defaults below. * object matching the `Setup` interface: provision with the declared overrides. See the `Setup` interface in the package for the full shape. ### Defaults[​](#defaults "Direct link to Defaults") | Field | Value | | --------------------- | ------------------------------------------ | | `datasetId` | `walkerOS` (note capital O, S) | | `tableId` | `events` | | `location` | `EU` | | `storageBillingModel` | `PHYSICAL` (cheaper for compressible JSON) | | Partitioning | Day partitioning on `timestamp` | | Clustering | `(name, entity, action)` | Cost optimization Physical storage billing charges based on compressed size. Day partitioning and the `(name, entity, action)` clustering reduce scan costs for typical analytics queries. Always include a `timestamp` filter. ### Drift handling[​](#drift-handling "Direct link to Drift handling") If the existing table's partitioning, clustering, or schema differs from the declared configuration, setup logs `WARN setup.drift {...}` and continues. There is no auto-mutation. Migrations are an operator decision. ## GCP setup[​](#gcp-setup "Direct link to GCP setup") ### Enable BigQuery API[​](#enable-bigquery-api "Direct link to Enable BigQuery API") ``` gcloud services enable bigquery.googleapis.com ``` ### Create service accounts[​](#create-service-accounts "Direct link to Create service accounts") The provisioning step (setup) and the runtime push path need different permissions. We recommend separating them. **Operator (setup) service account**, used by `walkeros setup`: * `bigquery.datasets.create` * `bigquery.tables.create` * `bigquery.datasets.get` (for drift detection) * `bigquery.tables.get` **Runtime service account**, used by the running flow: * `bigquery.tables.updateData` (Storage Write API append) ``` # Runtime service account gcloud iam service-accounts create walkeros-flow \ --display-name="walkerOS Flow Runtime" # Grant Storage Write API append access gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \ --member="serviceAccount:walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/bigquery.dataEditor" ``` ### Authentication[​](#authentication "Direct link to Authentication") * Service Account Key * Workload Identity For environments where you need explicit credentials (Docker containers, external platforms): ``` gcloud iam service-accounts keys create ./sa-bigquery.json \ --iam-account=walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com ``` Set the environment variable to use the key: ``` export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-bigquery.json ``` caution Keep key files secure. Never commit them to version control or include in public Docker images. For GCP-native platforms (Cloud Run, GKE, Compute Engine), attach the service account directly to your workload. No key file needed. **Cloud Run example:** ``` gcloud run deploy walkeros-flow \ --service-account=walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com ``` See [Workload Identity documentation](https://cloud.google.com/iam/docs/workload-identity-federation) for other platforms. Inline credentials Instead of relying on `GOOGLE_APPLICATION_CREDENTIALS`, set service account credentials on `config.credentials` (a JSON string or parsed object, `$env` -resolvable). These merge into the BigQuery client and apply to both the control plane (setup, metadata) and the data plane (Storage Write API ingestion). For lower-level control you can still pass auth options through the raw `settings.bigquery` passthrough (for example `keyFilename` or `credentials`). A pre-built `settings.client` authenticates the control plane only; supply `config.credentials` or `settings.bigquery` for the data plane to use non-ADC credentials. ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Description | Default | | -------------------------------- | --------------------------- | ----------------------------------------- | | `GCP_PROJECT_ID` | Your GCP project ID | Required | | `BQ_DATASET` | BigQuery dataset name | `walkerOS` | | `BQ_TABLE` | BigQuery table name | `events` | | `BQ_LOCATION` | BigQuery dataset location | `EU` | | `GOOGLE_APPLICATION_CREDENTIALS` | Path to service account key | Required (unless using Workload Identity) | ## Storage Write API (data plane)[​](#storage-write-api-data-plane "Direct link to Storage Write API (data plane)") The destination uses BigQuery's [Storage Write API](https://cloud.google.com/bigquery/docs/write-api) for data ingestion. This replaces the legacy `tabledata.insertAll` path. * **Cost**: $25/TB after the 2 TiB/month free tier (vs \~$50/TB for the legacy path). Most low-volume deployments fit entirely in the free tier. * **Batching**: `pushBatch` is implemented. Set `config.batch` on the destination (a bare number is the debounce `wait` in ms) to flush all events in a window as a single `appendRows` call. No `'* *'` wildcard mapping rule is needed. The batch path awaits the write and surfaces row errors: a failed append fails the batch, which routes the events to the dead-letter buffer and the `failed` count rather than being logged and ignored. EXPERIMENTAL SDK The upstream `@google-cloud/bigquery-storage` package self-marks as `EXPERIMENTAL` (subject to change). Pinned at `^5.1.0`. ## Default table schema[​](#default-table-schema "Direct link to Default table schema") The default 15-column schema follows the [walkerOS Event v4](/docs/getting-started/event-model.md) canonical order. Object fields use the native `JSON` BigQuery type. Only `name` is `REQUIRED`; all other columns are `NULLABLE` for resilience against partial events. | Column | Type | Mode | | ----------- | --------- | -------- | | `name` | STRING | REQUIRED | | `data` | JSON | NULLABLE | | `context` | JSON | NULLABLE | | `globals` | JSON | NULLABLE | | `custom` | JSON | NULLABLE | | `user` | JSON | NULLABLE | | `nested` | JSON | NULLABLE | | `consent` | JSON | NULLABLE | | `id` | STRING | NULLABLE | | `trigger` | STRING | NULLABLE | | `entity` | STRING | NULLABLE | | `action` | STRING | NULLABLE | | `timestamp` | TIMESTAMP | NULLABLE | | `timing` | FLOAT64 | NULLABLE | | `source` | JSON | NULLABLE | There is no `createdAt` column. Use `timestamp` (event time) for partition filters. Query optimization Partitioning by day on `timestamp` and clustering on `(name, entity, action)` reduces scan costs for typical analytics queries. Always include a `timestamp` filter. ## Custom schema mapping[​](#custom-schema-mapping "Direct link to Custom schema mapping") You can send a custom schema by using the `data` configuration to map specific fields. This is useful when you only need a subset of the event data. ### Example: simple schema[​](#example-simple-schema "Direct link to Example: simple schema") This example sends only `name`, `id`, `data`, and `timestamp`: * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationBigQuery } from '@walkeros/server-destination-gcp'; await startFlow({ destinations: { bigquery: { code: destinationBigQuery, config: { settings: { projectId: 'YOUR_PROJECT_ID', datasetId: 'YOUR_DATASET_ID', tableId: 'events_simple', }, data: { map: { name: 'name', id: 'id', data: 'data', timestamp: 'timestamp', }, }, }, }, }, }); ``` ``` { "destinations": { "bigquery": { "package": "@walkeros/server-destination-gcp", "import": "destinationBigQuery", "config": { "settings": { "projectId": "YOUR_PROJECT_ID", "datasetId": "YOUR_DATASET_ID", "tableId": "events_simple" }, "data": { "map": { "name": "name", "id": "id", "data": "data", "timestamp": "timestamp" } } } } } } ``` With the corresponding simpler table: ``` CREATE TABLE IF NOT EXISTS `YOUR_PROJECT.walkeros.events_simple` ( name STRING, id STRING, data STRING, timestamp INT64 ); ``` ## Cleanup[​](#cleanup "Direct link to Cleanup") To remove BigQuery resources: * Delete the BigQuery dataset * Remove service account IAM bindings from the dataset * Delete the service account * Remove any downloaded key files ## Pub/Sub[​](#pubsub "Direct link to Pub/Sub") The same `@walkeros/server-destination-gcp` package also exports `destinationPubSub` for publishing events to a Pub/Sub topic. See the [Pub/Sub destination page](/docs/destinations/server/pubsub.md) for full settings, mapping, ordering, attributes, setup, and authentication reference. --- # HubSpot [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/hubspot) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/hubspot)[ ](https://www.npmjs.com/package/@walkeros/server-destination-hubspot) [Package](https://www.npmjs.com/package/@walkeros/server-destination-hubspot) Beta Server-side event delivery to [HubSpot](https://www.hubspot.com/) via the official [`@hubspot/api-client`](https://www.npmjs.com/package/@hubspot/api-client) SDK. Sends custom behavioral events, upserts contacts via the CRM API, and supports optional batch mode with graceful shutdown. Where this fits HubSpot is a **server destination** in the walkerOS flow: Receives events server-side from the collector, resolves contact identity (email or objectId), optionally upserts contact properties, then sends the custom event to HubSpot. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-hubspot ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationHubspot } from '@walkeros/server-destination-hubspot'; await startFlow({ destinations: { hubspot: { code: destinationHubspot, config: { settings: { accessToken: 'YOUR_HUBSPOT_ACCESS_TOKEN', eventNamePrefix: 'pe12345678_', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "hubspot": { "package": "@walkeros/server-destination-hubspot", "config": { "settings": { "accessToken": "YOUR_HUBSPOT_ACCESS_TOKEN", "eventNamePrefix": "pe12345678_" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `accessToken*` | `string` | HubSpot private app access token. Create one in HubSpot Settings > Integrations > Private Apps. Requires analytics.behavioral\_events.send scope. | | | `eventNamePrefix*` | `string` | Fully qualified event name prefix: pe{HubID}\_ (e.g. pe12345678\_). Find it in HubSpot under Data Management > Custom Events. | | | `email` | `string` | walkerOS mapping value path to resolve contact email from events (like user.email). Required for contact association. | | | `objectId` | `string` | walkerOS mapping value path to resolve HubSpot CRM objectId from events. Alternative to email for contact association. | | | `identify` | `any` | Destination-level contact upsert mapping. Resolves to { email, properties }. Fires contact update on first push and re-fires when values change. | | | `defaultProperties` | `Record` | Static event properties added to every event occurrence. Useful for hs\_touchpoint\_source, hs\_page\_content\_type, etc. | | | `batch` | `boolean` | Use batch API for events (accumulate and flush). Default: false. | | | `batchSize` | `integer` | Batch size before auto-flush. Only used when batch: true. Default: 50. Max: 500. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `eventName` | `string` | Override eventName for this rule. Without the prefix -- just the event name part (e.g. purchase\_completed). The eventNamePrefix is prepended automatically. | | | `identify` | `any` | Per-event contact upsert. Resolves to { email, properties }. Overrides destination-level identify. Use with silent: true on login/identify events. | | | `properties` | `any` | Additional event properties mapping. Resolved values are merged with defaultProperties and serialized to strings. | | ## Examples ### Default event A walker event is sent to HubSpot as a custom behavioral event keyed by the user email. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [], "consent": { "functional": true }, "id": "82a1ef3350641e5d", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` events.send.basicApi.send({ "eventName": "pe12345678_product_view", "email": "user@example.com", "occurredAt": {}, "properties": {} }) ``` ### Default properties Destination-level default properties are merged into every HubSpot event payload, such as traffic source metadata. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "4d6fd6fa5518e8c0", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` events.send.basicApi.send({ "eventName": "pe12345678_page_view", "email": "user@example.com", "occurredAt": {}, "properties": { "hs_touchpoint_source": "walkerOS", "hs_page_content_type": "STANDARD_PAGE" } }) ``` ### Destination identify Destination-level identify upserts the HubSpot contact with mapped properties before sending the behavioral event. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com", "firstName": "Jane", "lastName": "Doe" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ccb0890543f797f3", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` crm.contacts.basicApi.update("user@example.com", { "properties": { "firstname": "Jane", "lastname": "Doe" } }, "email"); events.send.basicApi.send({ "eventName": "pe12345678_page_view", "email": "user@example.com", "occurredAt": {}, "properties": {} }) ``` ### Custom event name A mapping supplies a custom HubSpot event name and maps order data into properties for the behavioral event. Event ``` { "name": "order complete", "data": { "total": 99.5, "currency": "EUR", "id": "ord-123" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "53798a1eba1f2cea", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "order complete", "settings": { "eventName": "purchase_completed", "properties": { "map": { "revenue": "data.total", "currency": "data.currency", "order_id": "data.id" } } } } ``` Out ``` events.send.basicApi.send({ "eventName": "pe12345678_purchase_completed", "email": "user@example.com", "occurredAt": {}, "properties": { "revenue": "99.5", "currency": "EUR", "order_id": "ord-123" } }) ``` ### Object id association The HubSpot event is associated via objectId instead of email, resolved from the walker user id. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "hs-contact-789" }, "nested": [], "consent": { "functional": true }, "id": "f80545512b019f0c", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` events.send.basicApi.send({ "eventName": "pe12345678_product_view", "objectId": "hs-contact-789", "occurredAt": {}, "properties": {} }) ``` ### User login identify A user login only upserts the HubSpot contact with profile and lifecycle properties, skipping the event send. Event ``` { "name": "user login", "data": { "email": "login@acme.com", "first_name": "Jane", "last_name": "Doe", "lifecycle": "lead" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "cf0530ad340c8e17", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "email": "data.email", "properties": { "map": { "firstname": "data.first_name", "lastname": "data.last_name", "lifecyclestage": "data.lifecycle" } } } } } } ``` Out ``` crm.contacts.basicApi.update("login@acme.com", { "properties": { "firstname": "Jane", "lastname": "Doe", "lifecyclestage": "lead" } }, "email") ``` Every event must be associated with a contact via `email` or `objectId`. The `email` path defaults to `user.email`. Events where neither value can be resolved are skipped with a warning. ## Contact upsert[​](#contact-upsert "Direct link to Contact upsert") Use the `identify` setting (destination-level or per-rule) to upsert HubSpot contact properties alongside event tracking. The resolved mapping must produce `{ email, properties }`. State-based dedup prevents redundant API calls when identity has not changed. --- # Apache Kafka [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/kafka) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/kafka)[ ](https://www.npmjs.com/package/@walkeros/server-destination-kafka) [Package](https://www.npmjs.com/package/@walkeros/server-destination-kafka) Beta Server-side event streaming to [Apache Kafka](https://kafka.apache.org/) via the [`kafkajs`](https://kafka.js.org/) client. Each event is serialized as JSON, keyed for partition-friendly ordering, and produced to a configurable topic. Supports SASL/SSL authentication (Confluent Cloud, AWS MSK, SCRAM), configurable compression (gzip, snappy, lz4, zstd), per-rule topic and key overrides, and graceful shutdown via `destroy()`. Where this fits Kafka is a **server destination** in the walkerOS flow: Receives events server-side from the collector, serializes them as JSON with a partition-friendly message key, and produces them to a Kafka topic for downstream stream processing (Flink, Spark, ksqlDB, Kafka Connect, consumers). ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-kafka ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationKafka } from '@walkeros/server-destination-kafka'; await startFlow({ destinations: { kafka: { code: destinationKafka, config: { settings: { kafka: { brokers: ['kafka1:9092', 'kafka2:9092'], topic: 'walkeros-events', }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "kafka": { "package": "@walkeros/server-destination-kafka", "config": { "settings": { "kafka": { "brokers": ["kafka1:9092", "kafka2:9092"], "topic": "walkeros-events" } } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------------ | ----------------------------------- | --------------------------------------------------------------------------------------------------- | ---- | | `kafka*` | `kafka` | Kafka connection and producer settings. | | | `brokers*` | `Array` | Kafka broker addresses (host:port). At least one required. | | | `clientId` | `string` | Kafka client ID. Default: walkeros. | | | `ssl` | `boolean \| object` | TLS configuration. Set true for default TLS, or provide a tls.ConnectionOptions object for mTLS. | | | `sasl` | `sasl` | SASL authentication config. Required for Confluent Cloud, AWS MSK with IAM, etc. | | | `mechanism*` | `'plain' \| 'scram-sha-256' \| 's` | SASL authentication mechanism. | | | `username` | `string` | Username for plain/scram mechanisms. | | | `password` | `string` | Password for plain/scram mechanisms. | | | `accessKeyId` | `string` | AWS access key ID for IAM auth (mechanism: aws). | | | `secretAccessKey` | `string` | AWS secret access key for IAM auth (mechanism: aws). | | | `sessionToken` | `string` | AWS session token for temporary credentials (mechanism: aws). | | | `authorizationIdentity` | `string` | AWS authorization identity (mechanism: aws). | | | `connectionTimeout` | `integer` | Connection timeout in ms. Default: 1000. | | | `requestTimeout` | `integer` | Request timeout in ms. Default: 30000. | | | `topic*` | `string` | Target Kafka topic name. | | | `acks` | `integer` | Acknowledgement level. -1 = all replicas, 0 = fire-and-forget, 1 = leader only. Default: -1. | | | `timeout` | `integer` | Broker response timeout in ms. Default: 30000. | | | `compression` | `'none' \| 'gzip' \| 'snappy' \| '` | Message compression codec. Default: gzip. Snappy/LZ4/ZSTD require additional npm packages. | | | `idempotent` | `boolean` | Enable idempotent producer for exactly-once delivery. Default: false. | | | `allowAutoTopicCreation` | `boolean` | Allow auto-creation of topics on the broker. Default: false. | | | `key` | `string` | Mapping value path for message key derivation (e.g. user.id, data.userId). Default: entity\_action. | | | `headers` | `Record` | Static headers added to every message. | | | `retry` | `retry` | Retry configuration for transient failures. | | | `maxRetryTime` | `integer` | Max total retry wait in ms. Default: 30000. | | | `initialRetryTime` | `integer` | First retry delay in ms. Default: 300. | | | `retries` | `integer` | Max retry count. Default: 5. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | --------------------------------------------------------------------------------------------------------- | ---- | | `key` | `string` | Override message key mapping path for this rule (e.g. data.id). Takes precedence over settings.kafka.key. | | | `topic` | `string` | Override Kafka topic for this rule. Takes precedence over settings.kafka.topic. | | ## Examples ### Default event An event is produced to the configured Kafka topic with the full JSON body and entity\_action as the message key. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "970120f666d4ae20", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` producer.send({ "topic": "walkeros-events", "messages": [ { "key": "page_view", "value": "json:event", "headers": { "content-type": "application/json" }, "timestamp": "1700000100" } ], "acks": -1, "compression": 1 }) ``` ### Key from user id A settings.kafka.key path resolves the message key from the event, here using user.id for per-user partitioning. Event ``` { "name": "user signup", "data": { "plan": "pro" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "usr-789" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "8ad96a8b8a1ba045", "trigger": "test", "entity": "user", "action": "signup", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` producer.send({ "topic": "walkeros-events", "messages": [ { "key": "usr-789", "value": "json:event", "headers": { "content-type": "application/json" }, "timestamp": "1700000103" } ], "acks": -1, "compression": 1 }) ``` ### Mapped payload A data mapping transforms the event payload before producing it as the Kafka message value. Event ``` { "name": "order complete", "data": { "id": "ORD-400", "total": 99.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "f8df7d62f630f038", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "data": { "map": { "order_id": "data.id", "revenue": "data.total", "currency": "data.currency" } } } ``` Out ``` producer.send({ "topic": "walkeros-events", "messages": [ { "key": "purchase", "value": "json:data", "headers": { "content-type": "application/json" }, "timestamp": "1700000102" } ], "acks": -1, "compression": 1 }) ``` ### Renamed event A mapping renames the event which also changes the default Kafka message key used for partitioning. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "d1209e75f754a806", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase" } ``` Out ``` producer.send({ "topic": "walkeros-events", "messages": [ { "key": "purchase", "value": "json:event", "headers": { "content-type": "application/json" }, "timestamp": "1700000101" } ], "acks": -1, "compression": 1 }) ``` ### Topic override A mapping rule overrides the destination topic so specific events are routed to a dedicated stream. Event ``` { "name": "order complete", "data": { "id": "ORD-500", "total": 42 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "75527a83b0314a8f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "topic": "orders-stream" } } ``` Out ``` producer.send({ "topic": "orders-stream", "messages": [ { "key": "order_complete", "value": "json:event", "headers": { "content-type": "application/json" }, "timestamp": "1700000104" } ], "acks": -1, "compression": 1 }) ``` The destination creates a single long-lived `kafkajs` producer during `init()` and calls `producer.connect()` before accepting events. On flow hot-swap or server shutdown, `destroy()` calls `producer.disconnect()` to flush in-flight messages and close TCP connections. ## Message format[​](#message-format "Direct link to Message format") Events are serialized as JSON and produced with the following structure: * **topic**, from `settings.kafka.topic` (or `mapping.settings.topic` override) * **key**, resolved from `settings.kafka.key` (or `mapping.settings.key` override) mapping path; defaults to the event name with spaces replaced by `_` (e.g. `page_view`, `order_complete`) for partition-based ordering * **value**, mapped payload (when `data.map` is configured) or the full walkerOS event, `JSON.stringify()`-ed * **headers**, `content-type: application/json` plus any static `settings.kafka.headers` * **timestamp**, event timestamp as string (ms since epoch) Use `mapping.settings.topic` to route specific events to dedicated topics (e.g. orders to `orders-stream`, identities to `identity-stream`). Use `mapping.settings.key` to set a key path per rule (e.g. `data.order_id` for order events). ## Setup[​](#setup "Direct link to Setup") No safe defaults Kafka topic creation requires explicit `numPartitions` and `replicationFactor`. There is no universally correct default for either: `replicationFactor` must be less than or equal to broker count, and `numPartitions` is a function of expected throughput and consumer parallelism. The boolean form `setup: true` is rejected with an error listing the required fields. Only the object form is valid. Provision a topic once per environment with the CLI: ``` walkeros setup destination.kafka ``` Output: `setup: ok destination.kafka` plus a JSON line reporting `{ topicCreated, schemaRegistered }`. The command is idempotent, safe to re-run. Drift on `numPartitions`, `replicationFactor`, or `configEntries` is logged as `WARN setup.drift {...}` and never auto-mutates the broker. `config.setup`: * `false` (default): no provisioning. Operator must run setup explicitly. * `true`: rejected at runtime with an actionable error. There are no safe defaults for partition count or replication factor. * `{ numPartitions, replicationFactor, ... }`: object form is the only valid form. See the `Setup` interface in the package for full options. ### Required fields[​](#required-fields "Direct link to Required fields") | Field | Type | Notes | | ------------------- | ------ | -------------------------------------------------- | | `numPartitions` | number | Required at runtime. No safe default. | | `replicationFactor` | number | Required at runtime. Must be `<= broker count`. | | `topic` | string | Falls back to `settings.kafka.topic` when omitted. | ### Optional fields[​](#optional-fields "Direct link to Optional fields") | Field | Type | Notes | | ---------------- | ------- | ----------------------------------------------------------- | | `configEntries` | object | Topic-level config, e.g. `{ "retention.ms": "604800000" }`. | | `schemaRegistry` | object | Confluent Schema Registry binding (see below). | | `validateOnly` | boolean | kafkajs broker-side dry-run. No topic is created. | ### Example[​](#example "Direct link to Example") ``` { "destinations": { "kafka": { "package": "@walkeros/server-destination-kafka", "config": { "settings": { "kafka": { "brokers": ["broker:9092"], "topic": "walkeros-events" } }, "setup": { "numPartitions": 6, "replicationFactor": 3, "configEntries": { "retention.ms": "604800000" } } } } } } ``` ### Schema Registry (optional)[​](#schema-registry-optional "Direct link to Schema Registry (optional)") ``` { "setup": { "numPartitions": 6, "replicationFactor": 3, "schemaRegistry": { "url": "https://schema-registry.example.com", "subject": "walkeros-events-value", "schemaType": "JSON", "schema": "{ \"type\": \"object\" }", "compatibility": "BACKWARD" } } } ``` The schema is registered via the Confluent Schema Registry REST API. The optional `compatibility` level is set on the subject after registration. ### Runtime error when the topic is missing[​](#runtime-error-when-the-topic-is-missing "Direct link to Runtime error when the topic is missing") When `setup` was not run and the topic does not exist on the cluster, `push()` catches the kafkajs `UNKNOWN_TOPIC_OR_PARTITION` error and logs an actionable message pointing the operator at `walkeros setup destination.`. Run setup with explicit `numPartitions` and `replicationFactor` to provision the topic. ## Authentication[​](#authentication "Direct link to Authentication") ### Confluent Cloud (SASL/PLAIN)[​](#confluent-cloud-saslplain "Direct link to Confluent Cloud (SASL/PLAIN)") ``` { "kafka": { "brokers": ["pkc-xxxxx.us-east-1.aws.confluent.cloud:9092"], "topic": "walkeros-events", "ssl": true, "sasl": { "mechanism": "plain", "username": "$env.CONFLUENT_API_KEY", "password": "$env.CONFLUENT_API_SECRET" } } } ``` ### AWS MSK (IAM)[​](#aws-msk-iam "Direct link to AWS MSK (IAM)") ``` { "kafka": { "brokers": ["broker.msk.us-east-1.amazonaws.com:9098"], "topic": "walkeros-events", "ssl": true, "sasl": { "mechanism": "aws", "accessKeyId": "$env.AWS_ACCESS_KEY", "secretAccessKey": "$env.AWS_SECRET_KEY" } } } ``` --- # Klaviyo [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/klaviyo) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/klaviyo)[ ](https://www.npmjs.com/package/@walkeros/server-destination-klaviyo) [Package](https://www.npmjs.com/package/@walkeros/server-destination-klaviyo) Beta Server-side event delivery to [Klaviyo](https://www.klaviyo.com/) via the official [`klaviyo-api`](https://www.npmjs.com/package/klaviyo-api) SDK. Tracks events with `EventsApi.createEvent()` and manages profiles with `ProfilesApi.createOrUpdateProfile()`, including revenue tracking, ecommerce metric name mapping, and identify state diffing. Where this fits Klaviyo is a **server destination** in the walkerOS flow: Receives events server-side from the collector, resolves a profile identifier (email, externalId, or phoneNumber), optionally upserts the profile, then forwards the event to Klaviyo for flows, segmentation, and campaigns. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-klaviyo ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationKlaviyo } from '@walkeros/server-destination-klaviyo'; await startFlow({ destinations: { klaviyo: { code: destinationKlaviyo, config: { settings: { apiKey: 'YOUR_KLAVIYO_PRIVATE_API_KEY', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "klaviyo": { "package": "@walkeros/server-destination-klaviyo", "config": { "settings": { "apiKey": "YOUR_KLAVIYO_PRIVATE_API_KEY" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `apiKey*` | `string` | Klaviyo private API key. Starts with pk\_. Find it under Settings > API Keys in your Klaviyo account. | | | `email` | `string` | walkerOS mapping value path to resolve email from each event (like user.email). | | | `phoneNumber` | `string` | walkerOS mapping value path to resolve phone number in E.164 format from each event. | | | `externalId` | `string` | walkerOS mapping value path to resolve external ID from each event (like user.id). | | | `identify` | `any` | Destination-level identity mapping. Resolves to profile attributes { firstName?, lastName?, organization?, properties? }. Fires createOrUpdateProfile() on first push and re-fires when values change. | | | `currency` | `string` | Default ISO 4217 currency code for revenue events (like USD, EUR). Sets valueCurrency on Klaviyo events. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `identify` | `any` | Per-event identify mapping. Resolves to profile attributes for createOrUpdateProfile(). Use with rule-level silent: true on login/signup events. | | | `value` | `any` | Revenue value mapping. Resolves to a numeric value for Klaviyo revenue tracking. Sets the value property and valueCurrency on the event. | | ## Examples ### Default event An event is sent to Klaviyo as a metric with an inline profile resolved from the user email and id. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "email": "user@example.com" }, "nested": [], "consent": { "functional": true }, "id": "301460d72f67bf20", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` eventsApi.createEvent({ "data": { "type": "event", "attributes": { "profile": { "data": { "type": "profile", "attributes": { "email": "user@example.com", "externalId": "us3r" } } }, "metric": { "data": { "type": "metric", "attributes": { "name": "product view" } } }, "properties": {}, "time": "1970-01-20T16:13:20.100Z" } } }) ``` ### Destination identify Destination-level identify upserts the Klaviyo profile with a first name before each event is sent. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "email": "user@example.com", "firstName": "Jane" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "e9d55e4648c1c4f0", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` profilesApi.createOrUpdateProfile({ "data": { "type": "profile", "attributes": { "email": "user@example.com", "externalId": "us3r", "firstName": "Jane" } } }); eventsApi.createEvent({ "data": { "type": "event", "attributes": { "profile": { "data": { "type": "profile", "attributes": { "email": "user@example.com", "externalId": "us3r" } } }, "metric": { "data": { "type": "metric", "attributes": { "name": "page view" } } }, "properties": {}, "time": "1970-01-20T16:13:20.104Z" } } }) ``` ### Email only A newsletter signup uses only the email address as the Klaviyo profile identifier, with no external id. Event ``` { "name": "newsletter signup", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "subscriber@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "fcdc2420bf88b71f", "trigger": "test", "entity": "newsletter", "action": "signup", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` eventsApi.createEvent({ "data": { "type": "event", "attributes": { "profile": { "data": { "type": "profile", "attributes": { "email": "subscriber@example.com" } } }, "metric": { "data": { "type": "metric", "attributes": { "name": "newsletter signup" } } }, "properties": {}, "time": "1970-01-20T16:13:20.105Z" } } }) ``` ### Viewed product A product view is mapped to the Klaviyo Viewed Product metric with properties such as product name and price. Event ``` { "name": "product view", "data": { "name": "USB Cable", "id": "PROD-1", "price": 9.99 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "email": "user@example.com" }, "nested": [], "consent": { "functional": true }, "id": "846b3b677dc9581e", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Viewed Product", "data": { "map": { "ProductName": "data.name", "ProductID": "data.id", "Price": "data.price" } } } ``` Out ``` eventsApi.createEvent({ "data": { "type": "event", "attributes": { "profile": { "data": { "type": "profile", "attributes": { "email": "user@example.com", "externalId": "us3r" } } }, "metric": { "data": { "type": "metric", "attributes": { "name": "Viewed Product" } } }, "properties": { "ProductName": "USB Cable", "ProductID": "PROD-1", "Price": 9.99 }, "time": "1970-01-20T16:13:20.101Z" } } }) ``` ### Placed order An order complete is sent to Klaviyo as Placed Order with value and currency for revenue attribution. Event ``` { "name": "order complete", "data": { "id": "ORD-123", "total": 99.99, "itemNames": [ "Widget A", "Widget B" ] }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "email": "user@example.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "07b1cc9e0d664d89", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Placed Order", "data": { "map": { "OrderId": "data.id", "value": "data.total", "ItemNames": "data.itemNames" } }, "settings": { "value": "data.total" } } ``` Out ``` eventsApi.createEvent({ "data": { "type": "event", "attributes": { "profile": { "data": { "type": "profile", "attributes": { "email": "user@example.com", "externalId": "us3r" } } }, "metric": { "data": { "type": "metric", "attributes": { "name": "Placed Order" } } }, "properties": { "OrderId": "ORD-123", "value": 99.99, "ItemNames": [ "Widget A", "Widget B" ] }, "time": "1970-01-20T16:13:20.102Z", "valueCurrency": "EUR" } } }) ``` ### User login identify A user login upserts the Klaviyo profile with name, organization, and custom properties without firing an event. Event ``` { "name": "user login", "data": { "firstName": "Jane", "lastName": "Doe", "company": "Acme Corp", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "email": "user@acme.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "05174c073faf2a23", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "firstName": "data.firstName", "lastName": "data.lastName", "organization": "data.company", "properties": { "map": { "plan": "data.plan" } } } } } } ``` Out ``` profilesApi.createOrUpdateProfile({ "data": { "type": "profile", "attributes": { "email": "user@acme.com", "externalId": "us3r", "firstName": "Jane", "lastName": "Doe", "organization": "Acme Corp", "properties": { "plan": "premium" } } } }) ``` Identity is resolved automatically from each event: `email` defaults to `user.email` and `externalId` defaults to `user.id`. Klaviyo requires at least one identifier (`email`, `phoneNumber`, or `externalId`) per event. Events without any identifier are skipped with a warning. ## Revenue tracking[​](#revenue-tracking "Direct link to Revenue tracking") Map a mapping rule's `settings.value` to a numeric event property. When `settings.currency` is also set, the destination adds `valueCurrency` on the Klaviyo event for revenue reporting. ## Ecommerce metric naming[​](#ecommerce-metric-naming "Direct link to Ecommerce metric naming") Klaviyo's built-in flows and reports key off specific metric names. Use `mapping.name` to rename walkerOS events to Klaviyo's expected values: | walkerOS Event | Klaviyo Metric | Unlocks | | ---------------- | ---------------- | ---------------------- | | `product view` | `Viewed Product` | Product analytics | | `product add` | `Added to Cart` | Cart abandonment flows | | `order complete` | `Placed Order` | Revenue reporting, CLV | --- # LinkedIn Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/linkedin) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/linkedin)[ ](https://www.npmjs.com/package/@walkeros/server-destination-linkedin) [Package](https://www.npmjs.com/package/@walkeros/server-destination-linkedin) Server-side event delivery to LinkedIn's Conversions API for enhanced ad attribution and privacy-compliant conversion tracking. Where this fits LinkedIn Conversions API is a **server destination** in the walkerOS flow: Sends conversion events server-side to LinkedIn's Conversions API, bypassing browser limitations for improved data quality. Deduplicates with the browser-side Insight Tag via \`eventId\`. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-linkedin ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationLinkedIn } from '@walkeros/server-destination-linkedin'; await startFlow({ destinations: { linkedin: { code: destinationLinkedIn, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', conversionRuleId: 'YOUR_CONVERSION_RULE_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "linkedin": { "package": "@walkeros/server-destination-linkedin", "import": "destinationLinkedIn", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "conversionRuleId": "YOUR_CONVERSION_RULE_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------- | ------------------------ | ------------------------------------------------------------------ | ---- | | `accessToken*` | `string` | LinkedIn OAuth 2.0 Bearer token for Conversions API authentication | | | `conversionRuleId*` | `string` | Default LinkedIn conversion rule ID from Campaign Manager | | | `apiVersion` | `string` | Linkedin-Version header value in YYYYMM format | | | `doNotHash` | `Array` | Array of user data fields that should not be hashed | | | `url` | `string` | Custom URL for LinkedIn Conversions API endpoint | | | `user_data` | `Record` | Mapping configuration for user data fields | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------ | ------------------ | -------------------------------------------------------------- | ---- | | `conversion` | `conversion` | Per-event conversion override with ruleId, value, and currency | | | `ruleId` | `string` | Override conversion rule ID for this event | | | `value` | `string \| number` | Conversion monetary value | | | `currency` | `string` | ISO 4217 currency code | | ## Examples ### Lead A form submission is posted to LinkedIn as a conversion with the SHA-256 hashed email as the user identifier. Event ``` { "name": "form submit", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000901000", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` sendServer("https://api.linkedin.com/rest/conversionEvents", "{\"elements\":[{\"conversion\":\"urn:lla:llaPartnerConversion:12345678\",\"conversionHappenedAt\":1700000901000,\"user\":{\"userIds\":[{\"idType\":\"SHA256_EMAIL\",\"idValue\":\"b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514\"}]},\"eventId\":\"ev-1700000901000\"}]}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json", "X-Restli-Protocol-Version": "2.0.0", "X-RestLi-Method": "BATCH_CREATE", "Linkedin-Version": "202604" } }) ``` ### Purchase A completed order is sent to the LinkedIn Conversions API with conversion value, currency, and hashed email. Event ``` { "name": "order complete", "data": { "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "jane@example.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000900000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } } ``` Out ``` sendServer("https://api.linkedin.com/rest/conversionEvents", "{\"elements\":[{\"conversion\":\"urn:lla:llaPartnerConversion:12345678\",\"conversionHappenedAt\":1700000900000,\"user\":{\"userIds\":[{\"idType\":\"SHA256_EMAIL\",\"idValue\":\"8c87b489ce35cf2e2f39f80e282cb2e804932a56a213983eeeb428407d43b52d\"}]},\"eventId\":\"ev-1700000900000\",\"conversionValue\":{\"currencyCode\":\"EUR\",\"amount\":\"249.99\"}}]}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json", "X-Restli-Protocol-Version": "2.0.0", "X-RestLi-Method": "BATCH_CREATE", "Linkedin-Version": "202604" } }) ``` ### Purchase with li\_fat\_id A purchase is sent to LinkedIn with both the hashed email and the first-party li\_fat\_id tracking identifier. Event ``` { "name": "order complete", "data": { "total": 89.99, "currency": "USD" }, "context": { "li_fat_id": [ "abc123-fat-id", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "buyer@co.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000902000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "USD" } } } }, "data": { "map": { "user_data": { "map": { "li_fat_id": "context.li_fat_id" } } } } } ``` Out ``` sendServer("https://api.linkedin.com/rest/conversionEvents", "{\"elements\":[{\"conversion\":\"urn:lla:llaPartnerConversion:12345678\",\"conversionHappenedAt\":1700000902000,\"user\":{\"userIds\":[{\"idType\":\"SHA256_EMAIL\",\"idValue\":\"484c39bfb51212665d9673805c112b5ba04cbf0460b6d3f00bcdc18b92afed66\"},{\"idType\":\"LINKEDIN_FIRST_PARTY_ADS_TRACKING_UUID\",\"idValue\":\"abc123-fat-id\"}]},\"eventId\":\"ev-1700000902000\",\"conversionValue\":{\"currencyCode\":\"USD\",\"amount\":\"89.99\"}}]}", { "headers": { "Authorization": "Bearer s3cr3t", "Content-Type": "application/json", "X-Restli-Protocol-Version": "2.0.0", "X-RestLi-Method": "BATCH_CREATE", "Linkedin-Version": "202604" } }) ``` ## User identification[​](#user-identification "Direct link to User identification") The destination builds a `userIds` array from the event data. At least one identifier is required or the event is silently skipped. **SHA256\_EMAIL**: resolved from `event.user.email` or a `user_data.email` mapping. The email is normalized (trimmed, lowercased) and SHA-256 hashed before sending. Set `doNotHash: ['email']` if you are pre-hashing upstream. **LINKEDIN\_FIRST\_PARTY\_ADS\_TRACKING\_UUID**: the LinkedIn click ID (`li_fat_id`) forwarded from the browser session. Map it via `user_data` in settings or through event mapping: ``` "settings": { "accessToken": "$env.LINKEDIN_ACCESS_TOKEN", "conversionRuleId": "$env.LINKEDIN_CONVERSION_RULE_ID", "user_data": { "li_fat_id": "context.li_fat_id" } } ``` ## Per-event conversion override[​](#per-event-conversion-override "Direct link to Per-event conversion override") The default `conversionRuleId` from settings applies to all events. Override it per event via mapping to route different events to different Campaign Manager conversion rules: ``` "mapping": { "order": { "complete": { "settings": { "conversion": { "map": { "ruleId": { "value": "99887766" }, "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } } } } ``` ## Browser deduplication[​](#browser-deduplication "Direct link to Browser deduplication") When running both the web [LinkedIn Insight Tag](/docs/destinations/web/linkedin.md) and this server destination, use the same `eventId` (from `event.id`) on both sides. LinkedIn deduplicates conversions that share the same `eventId` and conversion rule, so only one conversion is counted. --- # Meta Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/meta) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/meta)[ ](https://www.npmjs.com/package/@walkeros/server-destination-meta) [Package](https://www.npmjs.com/package/@walkeros/server-destination-meta) Server-side event delivery to Meta's Conversions API for enhanced tracking accuracy and privacy compliance. Where this fits Meta CAPI is a **server destination** in the walkerOS flow: Sends events server-side to Meta's Conversions API, bypassing browser limitations for improved data quality and privacy-compliant tracking. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-meta ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationMeta } from '@walkeros/server-destination-meta'; await startFlow({ destinations: { meta: { code: destinationMeta, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', pixelId: 'YOUR_PIXEL_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "meta": { "package": "@walkeros/server-destination-meta", "import": "destinationMeta", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------- | ---------------------------------- | --------------------------------------------------------- | ---- | | `accessToken*` | `string` | Meta access token for Conversions API authentication | | | `pixelId*` | `string` | Meta Pixel ID from your Facebook Business account | | | `action_source` | `'email' \| 'website' \| 'app' \|` | Source of the event (website, app, phone\_call, etc.) | | | `doNotHash` | `Array` | Array of user\_data fields that should not be hashed | | | `test_event_code` | `string` | Test event code for debugging Meta Conversions API events | | | `url` | `string` | Custom URL for Meta Conversions API endpoint | | | `user_data` | `Record` | Mapping configuration for user data fields | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Form submit A form submission is forwarded to Meta CAPI as a custom event with the event source URL. Event ``` { "name": "form submit", "data": { "type": "newsletter" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "c1d2e3f4a5b60002", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000901, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com" } } ``` Out ``` sendServer("https://graph.facebook.com/v22.0/p1x3l1d/events", "{\"data\":[{\"event_name\":\"form submit\",\"event_id\":\"c1d2e3f4a5b60002\",\"event_time\":1700001,\"action_source\":\"website\",\"user_data\":{},\"event_source_url\":\"https://example.com\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Purchase A completed order is sent to the Meta Conversions API as a Purchase event with value, currency, and contents. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "price": 129.99, "quantity": 2 } } ], "consent": { "functional": true }, "id": "c1d2e3f4a5b60001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com" } } ``` Mapping ``` { "name": "Purchase", "data": { "map": { "order_id": "data.id", "currency": { "key": "data.currency", "value": "EUR" }, "value": "data.total", "contents": { "loop": [ "nested", { "condition": { "$code": "e=>y(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "item_price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] }, "num_items": { "fn": { "$code": "e=>e.nested.filter(e=>\"product\"===e.entity).length" } } } } } ``` Out ``` sendServer("https://graph.facebook.com/v22.0/p1x3l1d/events", "{\"data\":[{\"event_name\":\"Purchase\",\"event_id\":\"c1d2e3f4a5b60001\",\"event_time\":1700001,\"action_source\":\"website\",\"order_id\":\"ORD-300\",\"currency\":\"EUR\",\"value\":249.99,\"contents\":[{\"id\":\"SKU-A1\",\"item_price\":129.99,\"quantity\":2}],\"num_items\":1,\"user_data\":{},\"event_source_url\":\"https://shop.example.com\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Purchase with fbclid A purchase is sent to Meta CAPI with an external\_id and a formatted fbc click id for ads attribution. Event ``` { "name": "order complete", "data": { "id": "ORD-700", "total": 89.99, "currency": "USD" }, "context": { "fbclid": [ "abc123xyz", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "cust-42" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "c1d2e3f4a5b60003", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000902, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com" } } ``` Mapping ``` { "name": "Purchase", "data": { "map": { "currency": { "key": "data.currency", "value": "EUR" }, "value": "data.total", "order_id": "data.id", "user_data": { "map": { "external_id": "user.id", "fbclid": "context.fbclid" } } } } } ``` Out ``` sendServer("https://graph.facebook.com/v22.0/p1x3l1d/events", "{\"data\":[{\"event_name\":\"Purchase\",\"event_id\":\"c1d2e3f4a5b60003\",\"event_time\":1700001,\"action_source\":\"website\",\"currency\":\"USD\",\"value\":89.99,\"order_id\":\"ORD-700\",\"user_data\":{\"external_id\":\"8a3c5a67cad508582b5edf6b8352cea3ffbad7f44812c1a736b4444c0f5746aa\",\"fbc\":\"fb.1.1700000902.abc123xyz,0\"},\"event_source_url\":\"https://shop.example.com\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` --- # Mixpanel [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/mixpanel) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/mixpanel)[ ](https://www.npmjs.com/package/@walkeros/server-destination-mixpanel) [Package](https://www.npmjs.com/package/@walkeros/server-destination-mixpanel) Server-side event delivery to Mixpanel for product analytics, people profiles, and group analytics via the official [`mixpanel`](https://www.npmjs.com/package/mixpanel) Node.js SDK. Where this fits Mixpanel is a **server destination** in the walkerOS flow: Sends events server-side to Mixpanel via \`/track\` (real-time, up to 5 days old) or \`/import\` (historical, any age). Supports people profile operations, group analytics, and identity resolution. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-mixpanel ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationMixpanel } from '@walkeros/server-destination-mixpanel'; await startFlow({ destinations: { mixpanel: { code: destinationMixpanel, config: { settings: { apiKey: 'YOUR_PROJECT_TOKEN', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "mixpanel": { "package": "@walkeros/server-destination-mixpanel", "import": "destinationMixpanel", "config": { "settings": { "apiKey": "YOUR_PROJECT_TOKEN" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `apiKey*` | `string` | Your Mixpanel project token. Find it in Project Settings > Access Keys. Passed as the first argument to Mixpanel.init(). | | | `secret` | `string` | API secret for the /import endpoint (historical data). Required when useImport is true. | | | `host` | `string` | Mixpanel API host. Default: 'api.mixpanel.com' (US). Use 'api-eu.mixpanel.com' (EU) or 'api-in.mixpanel.com' (India). | | | `protocol` | `string` | Protocol for API requests. Default: 'https'. | | | `keepAlive` | `boolean` | Reuse HTTP connections. Default: true. | | | `geolocate` | `boolean` | Parse IP for geolocation. Default: false. Server IP caveat: all users map to server location unless $ip is overridden. | | | `debug` | `boolean` | Enable SDK debug logging. Default: false. | | | `verbose` | `boolean` | Enable verbose request logging. Default: false. | | | `test` | `boolean` | Enable dry-run mode. Default: false. | | | `useImport` | `boolean` | Use /import endpoint instead of /track. Accepts events of any age (no 5-day limit). Requires secret for authentication. | | | `identify` | `any` | walkerOS mapping value resolving to { distinctId, alias? }. distinctId is passed as distinct\_id on every SDK call. | | | `include` | `any` | Event data sections to flatten into track() properties. Example: \['data', 'globals']. Sections are prefixed (data\_, globals\_, etc.). | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { distinctId, alias? }. distinctId is passed as distinct\_id to all SDK calls. | | | `people` | `any` | Per-event people operations. Resolves to an object with any of: set, set\_once, increment, append, union, remove, unset, delete\_user. Each key fires a separate mp.people.\* call with distinct\_id as first arg. | | | `group` | `any` | Per-event group association. Resolves to { key, id }. The group key/id is added as a track() property. | | | `groupProfile` | `any` | Per-event group profile operations. Resolves to { key, id, set?, set\_once?, union?, remove?, unset?, delete? }. Fires mp.groups.\* calls. | | | `useImport` | `any` | Per-event import flag. When truthy, uses mp.import() instead of mp.track() for this rule. | | ## Examples ### Alias before track A user login merges a prior anonymous id into the new user id via mp.alias before sending the track event. Event ``` { "name": "user login", "data": { "user_id": "new-user-456", "anon_id": "anon-789" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0ae94de2675a8e7a", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "identify": { "map": { "distinctId": "data.user_id", "alias": "data.anon_id" } } } } ``` Out ``` mp.alias("new-user-456", "anon-789"); mp.track("user login", { "distinct_id": "new-user-456" }) ``` ### Common people operations A profile update exercises the common Mixpanel people vocabulary: set, set\_once, increment, append, union, remove, and unset. delete\_user is not covered by this example. Event ``` { "name": "profile update", "data": { "name": "Jane Doe", "email": "jane@acme.com", "page": "/docs/getting-started", "removed_tag": "trial", "source": "referral" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "de800028f1427d64", "trigger": "test", "entity": "profile", "action": "update", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "distinctId": "user.id" } }, "people": { "map": { "set": { "map": { "name": "data.name", "email": "data.email" } }, "set_once": { "map": { "signup_source": "data.source" } }, "increment": { "map": { "page_views": { "value": 1 } } }, "append": { "map": { "visited_pages": "data.page" } }, "union": { "map": { "unique_tags": { "value": [ "active" ] } } }, "remove": { "map": { "tags": "data.removed_tag" } }, "unset": { "value": [ "old_plan" ] } } } } } ``` Out ``` mp.people.set("us3r", { "name": "Jane Doe", "email": "jane@acme.com" }); mp.people.set_once("us3r", { "signup_source": "referral" }); mp.people.increment("us3r", { "page_views": 1 }); mp.people.append("us3r", { "visited_pages": "/docs/getting-started" }); mp.people.union("us3r", { "unique_tags": [ "active" ] }); mp.people.remove("us3r", { "tags": "trial" }); mp.people.unset("us3r", [ "old_plan" ]) ``` ### Group profile A company update sets Mixpanel group profile properties via groups.set and groups.set\_once. Event ``` { "name": "company update", "data": { "company_id": "acme-inc", "company_name": "Acme, Inc.", "plan": "enterprise", "employee_count": 250, "founded_year": 2010 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "7d7fb50a79dae71c", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "groupProfile": { "map": { "key": { "value": "company_id" }, "id": "data.company_id", "set": { "map": { "name": "data.company_name", "plan": "data.plan", "employee_count": "data.employee_count" } }, "set_once": { "map": { "founded": "data.founded_year" } } } } } } ``` Out ``` mp.groups.set("company_id", "acme-inc", { "name": "Acme, Inc.", "plan": "enterprise", "employee_count": 250 }); mp.groups.set_once("company_id", "acme-inc", { "founded": 2010 }) ``` ### Default track A walkerOS event is forwarded to Mixpanel as a track call with the user id as the distinct\_id. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "c1926fae0f3aa97b", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` mp.track("product view", { "distinct_id": "us3r" }) ``` ### Historical import Setting useImport routes the event through mp.import for backfilling historical Mixpanel data. Event ``` { "name": "order complete", "data": { "total": 99.99 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "55d0af6be4e6e40b", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` mp.import("order complete", 1700000107, { "distinct_id": "us3r" }) ``` ### Per-event identify A mapping-level identify overrides the destination default to resolve the distinct\_id from event data. Event ``` { "name": "user login", "data": { "user_id": "resolved-id", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "cff918d140256cb9", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "identify": { "map": { "distinctId": "data.user_id" } } } } ``` Out ``` mp.track("user login", { "distinct_id": "resolved-id" }) ``` ### Track with group A group key and id are attached as a Mixpanel track property so the event is associated with a company or account. Event ``` { "name": "page view", "data": { "company_id": "acme" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "046b4f610b57b4fa", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "group": { "map": { "key": { "value": "company_id" }, "id": "data.company_id" } } } } ``` Out ``` mp.track("page view", { "distinct_id": "us3r", "company_id": "acme" }) ``` ### Track with include A destination-level include flattens the event data section into prefixed Mixpanel track properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "b79c9ddfa30476db", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` mp.track("product view", { "distinct_id": "us3r", "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420 }) ``` ### User login people A user login fires Mixpanel people.set, set\_once, and increment operations without sending a track event. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "plan": "premium", "company": "Acme", "email": "user@acme.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "66d81138c5b731dc", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "distinctId": "data.user_id" } }, "people": { "map": { "set": { "map": { "plan": "data.plan", "company": "data.company", "email": "data.email" } }, "set_once": { "map": { "first_login": "timestamp" } }, "increment": { "map": { "login_count": { "value": 1 } } } } } } } ``` Out ``` mp.people.set("new-user-123", { "plan": "premium", "company": "Acme", "email": "user@acme.com" }); mp.people.set_once("new-user-123", { "first_login": 1700000104 }); mp.people.increment("new-user-123", { "login_count": 1 }) ``` --- # mParticle [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/mparticle) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/mparticle)[ ](https://www.npmjs.com/package/@walkeros/server-destination-mparticle) [Package](https://www.npmjs.com/package/@walkeros/server-destination-mparticle) Beta Server-side event delivery to [mParticle](https://www.mparticle.com/) via the HTTP Events API. Events are packaged into batches and POSTed to the regional mParticle pod using HTTP Basic auth (`apiKey` / `apiSecret`). Supports user identities, user attributes, consent state, and environment targeting. Where this fits mParticle is a **server destination** in the walkerOS flow: Receives events server-side from the collector and forwards them to mParticle's input feed for fan-out to downstream destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-mparticle ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationMParticle } from '@walkeros/server-destination-mparticle'; await startFlow({ destinations: { mparticle: { code: destinationMParticle, config: { settings: { apiKey: 'YOUR_MPARTICLE_API_KEY', apiSecret: 'YOUR_MPARTICLE_API_SECRET', pod: 'us1', environment: 'production', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "mparticle": { "package": "@walkeros/server-destination-mparticle", "config": { "settings": { "apiKey": "YOUR_MPARTICLE_API_KEY", "apiSecret": "YOUR_MPARTICLE_API_SECRET", "pod": "us1", "environment": "production" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `apiKey*` | `string` | mParticle input feed API key from the mParticle dashboard (Setup > Inputs > Feeds). | | | `apiSecret*` | `string` | mParticle input feed API secret paired with apiKey. Used for HTTP Basic auth. | | | `pod` | `'us1' \| 'us2' \| 'eu1' \| 'au1'` | mParticle data pod selecting the regional endpoint. Default: 'us1'. | | | `environment` | `'production' \| 'development'` | Environment the batch targets. Default: 'production'. | | | `userIdentities` | `Record` | Mapping that resolves to user\_identities per batch. Keys are mParticle identity types (like customer\_id, email); values are walkerOS mapping values. | | | `userAttributes` | `any` | Mapping value that resolves to the user\_attributes object placed on the batch. | | | `consent` | `Record` | Static consent\_state envelope forwarded verbatim on the batch. See mParticle consent\_state docs. | | | `ip` | `any` | Mapping value resolving to the client IP for the batch. | | | `sourceRequestId` | `any` | Mapping value resolving to the source\_request\_id for the batch. Falls back to event.id when unset. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ----------------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ---- | | `eventType` | `'custom_event' \| 'screen_view'` | Per-event mParticle event type. Default: 'custom\_event'. | | | `customEventType` | `'navigation' \| 'location' \| 's` | Custom event type category for 'custom\_event'. Default: 'other'. | | | `commerce` | `any` | Mapping value resolving to the commerce fields (product\_action, currency\_code, products, ...) for a commerce\_event. | | | `userIdentities` | `Record` | Per-event override mapping for user\_identities. Merged over settings.userIdentities. | | | `userAttributes` | `any` | Per-event override mapping for user\_attributes. | | ## Examples ### Commerce purchase A completed order becomes an mParticle commerce\_event with a purchase product\_action block. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000300000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000300000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "eventType": "commerce_event", "commerce": { "map": { "currency_code": "data.currency", "product_action": { "map": { "action": { "value": "purchase" }, "transaction_id": "data.id", "total_amount": "data.total" } } } } } } ``` Out ``` sendServer("https://s2s.mparticle.com/v2/events", "{\"events\":[{\"event_type\":\"commerce_event\",\"data\":{\"currency_code\":\"EUR\",\"product_action\":{\"action\":\"purchase\",\"transaction_id\":\"ORD-300\",\"total_amount\":249.99},\"timestamp_unixtime_ms\":1700000300000,\"source_message_id\":\"ev-1700000300000\"}}],\"environment\":\"production\",\"user_identities\":{\"customer_id\":\"user-123\"},\"source_request_id\":\"ev-1700000300000\"}", { "headers": { "Authorization": "Basic a2V5OnNlY3JldA==", "Content-Type": "application/json" } }) ``` ### Custom event A walker event is sent to mParticle as a custom\_event with user identities resolved from destination settings. Event ``` { "name": "product view", "data": { "id": "SKU-A1", "name": "Shoe", "price": 129.99 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123" }, "nested": [], "consent": { "functional": true }, "id": "ev-1700000100000", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` sendServer("https://s2s.mparticle.com/v2/events", "{\"events\":[{\"event_type\":\"custom_event\",\"data\":{\"event_name\":\"product view\",\"custom_event_type\":\"other\",\"timestamp_unixtime_ms\":1700000100000,\"source_message_id\":\"ev-1700000100000\"}}],\"environment\":\"production\",\"user_identities\":{\"customer_id\":\"user-123\"},\"source_request_id\":\"ev-1700000100000\"}", { "headers": { "Authorization": "Basic a2V5OnNlY3JldA==", "Content-Type": "application/json" } }) ``` ### User identities A form submission sends a custom\_event whose batch carries user\_identities resolved from destination settings. Event ``` { "name": "form submit", "data": { "type": "newsletter" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000400000", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000400000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` sendServer("https://s2s.mparticle.com/v2/events", "{\"events\":[{\"event_type\":\"custom_event\",\"data\":{\"event_name\":\"form submit\",\"custom_event_type\":\"other\",\"timestamp_unixtime_ms\":1700000400000,\"source_message_id\":\"ev-1700000400000\"}}],\"environment\":\"production\",\"user_identities\":{\"customer_id\":\"user-123\",\"email\":\"user@example.com\"},\"source_request_id\":\"ev-1700000400000\"}", { "headers": { "Authorization": "Basic a2V5OnNlY3JldA==", "Content-Type": "application/json" } }) ``` ### Screen view A page view is mapped to an mParticle screen\_view event with the event name as the screen name. Event ``` { "name": "page view", "data": { "title": "Checkout", "path": "/checkout" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "user-123" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000200000", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000200000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "eventType": "screen_view" } } ``` Out ``` sendServer("https://s2s.mparticle.com/v2/events", "{\"events\":[{\"event_type\":\"screen_view\",\"data\":{\"screen_name\":\"page view\",\"timestamp_unixtime_ms\":1700000200000,\"source_message_id\":\"ev-1700000200000\"}}],\"environment\":\"production\",\"user_identities\":{\"customer_id\":\"user-123\"},\"source_request_id\":\"ev-1700000200000\"}", { "headers": { "Authorization": "Basic a2V5OnNlY3JldA==", "Content-Type": "application/json" } }) ``` The `apiKey` and `apiSecret` come from an mParticle **input feed** (Setup, Inputs, Feeds, Custom Feed). Each batch posts to `https://s2s.{pod}.mparticle.com/v2/events` using HTTP Basic auth. ## Identities and attributes[​](#identities-and-attributes "Direct link to Identities and attributes") * `userIdentities`: map walkerOS event fields to mParticle identity types (`customer_id`, `email`, `other`, etc.). * `userAttributes`: resolved from each event and placed on the batch's `user_attributes` object. * `consent`: forwarded verbatim as `consent_state` on the batch. * `sourceRequestId`: defaults to `event.id` if unset; used by mParticle for deduplication. --- # Pinterest Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/pinterest) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/pinterest)[ ](https://www.npmjs.com/package/@walkeros/server-destination-pinterest) [Package](https://www.npmjs.com/package/@walkeros/server-destination-pinterest) Server-side event delivery to Pinterest's Conversions API for enhanced conversion tracking, bypassing browser limitations for improved data quality and privacy compliance. Where this fits Pinterest Conversions API is a **server destination** in the walkerOS flow: Sends events server-side to Pinterest's Conversions API via raw HTTP (\`POST https\://api.pinterest.com/v5/ad\_accounts/{adAccountId}/events\`), hashing 12 user\_data fields with SHA-256 before transmission. Supports deduplication with the web Pinterest Tag via \`event\_id\` + \`event\_name\`. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-pinterest ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationPinterest } from '@walkeros/server-destination-pinterest'; await startFlow({ destinations: { pinterest: { code: destinationPinterest, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', adAccountId: 'YOUR_AD_ACCOUNT_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "pinterest": { "package": "@walkeros/server-destination-pinterest", "import": "destinationPinterest", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "adAccountId": "YOUR_AD_ACCOUNT_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | ---------------------------------- | ----------------------------------------------------------- | ---- | | `accessToken*` | `string` | Pinterest conversion access token for Bearer authentication | | | `adAccountId*` | `string` | Pinterest ad account ID from Ads Manager | | | `action_source` | `'app_android' \| 'app_ios' \| 'w` | Source of the event (web, app\_android, app\_ios, offline) | | | `doNotHash` | `Array` | Array of user\_data fields that should not be hashed | | | `test` | `boolean` | Enable test mode by appending ?test=true to the API URL | | | `url` | `string` | Custom URL for Pinterest Conversions API endpoint | | | `user_data` | `Record` | Mapping configuration for user data fields | | | `partner_name` | `string` | Third-party partner name for attribution | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add is sent to Pinterest as an add\_to\_cart conversion with the added item details. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Cool Cap", "price": "42.00", "quantity": 1 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-456" }, "nested": [], "consent": { "functional": true }, "id": "a1b2c3d4e5f60002", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000901, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products" } } ``` Mapping ``` { "name": "add_to_cart", "data": { "map": { "custom_data": { "map": { "value": "data.price", "currency": { "value": "EUR" }, "contents": { "set": [ { "map": { "id": "data.id", "item_name": "data.name", "item_price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } } } ``` Out ``` sendServer("https://api.pinterest.com/v5/ad_accounts/123456789/events", "{\"data\":[{\"event_name\":\"add_to_cart\",\"event_id\":\"a1b2c3d4e5f60002\",\"event_time\":1700001,\"action_source\":\"web\",\"user_data\":{},\"custom_data\":{\"value\":\"42.00\",\"currency\":\"EUR\",\"contents\":[{\"id\":\"SKU-B2\",\"item_name\":\"Cool Cap\",\"item_price\":\"42.00\",\"quantity\":1}]},\"event_source_url\":\"https://shop.example.com/products\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Checkout A completed order is sent to the Pinterest Conversions API as a checkout event with value, currency, and contents. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Everyday Ruck Snack", "price": "129.99", "quantity": 2 } } ], "consent": { "functional": true }, "id": "a1b2c3d4e5f60001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com" } } ``` Mapping ``` { "name": "checkout", "data": { "map": { "custom_data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "order_id": "data.id", "num_items": { "fn": { "$code": "e=>e.nested.filter(e=>\"product\"===e.entity).length" } }, "contents": { "loop": [ "nested", { "condition": { "$code": "e=>b(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "item_name": "data.name", "item_price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } } } ``` Out ``` sendServer("https://api.pinterest.com/v5/ad_accounts/123456789/events", "{\"data\":[{\"event_name\":\"checkout\",\"event_id\":\"a1b2c3d4e5f60001\",\"event_time\":1700001,\"action_source\":\"web\",\"user_data\":{},\"custom_data\":{\"value\":249.99,\"currency\":\"EUR\",\"order_id\":\"ORD-300\",\"num_items\":1,\"contents\":[{\"id\":\"SKU-A1\",\"item_name\":\"Everyday Ruck Snack\",\"item_price\":\"129.99\",\"quantity\":2}]},\"event_source_url\":\"https://shop.example.com\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Page visit A page view is sent to Pinterest as a page\_visit conversion with the source URL. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "user-789" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a1b2c3d4e5f60003", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000902, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://www.example.com/docs/" } } ``` Mapping ``` { "name": "page_visit" } ``` Out ``` sendServer("https://api.pinterest.com/v5/ad_accounts/123456789/events", "{\"data\":[{\"event_name\":\"page_visit\",\"event_id\":\"a1b2c3d4e5f60003\",\"event_time\":1700001,\"action_source\":\"web\",\"user_data\":{},\"event_source_url\":\"https://www.example.com/docs/\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Search A site search event is forwarded to Pinterest as a search conversion with the query in custom data. Event ``` { "name": "entity action", "data": { "query": "walkerOS destinations" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "user-101" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a1b2c3d4e5f60004", "trigger": "test", "entity": "entity", "action": "action", "timestamp": 1700000903, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://www.example.com/search" } } ``` Mapping ``` { "name": "search", "data": { "map": { "custom_data": { "map": { "search_string": "data.query" } } } } } ``` Out ``` sendServer("https://api.pinterest.com/v5/ad_accounts/123456789/events", "{\"data\":[{\"event_name\":\"search\",\"event_id\":\"a1b2c3d4e5f60004\",\"event_time\":1700001,\"action_source\":\"web\",\"user_data\":{},\"custom_data\":{\"search_string\":\"walkerOS destinations\"},\"event_source_url\":\"https://www.example.com/search\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Signup A user signup is sent to Pinterest as a signup conversion with the hashed email and external id. Event ``` { "name": "entity action", "data": { "method": "email" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "new-user-1", "email": "new@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a1b2c3d4e5f60005", "trigger": "test", "entity": "entity", "action": "action", "timestamp": 1700000904, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://www.example.com/register" } } ``` Mapping ``` { "name": "signup", "data": { "map": { "user_data": { "map": { "em": { "set": [ "user.email" ] }, "external_id": { "set": [ "user.id" ] } } } } } } ``` Out ``` sendServer("https://api.pinterest.com/v5/ad_accounts/123456789/events", "{\"data\":[{\"event_name\":\"signup\",\"event_id\":\"a1b2c3d4e5f60005\",\"event_time\":1700001,\"action_source\":\"web\",\"user_data\":{\"em\":[\"f0030501023327437b06e5c6f87df7871b8e704ae608d1d0b7b24fdd2a06c716\"],\"external_id\":[\"b45cf5f6ebc2c6974ea3bd9fab19f8cc3a7cf63054727a9fcd22f1fda97d6dde\"]},\"event_source_url\":\"https://www.example.com/register\"}]}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` --- # PostHog [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/posthog) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/posthog)[ ](https://www.npmjs.com/package/@walkeros/server-destination-posthog) [Package](https://www.npmjs.com/package/@walkeros/server-destination-posthog) Server-side event delivery to PostHog for product analytics, identity resolution, group analytics, and feature flags via the official `posthog-node` SDK. Where this fits PostHog is a **server destination** in the walkerOS flow: Sends events server-side to PostHog's ingestion API, translating walkerOS events into \`capture\`, \`identify\`, and \`groupIdentify\` calls with full control over identity and group resolution. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-posthog ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationPostHog } from '@walkeros/server-destination-posthog'; await startFlow({ destinations: { posthog: { code: destinationPostHog, config: { settings: { apiKey: 'phc_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', host: 'https://eu.i.posthog.com', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "posthog": { "package": "@walkeros/server-destination-posthog", "import": "destinationPostHog", "config": { "settings": { "apiKey": "phc_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", "host": "https://eu.i.posthog.com" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `apiKey*` | `string` | PostHog project API key (starts with "phc\_"). Find it in PostHog project settings. | | | `host` | `string` | PostHog API host. Defaults to https\://us.i.posthog.com. Use https\://eu.i.posthog.com for EU or your self-hosted URL. | | | `flushAt` | `number` | Number of events queued before auto-flush. Default: 20. | | | `flushInterval` | `number` | Milliseconds between periodic flushes. Default: 10000. | | | `personalApiKey` | `string` | Personal API key (phx\_...) for local feature flag evaluation. | | | `featureFlagsPollingInterval` | `number` | Milliseconds between feature flag definition polls. Default: 30000. | | | `disableGeoip` | `boolean` | Disable GeoIP lookups globally. Useful for GDPR compliance. | | | `debug` | `boolean` | Enable PostHog SDK debug logging. Default: false. | | | `identify` | `any` | walkerOS mapping value resolving to an identity object. Keys: distinctId, $set, $set\_once. Resolved on every push (server is stateless). | | | `group` | `any` | walkerOS mapping value resolving to a group object. Keys: type, key, properties. Resolved on every push. | | | `include` | `Array` | Event sections to flatten into capture() properties (e.g. \["data", "globals"]). | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ----- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { distinctId, $set?, $set\_once? }. Fires client.identify() when $set/$set\_once present. | | | `group` | `any` | Group assignment. Resolves to { type, key, properties? }. Fires client.groupIdentify() when properties present, adds groups to capture(). | | ## Examples ### Capture with group A destination-level group mapping attaches the resolved group context to every PostHog capture call. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs", "company_id": "company_123" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "10afa9c8122c872c", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.capture({ "distinctId": "us3r", "event": "page view", "properties": {}, "groups": { "company": "company_123" } }) ``` ### Capture with include Destination-level include flattens data and globals sections into prefixed PostHog event properties. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "4ece047572218ea1", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.capture({ "distinctId": "us3r", "event": "order complete", "properties": { "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "globals_pagegroup": "shop" } }) ``` ### Consent granted A walker consent command with analytics granted calls client.enable on the PostHog client. Event ``` { "analytics": true } ``` Out ``` client.enable() ``` ### Consent revoked A walker consent command with analytics denied calls client.disable on the PostHog client. Event ``` { "analytics": false } ``` Out ``` client.enable(); client.disable() ``` ### Default capture A walker event becomes a PostHog capture call with the user id as distinctId and no extra properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "78d5e98ac9042274", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.capture({ "distinctId": "us3r", "event": "product view", "properties": {} }) ``` ### Group identify A company update fires PostHog groupIdentify with group type, key, and associated group properties. Event ``` { "name": "company update", "data": { "company_id": "company_123", "company_name": "Acme", "plan": "enterprise" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "18d4b2dc5be72473", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "group": { "map": { "type": { "value": "company" }, "key": "data.company_id", "properties": { "map": { "name": "data.company_name", "plan": "data.plan" } } } } } } ``` Out ``` client.groupIdentify({ "groupType": "company", "groupKey": "company_123", "properties": { "name": "Acme", "plan": "enterprise" } }) ``` ### Identify with $set A user login fires PostHog identify with $set and $set\_once person properties and skips the capture. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "email": "user@acme.com", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "bdc21a0e4502ec9e", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "distinctId": "data.user_id", "$set": { "map": { "email": "data.email", "plan": "data.plan" } }, "$set_once": { "map": { "first_login": "timestamp" } } } } } } ``` Out ``` client.identify({ "distinctId": "new-user-123", "properties": { "$set": { "email": "user@acme.com", "plan": "premium" }, "$set_once": { "first_login": 1700000102 } } }) ``` --- # Google Pub/Sub [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/gcp) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/gcp)[ ](https://www.npmjs.com/package/@walkeros/server-destination-gcp) [Package](https://www.npmjs.com/package/@walkeros/server-destination-gcp) Server-side event publishing to [Google Cloud Pub/Sub](https://cloud.google.com/pubsub) topics. Each event is serialized as JSON, optionally keyed for per-message ordering, and published via the official `@google-cloud/pubsub` SDK. Supports Application Default Credentials, service account JSON, or pre-configured client injection. Per-rule topic, ordering-key, and attribute overrides. Idempotent topic provisioning with EU storage policy by default. The destination ships inside `@walkeros/server-destination-gcp` alongside BigQuery; install the package once and import `destinationPubSub` for Pub/Sub. Where this fits Pub/Sub is a **server destination** in the walkerOS flow: Publishes events to a Pub/Sub topic for fan-out, asynchronous processing, cross-region delivery, and integration with downstream consumers (Dataflow, Cloud Functions, Cloud Run, custom subscribers). ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-gcp ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationPubSub } from '@walkeros/server-destination-gcp'; await startFlow({ destinations: { pubsub: { code: destinationPubSub, config: { settings: { projectId: 'YOUR_PROJECT_ID', topic: 'events', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "pubsub": { "package": "@walkeros/server-destination-gcp", "import": "destinationPubSub", "config": { "settings": { "projectId": "YOUR_PROJECT_ID", "topic": "events" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `client` | `any` | Google Cloud Pub/Sub client instance | | | `projectId*` | `string` | Google Cloud Project ID | | | `topic*` | `string` | Pub/Sub topic short name (like events). The full resource name projects/\/topics/\ is built by the SDK. | | | `credentials` | `any` | Service account credentials as a JSON string or an object with client\_email and private\_key. Default: Application Default Credentials (ADC). (deprecated: use config.credentials) | | | `apiEndpoint` | `string` | Override Pub/Sub API endpoint. Useful for the local emulator (like localhost:8085). | | | `orderingKey` | `any` | Mapping value resolved per-event. Truthy enables per-key ordering for the publish. | | | `attributes` | `any` | Default per-event attribute map merged into every published message. Mapping.Map shape. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------- | -------- | ----------------------------------------------------------------------------- | ---- | | `topic` | `string` | Per-rule topic override. Falls back to settings.topic if absent. | | | `orderingKey` | `any` | Per-rule ordering-key Mapping.Value. Overrides settings.orderingKey when set. | | | `attributes` | `any` | Per-rule attribute Mapping.Map merged on top of settings.attributes. | | ## Authentication[​](#authentication "Direct link to Authentication") Three modes, evaluated in order: 1. **Application Default Credentials (ADC).** Nothing to configure beyond `projectId`. Works on GCP-native runtimes (Cloud Run, Cloud Functions, GKE, GCE) where the runtime service account is attached, and locally with `gcloud auth application-default login`. 2. **Service account JSON.** Pass credentials via `config.credentials`, either as a parsed object or a JSON string (the destination JSON-parses strings). Pair with `$env.NAME` to inject from an environment variable. (The package-specific `settings.credentials` still works but is deprecated.) 3. **Pre-configured client.** Pass an existing `PubSub` SDK instance as `settings.client`. Useful for shared clients across destinations or custom transport configuration. When `settings.projectId` and the credentials' `project_id` both resolve, the top-level `settings.projectId` wins. The runtime publish target stays unambiguous. ## Ordering[​](#ordering "Direct link to Ordering") `orderingKey` is a `Mapping.Value` resolved per event. Truthy resolved values enable per-key ordering, the topic handle is constructed with `messageOrdering: true`, and the resolved key flows on the publish. ``` { "settings": { "projectId": "my-project", "topic": "user-events", "orderingKey": "user.id" } } ``` Per-rule overrides live under `mapping...settings.orderingKey`. If a publish fails for an ordered key, Pub/Sub permanently halts subsequent publishes for that key until `topic.resumePublishing(key)` is called. The destination handles this automatically: on publish failure for an ordered key it calls `resumePublishing` immediately and re-throws so the caller observes the original error. ## Attributes[​](#attributes "Direct link to Attributes") `settings.attributes` is a `Mapping.Map` of attribute name to mapping value. Each value is resolved per event and stringified. Per-rule overrides under `mapping...settings.attributes` are merged onto the defaults. ``` { "settings": { "projectId": "my-project", "topic": "events", "attributes": { "tenant": "user.tenant_id", "schema_version": { "value": "v4" } } } } ``` ## Setup[​](#setup "Direct link to Setup") Provision the topic once per environment: ``` walkeros setup destination.pubsub ``` Idempotent: re-running on an existing topic logs `setup: topic exists` and runs drift detection. Drift on `messageStoragePolicy.allowedPersistenceRegions`, `messageRetentionDuration`, `kmsKeyName`, or `labels` emits `WARN setup.drift {...}` and never auto-mutates. Migrations are an operator decision. `config.setup`: * `false` (default): no provisioning. Operator must run setup explicitly. * `true`: provisions with safe defaults (EU multi-region storage policy). * `{ messageStoragePolicy, messageRetentionDuration, kmsKeyName, labels }`: object form for explicit overrides. Default storage regions: `['europe-west1', 'europe-west3', 'europe-west4']` (EU multi-region). Override `messageStoragePolicy.allowedPersistenceRegions` for projects with org policies that restrict geography. The provisioning identity needs `pubsub.topics.create` and `pubsub.topics.get` on the project. The runtime publish identity needs `roles/pubsub.publisher` on the topic itself. Subscription provisioning is owned by the [Pub/Sub source](/docs/sources/server/pubsub.md), not this destination. ## Emulator[​](#emulator "Direct link to Emulator") The official Pub/Sub emulator runs locally for development: ``` gcloud beta emulators pubsub start --host-port=localhost:8085 export PUBSUB_EMULATOR_HOST=localhost:8085 ``` The SDK automatically picks up `PUBSUB_EMULATOR_HOST`. For explicit configuration, set `settings.apiEndpoint` (e.g. `localhost:8085`). ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") * **`NOT_FOUND` on publish**: the topic does not exist. Run `walkeros setup destination.` once. * **`PERMISSION_DENIED` / `UNAUTHENTICATED`**: the runtime service account lacks `roles/pubsub.publisher` on the topic, or ADC is not configured. The destination logs the actionable message including the topic and project IDs. * **Ordering stuck**: a previous publish for an ordering key failed. The destination calls `resumePublishing` automatically; if stalls persist, check publish-side error logs for the original failure. ## Next steps[​](#next-steps "Direct link to Next steps") * [Pub/Sub source](/docs/sources/server/pubsub.md) for consuming events from a Pub/Sub topic * [GCP package overview](/docs/destinations/server/gcp.md) for BigQuery and shared GCP setup --- # Reddit Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/reddit) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/reddit)[ ](https://www.npmjs.com/package/@walkeros/server-destination-reddit) [Package](https://www.npmjs.com/package/@walkeros/server-destination-reddit) Beta Server-side event delivery to Reddit's Conversions API v2.0 for enhanced conversion tracking, bypassing browser limitations and improving match quality for ad optimization. Where this fits Reddit Conversions API is a **server destination** in the walkerOS flow: Sends events server-side to Reddit's Conversions API using raw HTTP (no SDK). Identity fields (\`email\`, \`external\_id\`, \`ip\_address\`, \`user\_agent\`, \`idfa\`, \`aaid\`) are SHA-256 hashed before sending. Events include a stable \`event\_metadata.conversion\_id\` for deduplication with the Reddit Pixel. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-reddit ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationReddit } from '@walkeros/server-destination-reddit'; await startFlow({ destinations: { reddit: { code: destinationReddit, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', pixelId: 'YOUR_PIXEL_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "reddit": { "package": "@walkeros/server-destination-reddit", "import": "destinationReddit", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | ---------------------------------- | ---------------------------------------------------------------- | ---- | | `accessToken*` | `string` | Reddit Conversion Access Token for Bearer authentication | | | `pixelId*` | `string` | Reddit Pixel ID used as the API path parameter | | | `action_source` | `'WEBSITE' \| 'APP' \| 'PHYSICAL_` | Source of the event (WEBSITE, APP, PHYSICAL\_STORE) | | | `doNotHash` | `Array` | Array of user fields that should not be hashed | | | `test_mode` | `boolean` | Enable test mode by sending test\_mode: true in the request body | | | `url` | `string` | Custom URL for Reddit Conversions API endpoint | | | `user_data` | `Record` | Mapping configuration for user fields | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add is sent to Reddit as an AddToCart conversion with value and product details. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Cool Cap", "category": "hats", "price": "42.00", "quantity": 1 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-456" }, "nested": [], "consent": { "functional": true }, "id": "ev-1700000901", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000901, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "AddToCart", "data": { "map": { "event_metadata": { "map": { "value_decimal": "data.price", "currency": { "value": "EUR" }, "item_count": { "value": 1 }, "products": { "set": [ { "map": { "id": "data.id", "name": "data.name", "category": { "key": "data.category", "value": "uncategorized" } } } ] } } } } } } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.901Z\",\"event_at_ms\":1700000901,\"event_type\":{\"tracking_type\":\"AddToCart\"},\"user\":{},\"event_metadata\":{\"conversion_id\":\"ev-1700000901\",\"value_decimal\":\"42.00\",\"currency\":\"EUR\",\"item_count\":1,\"products\":[{\"id\":\"SKU-B2\",\"name\":\"Cool Cap\",\"category\":\"hats\"}]}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Lead A form submission is sent to Reddit as a Lead conversion with the SHA-256 hashed email and external id. Event ``` { "name": "form submit", "data": { "form": "contact" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "user-lead-1", "email": "lead@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000903", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000903, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "Lead", "data": { "map": { "user": { "map": { "email": "user.email", "external_id": "user.id" } } } } } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.903Z\",\"event_at_ms\":1700000903,\"event_type\":{\"tracking_type\":\"Lead\"},\"user\":{\"email\":\"9fbdefe2837a03c9225be80e741f316f4d174d1732b719b6abb6477efc1ae9d2\",\"external_id\":\"ee818eebb052cf288ffeeb2e09ee35c9946e1a7f53a959cb3ef06d5d4adb78e8\"},\"event_metadata\":{\"conversion_id\":\"ev-1700000903\"}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Page visit A page view is sent to Reddit as a PageVisit conversion used for retargeting audiences. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "user-789" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000902", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000902, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "PageVisit" } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.902Z\",\"event_at_ms\":1700000902,\"event_type\":{\"tracking_type\":\"PageVisit\"},\"user\":{},\"event_metadata\":{\"conversion_id\":\"ev-1700000902\"}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Purchase A completed order is sent to the Reddit Conversions API as a Purchase event with value, currency, and items. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Everyday Ruck Snack", "category": "bags", "price": "129.99", "quantity": 2 } } ], "consent": { "functional": true }, "id": "ev-1700000900", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "Purchase", "data": { "map": { "event_metadata": { "map": { "value_decimal": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "item_count": { "fn": { "$code": "e=>e.nested.filter(e=>\"product\"===e.entity).length" } }, "products": { "loop": [ "nested", { "condition": { "$code": "e=>h(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "name": "data.name", "category": { "key": "data.category", "value": "uncategorized" } } } ] } } } } } } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.900Z\",\"event_at_ms\":1700000900,\"event_type\":{\"tracking_type\":\"Purchase\"},\"user\":{},\"event_metadata\":{\"conversion_id\":\"ev-1700000900\",\"value_decimal\":249.99,\"currency\":\"EUR\",\"item_count\":1,\"products\":[{\"id\":\"SKU-A1\",\"name\":\"Everyday Ruck Snack\",\"category\":\"bags\"}]}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Search A site search is sent to Reddit as a Search conversion with an item count in event\_metadata. Event ``` { "name": "site search", "data": { "query": "walkerOS destinations" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "user-101" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000905", "trigger": "test", "entity": "site", "action": "search", "timestamp": 1700000905, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "Search", "data": { "map": { "event_metadata": { "map": { "item_count": { "value": 1 } } } } } } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.905Z\",\"event_at_ms\":1700000905,\"event_type\":{\"tracking_type\":\"Search\"},\"user\":{},\"event_metadata\":{\"conversion_id\":\"ev-1700000905\",\"item_count\":1}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ### Sign up A user signup is sent to Reddit as a SignUp conversion with hashed user identifiers. Event ``` { "name": "user signup", "data": { "method": "email" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "new-user-1", "email": "new@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000904", "trigger": "test", "entity": "user", "action": "signup", "timestamp": 1700000904, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "name": "SignUp", "data": { "map": { "user": { "map": { "email": "user.email", "external_id": "user.id" } } } } } ``` Out ``` sendServer("https://ads-api.reddit.com/api/v2.0/conversions/events/a2_abcdef123456", "{\"data\":{\"events\":[{\"event_at\":\"1970-01-20T16:13:20.904Z\",\"event_at_ms\":1700000904,\"event_type\":{\"tracking_type\":\"SignUp\"},\"user\":{\"email\":\"f0030501023327437b06e5c6f87df7871b8e704ae608d1d0b7b24fdd2a06c716\",\"external_id\":\"b45cf5f6ebc2c6974ea3bd9fab19f8cc3a7cf63054727a9fcd22f1fda97d6dde\"},\"event_metadata\":{\"conversion_id\":\"ev-1700000904\"}}]}}", { "headers": { "Authorization": "Bearer s3cr3t" } }) ``` ## Event mapping[​](#event-mapping "Direct link to Event mapping") Reddit uses a **rigid taxonomy**. Map walkerOS events to a standard `tracking_type` via each rule's `name` field: | walkerOS Event | Reddit `tracking_type` | | ------------------ | ---------------------- | | `page view` | `PageVisit` | | `product view` | `ViewContent` | | `site search` | `Search` | | `product add` | `AddToCart` | | `product wishlist` | `AddToWishlist` | | `order complete` | `Purchase` | | `form submit` | `Lead` | | `user signup` | `SignUp` | Any non-standard name becomes `{ tracking_type: 'Custom', custom_event_name: '' }` automatically. Reddit's AI optimization is trained on the standard events, so prefer them when possible. ``` "mapping": { "order": { "complete": { "name": "Purchase", "data": { "map": { "event_metadata": { "map": { "value_decimal": "data.total", "currency": "data.currency", "item_count": { "value": 1 } } } } } } } } ``` ### event\_metadata structure[​](#event_metadata-structure "Direct link to event_metadata structure") Unlike Meta's CAPI (which uses `custom_data`) and Pinterest (which also uses `custom_data`), Reddit places conversion details under `event_metadata`. Always map into `event_metadata.map`. Products belong in `event_metadata.products[]` with `{ id, name?, category }`. ### Payload envelope[​](#payload-envelope "Direct link to Payload envelope") Reddit wraps events inside `{ data: { events: [...] } }` (nested), unlike Meta/Pinterest's flat `{ data: [...] }`. The destination handles this automatically. ## Hashing[​](#hashing "Direct link to Hashing") The destination automatically SHA-256 hashes the 6 standard identity fields before sending: `email`, `external_id`, `ip_address`, `user_agent`, `idfa`, `aaid`. Pass raw values and hashing is handled for you. Different from Meta/Pinterest Reddit hashes `ip_address` and `user_agent`, which Meta CAPI and Pinterest CAPI pass through as plaintext. Be sure to pass the raw values, not pre-hashed ones, unless you use `doNotHash`. Non-hashable fields pass through as-is: `uuid`, `opt_out`, `screen_dimensions`, `data_processing_options`. If a value is already hashed or you want to skip hashing for a specific field, use `doNotHash`: ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID", "doNotHash": ["email"] } ``` ## Deduplication[​](#deduplication "Direct link to Deduplication") Each event is sent with `event_metadata.conversion_id` set to the walkerOS event `id`. If you also run the Reddit Pixel in the browser, Reddit deduplicates server and browser events that share the same `conversion_id`, so conversions are not double-counted. ## Test mode[​](#test-mode "Direct link to Test mode") Set `test_mode: true` in settings to send `"test_mode": true` as a **top-level boolean in the request body** (not a query parameter, unlike Pinterest). Useful during development. ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID", "test_mode": true } ``` ## Links[​](#links "Direct link to Links") * [Reddit Conversions API documentation](https://ads-api.reddit.com/docs/v2/#tag/Conversions-API) * [Reddit Ads Manager](https://ads.reddit.com/) --- # Redis Streams [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/redis) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/redis)[ ](https://www.npmjs.com/package/@walkeros/server-destination-redis) [Package](https://www.npmjs.com/package/@walkeros/server-destination-redis) Beta Server-side event streaming to [Redis Streams](https://redis.io/docs/latest/develop/data-types/streams/) via the [`ioredis`](https://github.com/redis/ioredis) client. Each event is appended to a configurable stream via `XADD`, with optional approximate MAXLEN trimming for bounded memory usage, JSON or flat serialization modes, and graceful shutdown via `destroy()`. Where this fits Redis Streams is a **server destination** in the walkerOS flow: Receives events server-side from the collector, serializes them as JSON (or flat fields), and appends them to a Redis Stream for downstream consumers (stream processors, worker queues, pub/sub fan-out, XREAD consumer groups). ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-redis ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationRedis } from '@walkeros/server-destination-redis'; await startFlow({ destinations: { redis: { code: destinationRedis, config: { settings: { redis: { streamKey: 'walkeros:events', url: 'redis://localhost:6379', }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "redis": { "package": "@walkeros/server-destination-redis", "config": { "settings": { "redis": { "streamKey": "walkeros:events", "url": "redis://localhost:6379" } } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `redis*` | `redis` | Redis Streams configuration | | | `streamKey*` | `string` | Redis stream key name (like 'walkeros:events'). All events are appended to this stream via XADD. | | | `url` | `string` | Redis connection URL (like 'redis\://localhost:6379' or 'rediss\://:password\@host:6380'). Supports redis\:// and rediss\:// (TLS) protocols. | | | `options` | `Record` | ioredis connection options. Used when url is not provided. Supports host, port, password, db, tls, and all other ioredis options. | | | `maxLen` | `integer` | Maximum stream length. Enables approximate MAXLEN trimming on every XADD to bound memory usage (like 50000). | | | `exactTrimming` | `boolean` | Use exact MAXLEN instead of approximate (\~). Not recommended for production. Default: false. | | | `serialization` | `'json' \| 'flat'` | Serialization mode. 'json' stores the full event as a single 'event' field (default). 'flat' stores top-level event fields as separate stream entry fields. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ----------- | -------- | ---------------------------------------------------------------------------------------- | ---- | | `streamKey` | `string` | Override Redis stream key for this rule. Takes precedence over settings.redis.streamKey. | | ## Examples ### Default stream An event is appended to the configured Redis stream via XADD with the full event JSON as a single field. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "1caf043a9a6bc2ae", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.xadd([ "walkeros:events", "*", "event", "json:event" ]) ``` ### Order event An order complete event is appended to the Redis stream alongside other event types for downstream consumers. Event ``` { "name": "order complete", "data": { "id": "ORD-400", "total": 99.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "199fb241692edaad", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.xadd([ "walkeros:events", "*", "event", "json:event" ]) ``` ### Stream key override A mapping rule routes the event to a dedicated Redis stream instead of the destination default. Event ``` { "name": "order complete", "data": { "id": "ORD-500", "total": 42 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "fa49b8680079010f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "streamKey": "walkeros:orders" } } ``` Out ``` client.xadd([ "walkeros:orders", "*", "event", "json:event" ]) ``` ### Exact trim XADD uses exact MAXLEN trimming to enforce a precise Redis stream length at the cost of extra work. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "5eaaa36623def740", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.xadd([ "walkeros:events", "MAXLEN", 5000, "*", "event", "json:event" ]) ``` ### MAXLEN trim XADD uses approximate MAXLEN trimming to cap the Redis stream length, discarding older entries efficiently. Event ``` { "name": "product view", "data": { "id": "SKU-123", "name": "Widget" }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "2afed5ee97c575bf", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.xadd([ "walkeros:events", "MAXLEN", "~", 50000, "*", "event", "json:event" ]) ``` The destination creates a single long-lived `ioredis` client during `init()`. On flow hot-swap or server shutdown, `destroy()` calls `client.quit()` to flush in-flight commands and close the TCP connection gracefully. User-provided clients (wired in via `_client`) are left untouched. ## Stream entry format[​](#stream-entry-format "Direct link to Stream entry format") Each event is appended via `XADD [MAXLEN [~] ] * ...`: * **streamKey**: from `settings.redis.streamKey` (or `mapping.settings.streamKey` override) * **`*`**: auto-generated entry ID (ms-timestamp + sequence) * **fields**: in `json` mode, a single `event` field with the full event JSON-stringified; in `flat` mode, top-level event keys as separate fields (nested objects JSON-encoded) * **MAXLEN**: when `settings.redis.maxLen` is set, trims the stream to \~N entries. Approximate (`~`) by default; set `exactTrimming: true` to trim exactly (slower) Use `mapping.settings.streamKey` to route specific events to dedicated streams (e.g. orders to `walkeros:orders`, identities to `walkeros:identities`). ## Serialization modes[​](#serialization-modes "Direct link to Serialization modes") ### JSON (default)[​](#json-default "Direct link to JSON (default)") Stores the full walkerOS event as a single `event` field. Easiest to consume, decode with `JSON.parse()`. ``` XADD walkeros:events * event '{"entity":"page","action":"view",...}' ``` ### Flat[​](#flat "Direct link to Flat") Stores top-level event fields as separate stream entry fields. Nested objects are JSON-encoded. Useful when downstream consumers want to filter or project specific fields without parsing the full event. ``` XADD walkeros:events * entity page action view timestamp 1700000100 data '{"id":"Home"}' ``` Configure via `settings.redis.serialization: 'flat'`. ## Authentication[​](#authentication "Direct link to Authentication") ### Redis Cloud / Upstash (TLS + password)[​](#redis-cloud--upstash-tls--password "Direct link to Redis Cloud / Upstash (TLS + password)") ``` { "redis": { "streamKey": "walkeros:events", "url": "rediss://default:$env.REDIS_PASSWORD@my-endpoint.upstash.io:6380" } } ``` ### AWS ElastiCache (in-VPC, no auth)[​](#aws-elasticache-in-vpc-no-auth "Direct link to AWS ElastiCache (in-VPC, no auth)") ``` { "redis": { "streamKey": "walkeros:events", "url": "redis://my-cluster.cache.amazonaws.com:6379" } } ``` ### Custom options (ACL user, non-default db)[​](#custom-options-acl-user-non-default-db "Direct link to Custom options (ACL user, non-default db)") ``` { "redis": { "streamKey": "walkeros:events", "options": { "host": "redis.example.com", "port": 6379, "username": "$env.REDIS_USER", "password": "$env.REDIS_PASSWORD", "db": 1, "tls": {} } } } ``` ## MAXLEN trimming[​](#maxlen-trimming "Direct link to MAXLEN trimming") Redis Streams grow unbounded by default. For high-volume pipelines, cap stream length with `maxLen`: ``` { "redis": { "streamKey": "walkeros:events", "url": "redis://localhost:6379", "maxLen": 50000 } } ``` Approximate trimming (`~`) is used by default. Redis trims to "about N" entries with minimal overhead. For exact trimming, set `exactTrimming: true` (slower, use only when strict bounds are required). --- # RudderStack [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/rudderstack) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/rudderstack)[ ](https://www.npmjs.com/package/@walkeros/server-destination-rudderstack) [Package](https://www.npmjs.com/package/@walkeros/server-destination-rudderstack) Beta Server-side event delivery to [RudderStack](https://rudderstack.com/) via the official [`@rudderstack/rudder-sdk-node`](https://www.npmjs.com/package/@rudderstack/rudder-sdk-node) SDK. Implements the full Segment Spec surface (`track`, `identify`, `group`, `page`, `screen`, and `alias`) with automatic identity deduplication and graceful shutdown via `flush()`. Where this fits RudderStack is a **server destination** in the walkerOS flow: Receives events server-side from the collector, resolves identity, and forwards them to RudderStack's data plane for routing to downstream destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-rudderstack ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationRudderStack } from '@walkeros/server-destination-rudderstack'; await startFlow({ destinations: { rudderstack: { code: destinationRudderStack, config: { settings: { writeKey: 'YOUR_RUDDERSTACK_WRITE_KEY', dataPlaneUrl: 'https://your-data-plane.rudderstack.com', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "rudderstack": { "package": "@walkeros/server-destination-rudderstack", "config": { "settings": { "writeKey": "YOUR_RUDDERSTACK_WRITE_KEY", "dataPlaneUrl": "https://your-data-plane.rudderstack.com" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `writeKey*` | `string` | RudderStack source write key. Find it in the RudderStack dashboard under Sources > Setup. | | | `dataPlaneUrl*` | `string` | RudderStack data plane URL. Required. Example: https\://your-data-plane.rudderstack.com | | | `userId` | `string` | walkerOS mapping value path to resolve userId from each event (like user.id). | | | `anonymousId` | `string` | walkerOS mapping value path to resolve anonymousId from each event (like user.session). | | | `path` | `string` | API path route (like /v1/batch). | | | `flushAt` | `integer` | Events to enqueue before flushing a batch. Default: 20. | | | `flushInterval` | `integer` | Max milliseconds before auto-flush. Default: 10000. | | | `maxQueueSize` | `integer` | Maximum batch payload size in bytes. Default: 460800 (\~500KB). | | | `maxInternalQueueSize` | `integer` | Maximum in-memory queue length. Default: 20000. | | | `logLevel` | `string` | SDK log level: 'info', 'debug', 'error', or 'silly'. Default: 'info'. | | | `retryCount` | `integer` | Retry attempts for failed batches. Default: 3. | | | `enable` | `boolean` | Set to false to no-op all SDK calls. Default: true. | | | `gzip` | `boolean` | Enable gzip compression for requests. Default: true. | | | `identify` | `any` | Destination-level identity mapping. Resolves to { traits } object. Fires identify() on the first push and re-fires when values change. | | | `group` | `any` | Destination-level group mapping. Resolves to { groupId, traits }. Fires group() on the first push and re-fires on change. | | | `integrations` | `Record` | Enable/disable downstream RudderStack destinations. Example: { "All": true, "Mixpanel": false }. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { userId?, traits? }. Use with rule-level silent: true on login/identify events. | | | `group` | `any` | Per-event group assignment. Resolves to { groupId, traits? }. Use with rule-level silent: true on company/team events. | | | `page` | `any` | Per-event page call. Resolves to { name, properties? } or true for minimal page(). RudderStack requires name (falls back to empty string). Use with silent: true. | | | `screen` | `any` | Per-event screen call (mobile backends). Resolves to { name, properties? } or true for minimal screen(). RudderStack requires name (falls back to empty string). Use with silent: true. | | | `alias` | `any` | Per-event alias call. Resolves to { previousId }. Links anonymous user to identified user. Use with rule-level silent: true. | | ## Examples ### Alias user An identity merge fires analytics.alias to link a previous anonymous id to the newly registered user id. Event ``` { "name": "identity merge", "data": { "anonymous_id": "anonymous-123" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "registered-456", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0b3b4d607de27080", "trigger": "test", "entity": "identity", "action": "merge", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "alias": { "map": { "previousId": "data.anonymous_id" } } } } ``` Out ``` analytics.alias({ "userId": "registered-456", "previousId": "anonymous-123", "timestamp": {} }) ``` ### Anonymous only When no userId is resolved RudderStack accepts the track call keyed solely by anonymousId. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "ddabb930e99c87f2", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.track({ "anonymousId": "s3ss10n", "event": "product view", "properties": {}, "timestamp": {} }) ``` ### Group company A company update fires analytics.group with the groupId and group traits for B2B account tracking. Event ``` { "name": "company update", "data": { "company_id": "comp-456", "company_name": "Acme", "industry": "tech", "employees": 50 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "2d2d6c51e6ff41a6", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "group": { "map": { "groupId": "data.company_id", "traits": { "map": { "name": "data.company_name", "industry": "data.industry", "employees": "data.employees" } } } } } } ``` Out ``` analytics.group({ "userId": "us3r", "anonymousId": "s3ss10n", "groupId": "comp-456", "traits": { "name": "Acme", "industry": "tech", "employees": 50 }, "timestamp": {} }) ``` ### Default track A walker event becomes a RudderStack analytics.track call with userId and anonymousId resolved from the event. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "3f77a089d33deac0", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "product view", "properties": {}, "timestamp": {} }) ``` ### Destination identify Destination-level identify fires analytics.identify with traits on the first push only, then the track call follows. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n", "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "45f74a418bf42e54", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.identify({ "userId": "us3r", "anonymousId": "s3ss10n", "traits": { "email": "user@example.com" }, "timestamp": {} }); analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "page view", "properties": {}, "timestamp": {} }) ``` ### Renamed event A mapping renames the event so the RudderStack track call uses a canonical name such as Order Completed. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "630449dd28d82a67", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Order Completed" } ``` Out ``` analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "Order Completed", "properties": {}, "timestamp": {} }) ``` ### Page view A page view fires analytics.page with a resolved page name and properties instead of a generic track. Event ``` { "name": "page view", "data": { "title": "Getting Started", "section": "tutorials" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "b31cb71946586cdc", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "page": { "map": { "name": "data.title", "properties": { "map": { "section": "data.section" } } } } } } ``` Out ``` analytics.page({ "userId": "us3r", "anonymousId": "s3ss10n", "name": "Getting Started", "properties": { "section": "tutorials" }, "timestamp": {} }) ``` ### Screen view A screen view from a mobile app backend fires analytics.screen with a screen name and properties. Event ``` { "name": "screen view", "data": { "screen_name": "Welcome", "section": "onboarding", "build": "1.2.3" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0f089c989d368173", "trigger": "test", "entity": "screen", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "screen": { "map": { "name": "data.screen_name", "properties": { "map": { "build": "data.build" } } } } } } ``` Out ``` analytics.screen({ "userId": "us3r", "anonymousId": "s3ss10n", "name": "Welcome", "properties": { "build": "1.2.3" }, "timestamp": {} }) ``` ### User login identify A user login fires only an analytics.identify call with the resolved userId and traits, skipping the track. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "email": "user@acme.com", "name": "Jane Doe", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d1b0b93596fbe05f", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "userId": "data.user_id", "traits": { "map": { "email": "data.email", "name": "data.name", "plan": "data.plan" } } } } } } ``` Out ``` analytics.identify({ "userId": "new-user-123", "anonymousId": "s3ss10n", "traits": { "email": "user@acme.com", "name": "Jane Doe", "plan": "premium" }, "timestamp": {} }) ``` Identity is resolved automatically from each event: `userId` defaults to `user.id` and `anonymousId` defaults to `user.session`. At least one must be present for a call to be sent. Override these paths in settings to match your event structure. ## Alias support[​](#alias-support "Direct link to Alias support") Unlike Segment, RudderStack also supports per-event `alias()` calls, useful for linking an anonymous identity to an identified user after signup or login. Configure via `mapping.settings.alias` with a `previousId` field. --- # Segment [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/segment) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/segment)[ ](https://www.npmjs.com/package/@walkeros/server-destination-segment) [Package](https://www.npmjs.com/package/@walkeros/server-destination-segment) Server-side event delivery to [Segment](https://segment.com/) via the official [`@segment/analytics-node`](https://www.npmjs.com/package/@segment/analytics-node) SDK. Implements the full Segment Spec surface (`track`, `identify`, `group`, `page`, and `screen`) with automatic identity deduplication and graceful shutdown via `closeAndFlush()`. Where this fits Segment is a **server destination** in the walkerOS flow: Receives events server-side from the collector, resolves identity and consent context, and forwards them to Segment for routing to 400+ downstream destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-segment ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationSegment } from '@walkeros/server-destination-segment'; await startFlow({ destinations: { segment: { code: destinationSegment, config: { settings: { writeKey: 'YOUR_SEGMENT_WRITE_KEY', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "segment": { "package": "@walkeros/server-destination-segment", "config": { "settings": { "writeKey": "YOUR_SEGMENT_WRITE_KEY" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `writeKey*` | `string` | Segment source write key. Find it in your Segment workspace under Connections > Sources > Settings > API Keys. | | | `userId` | `string` | walkerOS mapping value path to resolve userId from each event (like user.id). | | | `anonymousId` | `string` | walkerOS mapping value path to resolve anonymousId from each event (like user.session). | | | `host` | `string` | Base URL of Segment API. Set to https\://events.eu1.segmentapis.com for EU endpoint. | | | `path` | `string` | API path route (like /v1/batch). | | | `flushAt` | `integer` | Events to enqueue before flushing a batch. Default: 15. | | | `flushInterval` | `integer` | Max milliseconds before auto-flush. Default: 10000. | | | `maxRetries` | `integer` | Retry attempts for failed batches. Default: 3. | | | `httpRequestTimeout` | `integer` | HTTP request timeout in milliseconds. Default: 10000. | | | `disable` | `boolean` | Completely disable the SDK (no-ops all calls). Default: false. | | | `identify` | `any` | Destination-level identity mapping. Resolves to { traits } object. Fires identify() on the first push and re-fires when values change. | | | `group` | `any` | Destination-level group mapping. Resolves to { groupId, traits }. Fires group() on the first push and re-fires on change. | | | `consent` | `Record` | Mapping from walkerOS consent keys to Segment categoryPreferences keys. Example: { "marketing": "Advertising", "analytics": "Analytics" }. | | | `integrations` | `Record` | Enable/disable downstream Segment destinations. Example: { "All": true, "Mixpanel": false }. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { userId?, traits? }. Use with rule-level silent: true on login/identify events. | | | `group` | `any` | Per-event group assignment. Resolves to { groupId, traits? }. Use with rule-level silent: true on company/team events. | | | `page` | `any` | Per-event page call. Resolves to { category?, name?, properties? } or true for minimal page(). Use with silent: true. | | | `screen` | `any` | Per-event screen call (mobile backends). Resolves to { category?, name?, properties? } or true for minimal screen(). Use with silent: true. | | ## Examples ### Anonymous only When no userId is resolved Segment accepts a track call keyed solely by anonymousId. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "91468b9d49ff9d16", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.track({ "anonymousId": "s3ss10n", "event": "product view", "properties": {}, "timestamp": {} }) ``` ### Group company A company update fires Segment analytics.group with groupId and group traits for account-level tracking. Event ``` { "name": "company update", "data": { "company_id": "comp-456", "company_name": "Acme", "industry": "tech", "employees": 50 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d96edcd8da49c2d0", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "group": { "map": { "groupId": "data.company_id", "traits": { "map": { "name": "data.company_name", "industry": "data.industry", "employees": "data.employees" } } } } } } ``` Out ``` analytics.group({ "userId": "us3r", "anonymousId": "s3ss10n", "groupId": "comp-456", "traits": { "name": "Acme", "industry": "tech", "employees": 50 }, "timestamp": {} }) ``` ### Consent forwarding Walker consent keys are mapped to Segment categoryPreferences on the analytics context for downstream filtering. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [], "consent": { "analytics": true, "marketing": true }, "id": "052f1eb1078922b3", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "product view", "properties": {}, "timestamp": {}, "context": { "consent": { "categoryPreferences": { "Analytics": true, "Advertising": true } } } }) ``` ### Default track A walker event becomes a Segment analytics.track call with userId and anonymousId resolved from the event. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "6618dec05a6397c0", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "product view", "properties": {}, "timestamp": {} }) ``` ### Destination identify Destination-level identify fires Segment analytics.identify with traits on the first push only, then the track call follows. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n", "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "dd644f721ab9ce1e", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` analytics.identify({ "userId": "us3r", "anonymousId": "s3ss10n", "traits": { "email": "user@example.com" }, "timestamp": {} }); analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "page view", "properties": {}, "timestamp": {} }) ``` ### Renamed event A mapping renames the event so the Segment track call uses Segment's canonical 'Order Completed' name. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "412e41a64054f660", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Order Completed" } ``` Out ``` analytics.track({ "userId": "us3r", "anonymousId": "s3ss10n", "event": "Order Completed", "properties": {}, "timestamp": {} }) ``` ### Page view A page view fires Segment analytics.page with category, name, and properties instead of a generic track. Event ``` { "name": "page view", "data": { "category": "docs", "title": "Getting Started", "section": "tutorials" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9c86e4792a2dccee", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "page": { "map": { "category": "data.category", "name": "data.title", "properties": { "map": { "section": "data.section" } } } } } } ``` Out ``` analytics.page({ "userId": "us3r", "anonymousId": "s3ss10n", "category": "docs", "name": "Getting Started", "properties": { "section": "tutorials" }, "timestamp": {} }) ``` ### Screen view A screen view fires Segment analytics.screen with name, category, and properties for mobile app tracking. Event ``` { "name": "screen view", "data": { "screen_name": "Welcome", "section": "onboarding", "build": "1.2.3" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ca35e4d3a794494e", "trigger": "test", "entity": "screen", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "screen": { "map": { "name": "data.screen_name", "category": "data.section", "properties": { "map": { "build": "data.build" } } } } } } ``` Out ``` analytics.screen({ "userId": "us3r", "anonymousId": "s3ss10n", "name": "Welcome", "category": "onboarding", "properties": { "build": "1.2.3" }, "timestamp": {} }) ``` ### User login identify A user login fires only Segment analytics.identify with the userId and traits, skipping the track. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "email": "user@acme.com", "name": "Jane Doe", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "b3e1b15d4324fa5b", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "userId": "data.user_id", "traits": { "map": { "email": "data.email", "name": "data.name", "plan": "data.plan" } } } } } } ``` Out ``` analytics.identify({ "userId": "new-user-123", "anonymousId": "s3ss10n", "traits": { "email": "user@acme.com", "name": "Jane Doe", "plan": "premium" }, "timestamp": {} }) ``` Identity is resolved automatically from each event: `userId` defaults to `user.id` and `anonymousId` defaults to `user.session`. At least one must be present for a call to be sent. Override these paths in settings to match your event structure. --- # Slack [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/slack) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/slack)[ ](https://www.npmjs.com/package/@walkeros/server-destination-slack) [Package](https://www.npmjs.com/package/@walkeros/server-destination-slack) Send walkerOS events to Slack as formatted messages. Supports both Incoming Webhooks (simple, single-channel) and the Web API (multi-channel routing, threading, DMs, ephemeral messages, Block Kit). Where this fits Slack is a **server destination** in the walkerOS flow: Turns events into Slack messages for real-time alerts, sales notifications, moderation queues, or incident routing, without coupling your product code to Slack. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-slack ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationSlack } from '@walkeros/server-destination-slack'; await startFlow({ destinations: { slack: { code: destinationSlack, config: { settings: { token: 'xoxb-your-bot-token', channel: '#notifications', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "slack": { "package": "@walkeros/server-destination-slack", "import": "destinationSlack", "config": { "settings": { "token": "xoxb-your-bot-token", "channel": "#notifications" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `token` | `string` | Slack Bot token (xoxb-...). Enables Web API mode. Mutually exclusive with webhookUrl. | | | `webhookUrl` | `string` | Incoming Webhook URL. Enables webhook mode. Mutually exclusive with token. | | | `channel` | `string` | Default Slack channel ID or name (e.g. "#alerts" or "C024BE91L"). Required for Web API mode unless every rule supplies one. Ignored in webhook mode. | | | `text` | `string` | Default text template. Supports \`${data.field}\` interpolation against the walkerOS event. | | | `blocks` | `Array` | Default Block Kit blocks applied when no mapping override is set. | | | `includeHeader` | `boolean` | Auto-add an event-name header block when generating default blocks. Default: true. | | | `unfurlLinks` | `boolean` | Enable link unfurling. Default: false (cleaner for automated alerts). | | | `unfurlMedia` | `boolean` | Enable media unfurling. Default: false. | | | `mrkdwn` | `boolean` | Use mrkdwn formatting in text. Default: true. | | | `threadTs` | `string` | Static thread\_ts for replies (rarely set at destination level). | | | `retryConfig` | `'default' \| 'fiveRetriesInFive` | Retry policy passed to WebClient. Default: "default". | | ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------------- | --------------- | --------------------------------------------------------------------------------------------- | ---- | | `channel` | `string` | Override the destination channel for this rule. Web API mode only -- ignored in webhook mode. | | | `text` | `string` | Override the text template for this rule. Supports \`${data.field}\` interpolation. | | | `blocks` | `Array` | Override Block Kit blocks for this rule. | | | `threadTs` | `string` | thread\_ts for posting as a reply in a thread. | | | `replyBroadcast` | `boolean` | Also broadcast the threaded reply back to the channel. | | | `ephemeral` | `boolean` | Send via chat.postEphemeral. Requires \`user\`. | | | `user` | `string` | Slack user ID for ephemeral or DM delivery. | | | `dm` | `boolean` | Send as DM via conversations.open + chat.postMessage. Requires \`user\`. | | ## Examples ### Default blocks With no custom text the destination auto-generates a Block Kit message from event data and source. Event ``` { "name": "lead submit", "data": { "name": "Acme", "email": "sales@acme.test" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d907f522b5e2426e", "trigger": "test", "entity": "lead", "action": "submit", "timestamp": 1700000600, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "crm", "platform": "server" } } ``` Mapping ``` { "settings": { "channel": "#growth" } } ``` Out ``` slackClient.chat.postMessage({ "channel": "#growth", "text": "lead submit", "blocks": [ { "type": "header", "text": { "type": "plain_text", "text": "lead submit" } }, { "type": "section", "fields": [ { "type": "mrkdwn", "text": "*name:*\nAcme" }, { "type": "mrkdwn", "text": "*email:*\nsales@acme.test" } ] }, { "type": "context", "elements": [ { "type": "mrkdwn", "text": "Source: crm" } ] } ], "unfurl_links": false, "unfurl_media": false, "mrkdwn": true }) ``` ### Webhook deploy Without a token the destination posts to an incoming Slack webhook URL with the rendered message body. Event ``` { "name": "deploy complete", "data": { "version": "1.4.2", "environment": "prod" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "fc36c6cc2f22392d", "trigger": "test", "entity": "deploy", "action": "complete", "timestamp": 1700000700, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "text": ":rocket: Deployment complete: ${data.version} to ${data.environment}" } } ``` Out ``` sendServer("https://hooks.slack.com/services/T00/B00/xxx", "{\"text\":\":rocket: Deployment complete: 1.4.2 to prod\",\"unfurl_links\":false,\"unfurl_media\":false,\"mrkdwn\":true}") ``` ### Ephemeral message A quota warning posts an ephemeral Slack message visible only to a target admin user. Event ``` { "name": "quota warning", "data": { "remaining": 5 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d36432703e0e17cf", "trigger": "test", "entity": "quota", "action": "warning", "timestamp": 1700000500, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "channel": "#admin", "ephemeral": true, "user": "U-ADMIN-1", "text": "Heads up: ${data.remaining} requests remaining" } } ``` Out ``` slackClient.chat.postEphemeral({ "channel": "#admin", "text": "Heads up: 5 requests remaining", "unfurl_links": false, "unfurl_media": false, "mrkdwn": true, "user": "U-ADMIN-1" }) ``` ### Error alert An error event posts a critical alert to an engineering channel using a mapping-level channel override. Event ``` { "name": "error occur", "data": { "message": "Payment gateway timeout", "code": "PGW_TIMEOUT", "severity": "critical" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "c450b931cc3d58e5", "trigger": "test", "entity": "error", "action": "occur", "timestamp": 1700000200, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "channel": "#engineering-alerts", "text": ":rotating_light: Error: ${data.message}" } } ``` Out ``` slackClient.chat.postMessage({ "channel": "#engineering-alerts", "text": ":rotating_light: Error: Payment gateway timeout", "unfurl_links": false, "unfurl_media": false, "mrkdwn": true }) ``` ### Purchase alert A completed order posts a templated message to a sales channel via the Slack Web API. Event ``` { "name": "order complete", "data": { "id": "ORD-500", "total": 299.99, "currency": "EUR", "product": "Pro Plan" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "buyer-42" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "d8ac6f506ad055bd", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "channel": "#sales", "text": ":moneybag: New order: ${data.id} - ${data.total} ${data.currency}" } } ``` Out ``` slackClient.chat.postMessage({ "channel": "#sales", "text": ":moneybag: New order: ORD-500 - 299.99 EUR", "unfurl_links": false, "unfurl_media": false, "mrkdwn": true }) ``` ### Threaded reply A checkout step posts as a threaded reply in Slack via thread\_ts with broadcast to the channel. Event ``` { "name": "checkout step", "data": { "step": "payment", "sessionTs": "1700000000.000050" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "11d9b3f7168f9e0a", "trigger": "test", "entity": "checkout", "action": "step", "timestamp": 1700000400, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "channel": "#sales", "text": "Checkout step: ${data.step}", "threadTs": "1700000000.000050", "replyBroadcast": true } } ``` Out ``` slackClient.chat.postMessage({ "channel": "#sales", "text": "Checkout step: payment", "unfurl_links": false, "unfurl_media": false, "mrkdwn": true, "thread_ts": "1700000000.000050", "reply_broadcast": true }) ``` ### Welcome DM A user signup opens a Slack DM channel and posts a welcome message with the selected plan. Event ``` { "name": "user signup", "data": { "plan": "enterprise" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "U-NEW-USER" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "627bf54372963b7f", "trigger": "test", "entity": "user", "action": "signup", "timestamp": 1700000300, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "dm": true, "user": "U-NEW-USER", "text": ":wave: Welcome aboard! You signed up for the ${data.plan} plan." } } ``` Out ``` slackClient.conversations.open({ "users": "U-NEW-USER" }); slackClient.chat.postMessage({ "channel": "D-MOCK-DM", "text": ":wave: Welcome aboard! You signed up for the enterprise plan.", "unfurl_links": false, "unfurl_media": false, "mrkdwn": true }) ``` ## Webhook vs Web API modes[​](#webhook-vs-web-api-modes "Direct link to Webhook vs Web API modes") Pick exactly one auth method per destination. Each mode trades capability for simplicity. | Capability | Incoming Webhook (`webhookUrl`) | Web API (`token`) | | ------------------ | ------------------------------- | ---------------------------------------------- | | Setup | Paste a URL, done | Create a Slack app, install bot, manage scopes | | Channels | Fixed to the webhook's channel | Any channel the bot is in, routable per rule | | Threading | Not supported | `threadTs` + `replyBroadcast` | | Direct messages | Not supported | `dm: true` + `user` | | Ephemeral messages | Not supported | `ephemeral: true` | | Retries | Plain HTTP | SDK-managed (`retryConfig`) | | Block Kit | Yes | Yes | Use **webhooks** for a single firehose channel and minimal setup. Use the **Web API** whenever you need routing, threading, DMs, or ephemeral messages. ## Channel routing[​](#channel-routing "Direct link to Channel routing") With Web API mode, route events to different channels using nested `entity > action` mapping. Each rule can override `channel`, `text`, `blocks`, and threading behavior. ``` "destinations": { "slack": { "package": "@walkeros/server-destination-slack", "import": "destinationSlack", "config": { "settings": { "token": "$env.SLACK_BOT_TOKEN", "channel": "#events" }, "mapping": { "order": { "complete": { "settings": { "channel": "#sales", "text": "New order ${data.id}: ${data.total} ${data.currency}" } } }, "error": { "report": { "settings": { "channel": "#incidents", "threadTs": "data.incidentTs" } } }, "user": { "signup": { "settings": { "channel": "#growth" } } } } } } } ``` Events that don't match any rule fall back to the default `channel` in `settings`. ## Message formatting[​](#message-formatting "Direct link to Message formatting") Messages support two formatting styles, set at destination level or per rule: * **Text templates**: `${data.field}` placeholders interpolate against the walkerOS event. Good for terse alerts. * **Block Kit**: structured layouts via Slack's [Block Kit](https://api.slack.com/block-kit) (headers, sections, buttons, context). Use the `blocks` setting when you need rich formatting. When neither `text` nor `blocks` is provided, the destination generates a default message with an event-name header (toggle via `includeHeader`). --- # Snapchat Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/snapchat) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/snapchat)[ ](https://www.npmjs.com/package/@walkeros/server-destination-snapchat) [Package](https://www.npmjs.com/package/@walkeros/server-destination-snapchat) Beta Server-side event delivery to Snapchat's Conversions API v3 for enhanced conversion tracking, bypassing browser limitations and improving match quality for ad optimization. Where this fits Snapchat Conversions API is a **server destination** in the walkerOS flow: Sends events server-side to Snapchat's Conversions API using raw HTTP (no SDK). Identity fields (\`em\`, \`ph\`, \`fn\`, \`ln\`, \`db\`, \`ge\`, \`ct\`, \`st\`, \`zp\`, \`country\`, \`external\_id\`) are SHA-256 hashed before sending. Events include a stable \`event\_id\` for deduplication with the Snap Pixel. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-snapchat ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationSnapchat } from '@walkeros/server-destination-snapchat'; await startFlow({ destinations: { snapchat: { code: destinationSnapchat, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', pixelId: 'YOUR_PIXEL_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "snapchat": { "package": "@walkeros/server-destination-snapchat", "import": "destinationSnapchat", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | ---------------------------------- | -------------------------------------------------------------------------- | ---- | | `accessToken*` | `string` | Long-lived Conversions API access token from Snapchat Ads Manager | | | `pixelId*` | `string` | Snap Pixel ID | | | `url` | `string` | Custom Conversions API base URL (default https\://tr.snapchat.com/v3/) | | | `action_source` | `'WEB' \| 'MOBILE_APP' \| 'OFFLIN` | Event action source (default WEB) | | | `doNotHash` | `Array` | User data fields to skip hashing | | | `user_data` | `Record` | Mapping for user data fields | | | `testMode` | `boolean` | When true, sends events to /events/validate instead of /events for testing | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add is sent to Snapchat as an ADD\_CART conversion with value and product details. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "color": "blue" }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "quantity": 1 } } ], "consent": { "functional": true }, "id": "e1f2a3b4c5d60002", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products/running-shoes" } } ``` Mapping ``` { "name": "ADD_CART", "data": { "map": { "custom_data": { "map": { "value": "data.price", "currency": { "value": "EUR" }, "contents": { "loop": [ "nested", { "condition": { "$code": "e=>h(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "item_price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } } } ``` Out ``` sendServer("https://tr.snapchat.com/v3/p1x3l1d/events?access_token=s3cr3t", "{\"data\":[{\"event_name\":\"ADD_CART\",\"event_time\":1700000901,\"action_source\":\"WEB\",\"event_id\":\"e1f2a3b4c5d60002\",\"user_data\":{},\"custom_data\":{\"value\":89.99,\"currency\":\"EUR\",\"contents\":[{\"id\":\"SKU-B2\",\"item_price\":89.99,\"quantity\":1}]},\"event_source_url\":\"https://shop.example.com/products/running-shoes\"}]}") ``` ### Sign up A newsletter form submission is sent to Snapchat as a SIGN\_UP conversion with a hashed email. Event ``` { "name": "form submit", "data": { "type": "newsletter" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "e1f2a3b4c5d60004", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000903000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/contact" } } ``` Mapping ``` { "name": "SIGN_UP", "data": { "map": { "user_data": { "map": { "em": "user.email" } }, "custom_data": { "map": { "sign_up_method": { "value": "newsletter" } } } } } } ``` Out ``` sendServer("https://tr.snapchat.com/v3/p1x3l1d/events?access_token=s3cr3t", "{\"data\":[{\"event_name\":\"SIGN_UP\",\"event_time\":1700000903,\"action_source\":\"WEB\",\"event_id\":\"e1f2a3b4c5d60004\",\"user_data\":{\"em\":\"b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514\"},\"custom_data\":{\"sign_up_method\":\"newsletter\"},\"event_source_url\":\"https://example.com/contact\"}]}") ``` ### Page view A page view is forwarded to Snapchat with the event source URL and no extra custom data. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "e1f2a3b4c5d60003", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/docs/" } } ``` Out ``` sendServer("https://tr.snapchat.com/v3/p1x3l1d/events?access_token=s3cr3t", "{\"data\":[{\"event_name\":\"page view\",\"event_time\":1700000902,\"action_source\":\"WEB\",\"event_id\":\"e1f2a3b4c5d60003\",\"user_data\":{},\"custom_data\":{},\"event_source_url\":\"https://example.com/docs/\"}]}") ``` ### Purchase A completed order is sent to the Snapchat Conversions API as a PURCHASE with value, currency, and contents. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Widget Pro", "price": 124.99, "quantity": 2 } } ], "consent": { "functional": true }, "id": "e1f2a3b4c5d60001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/checkout/complete" } } ``` Mapping ``` { "name": "PURCHASE", "data": { "map": { "custom_data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "transaction_id": "data.id", "contents": { "loop": [ "nested", { "condition": { "$code": "e=>h(e)&&\"product\"===e.entity" }, "map": { "id": "data.id", "item_price": "data.price", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } }, "user_data": { "map": { "external_id": "user.id" } } } } } ``` Out ``` sendServer("https://tr.snapchat.com/v3/p1x3l1d/events?access_token=s3cr3t", "{\"data\":[{\"event_name\":\"PURCHASE\",\"event_time\":1700000900,\"action_source\":\"WEB\",\"event_id\":\"e1f2a3b4c5d60001\",\"user_data\":{\"external_id\":\"fcdec6df4d44dbc637c7c5b58efface52a7f8a88535423430255be0bb89bedd8\"},\"custom_data\":{\"value\":249.99,\"currency\":\"EUR\",\"transaction_id\":\"ORD-300\",\"contents\":[{\"id\":\"SKU-A1\",\"item_price\":124.99,\"quantity\":2}]},\"event_source_url\":\"https://shop.example.com/checkout/complete\"}]}") ``` ## Event mapping[​](#event-mapping "Direct link to Event mapping") Map walkerOS events to Snapchat standard event names via each rule's `name` field. Snapchat uses **UPPERCASE** standard event names (e.g. `PURCHASE`, not `Purchase`). ``` "mapping": { "order": { "complete": { "name": "PURCHASE", "data": { "map": { "custom_data": { "map": { "value": "data.total", "currency": "data.currency", "transaction_id": "data.id" } } } } } } } ``` ### custom\_data structure[​](#custom_data-structure "Direct link to custom_data structure") Unlike Meta's CAPI (which places properties flat on the event root), Snapchat requires event properties to be nested under `custom_data`. Always map into `custom_data.map`. ## Hashing[​](#hashing "Direct link to Hashing") The destination automatically SHA-256 hashes the 11 standard identity fields before sending: `em`, `ph`, `fn`, `ln`, `db`, `ge`, `ct`, `st`, `zp`, `country`, `external_id`. Pass raw values and hashing is handled for you. Non-hashable fields pass through as-is: `sc_cookie1`, `client_ip_address`, `client_user_agent`, `sc_click_id`, `idfv`, `madid`. If a value is already hashed or you want to skip hashing for a specific field, use `doNotHash`: ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID", "doNotHash": ["em"] } ``` ## Deduplication[​](#deduplication "Direct link to Deduplication") Each event is sent with `event_id` set to the walkerOS event `id`. If you also run the Snap Pixel in the browser, Snapchat deduplicates server and browser events that share the same `event_id` and `event_name` within a 48h window, so conversions are not double-counted. ## Test mode[​](#test-mode "Direct link to Test mode") Set `testMode: true` to send events to Snapchat's `/events/validate` endpoint for validation instead of the live `/events` endpoint. Useful during development. Events validated via this endpoint do **not** count toward ads delivery or optimization. ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelId": "YOUR_PIXEL_ID", "testMode": true } ``` ## Links[​](#links "Direct link to Links") * [Snapchat Conversions API documentation](https://businesshelp.snapchat.com/s/article/conversions-api) * [CAPI parameters reference](https://businesshelp.snapchat.com/s/article/capi-parameters) --- # AWS SNS [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/aws) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/aws)[ ](https://www.npmjs.com/package/@walkeros/server-destination-aws) [Package](https://www.npmjs.com/package/@walkeros/server-destination-aws) Server-side event publishing to [AWS Simple Notification Service](https://aws.amazon.com/sns/) topics. Each event is serialized as JSON and published via the official `@aws-sdk/client-sns` SDK. Supports the AWS default credential chain, profile-based authentication, and pre-configured client injection. Idempotent topic provisioning via the `setup()` lifecycle, with declared subscriptions, KMS encryption, FIFO ordering, and per-event message attributes. The destination ships inside `@walkeros/server-destination-aws` alongside Firehose. Import the named export `destinationSNS` and reference it via the `code` field in `flow.json`. Where this fits SNS is a **server destination** in the walkerOS flow: Publishes events to an SNS topic for fan-out delivery to SQS queues, Lambda functions, HTTPS endpoints, email, SMS, and custom subscribers. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-aws ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationSNS } from '@walkeros/server-destination-aws'; await startFlow({ destinations: { sns: { code: destinationSNS, config: { settings: { topicName: 'walkeros-events', region: 'eu-central-1', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "sns": { "package": "@walkeros/server-destination-aws", "import": "destinationSNS", "config": { "settings": { "topicName": "walkeros-events", "region": "eu-central-1" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------ | -------- | ------------------------------------------------------------------------------------------------------------ | ---- | | `topicName*` | `string` | SNS topic name (like walkeros-events). Required. | | | `region` | `string` | AWS region (like eu-central-1). | | | `client` | `any` | Pre-configured AWS SNSClient instance. | | | `config` | `any` | AWS SDK SNSClient configuration options. | | | `topicArn` | `string` | Topic ARN. Populated by init() from CreateTopic. Operators may pre-set to skip the runtime CreateTopic call. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------------------ | --------------------- | ----------- | ---- | | `messageAttributes` | `Record` | | | | `messageGroupId` | `any` | | | | `messageDeduplicationId` | `any` | | | ## Authentication[​](#authentication "Direct link to Authentication") Three modes, evaluated in order: 1. **Default credential chain.** Nothing to configure beyond `region`. The AWS SDK resolves credentials in the standard order: environment variables, shared credentials file, IAM role attached to the runtime (EC2, ECS, Lambda). 2. **Profile or explicit credentials via `settings.config`.** Pass `config.credentials` or `config.profile`, forwarded directly to the SDK. 3. **Pre-configured client.** Pass an existing `SNSClient` instance as `settings.client` for shared clients across destinations or custom transport. ## Setup[​](#setup "Direct link to Setup") Provision the topic, attributes, tags, and declared subscriptions in one shot: ``` walkeros setup destination.sns -c flow.json ``` Setup is **authoritative-apply**: declared state is written to declared resources via a single idempotent `CreateTopic` call (plus one `Subscribe` call per declared subscription). Non-declared subscriptions and tags are left untouched, never listed, never logged. Operators may freely manage subscriptions or tags outside walkerOS without interference. Re-running setup against a fully provisioned topic with declared state matching actual produces zero state mutations. The result is JSON-stringified to stdout: ``` { "topicArn": "arn:aws:sns:eu-central-1:000000000000:walkeros-events", "topicCreated": true, "tagsApplied": 2, "subscriptionsCreated": 1 } ``` `topicCreated` reflects whether the topic existed before this setup run (via a `GetTopicAttributes` probe against an STS-derived candidate ARN). `tagsApplied` and `subscriptionsCreated` count declared resources written. ### FIFO topics[​](#fifo-topics "Direct link to FIFO topics") Set `setup.fifoTopic: true`. The destination auto-appends `.fifo` to the topic name when missing and applies `FifoTopic`/`ContentBasedDeduplication` attributes at creation. Reverse: `setup.fifoTopic: false` with a `.fifo` suffix throws a clear error. ### KMS encryption[​](#kms-encryption "Direct link to KMS encryption") Set `setup.kmsMasterKeyId` to apply server-side encryption at creation. The AWS-managed default key alias is `alias/aws/sns`. ### Subscriptions[​](#subscriptions "Direct link to Subscriptions") Each declared subscription is created with one idempotent `Subscribe` call. Supported protocols: `sqs`, `lambda`, `https`, `http`, `email`, `sms`. Per-subscription attributes (`rawMessageDelivery`, `filterPolicy`, `deadLetterTargetArn`) are applied at the same call. ``` "setup": { "subscriptions": [ { "protocol": "sqs", "endpoint": "arn:aws:sqs:eu-central-1:000000000000:walkeros-q", "rawMessageDelivery": true } ] } ``` ## FIFO ordering[​](#fifo-ordering "Direct link to FIFO ordering") `messageGroupId` and `messageDeduplicationId` are `Mapping.Value` fields resolved per event. A string path drives the value from event data; a value-config form supplies a literal: ``` "mapping": { "order": { "complete": { "settings": { "messageGroupId": "user.id", "messageDeduplicationId": "id" } } } } ``` ## Message attributes[​](#message-attributes "Direct link to Message attributes") `messageAttributes` is a `Mapping.Map`. Each value resolves per event to the SDK's `{ DataType, StringValue }` shape. Bare strings are auto-wrapped as `{ DataType: 'String', StringValue }`: ``` "mapping": { "product": { "view": { "settings": { "messageAttributes": { "schema_version": { "value": { "DataType": "String", "StringValue": "v4" } }, "tenant": "data.tenant_id" } } } } } ``` ## IAM[​](#iam "Direct link to IAM") Setup role: * `sns:CreateTopic` * `sns:GetTopicAttributes` (existence probe) * `sns:Subscribe` * `sts:GetCallerIdentity` (account-ID resolution) Runtime push role: * `sns:Publish` * `sns:CreateTopic` (init's idempotent ARN capture). Drop this if you pre-populate `settings.topicArn` from setup output. Recommended: separate roles for setup and runtime. Setup runs from a provisioner identity; runtime push uses a least-privileged role. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **`NotFoundException` at push time.** The topic was deleted or the runtime role lacks `sns:Publish`. Re-run `walkeros setup destination.sns` or verify IAM. **`AccessDenied` at setup.** The setup role is missing one of the four permissions above. Check that `sts:GetCallerIdentity` is granted; SNS errors on a missing account-ID probe propagate verbatim. **FIFO suffix error.** A topic name ending in `.fifo` was supplied with `setup.fifoTopic: false`, or vice versa. Either set `fifoTopic: true` or rename the topic. **Subscription endpoint rejected.** AWS validates the endpoint at `Subscribe` time. SNS does NOT create the SQS queue, Lambda, or HTTPS endpoint. Provision those resources separately, then declare the subscription here. ## Next steps[​](#next-steps "Direct link to Next steps") * [AWS SQS source](/docs/sources/server/sqs.md) for the consumer side. Standard pattern: SNS topic fans out to one or more SQS queues that walkerOS pulls. * [Mapping configuration](/docs/mapping/.md) * [Flow configuration](/docs/getting-started/flow/.md) --- # SQLite [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/sqlite) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/sqlite)[ ](https://www.npmjs.com/package/@walkeros/server-destination-sqlite) [Package](https://www.npmjs.com/package/@walkeros/server-destination-sqlite) Beta Persists walkerOS events to SQLite. One destination, two drivers behind a single interface: [`better-sqlite3`](https://github.com/WiseLibs/better-sqlite3) for local files (and `:memory:`), and [`@libsql/client`](https://github.com/tursodatabase/libsql-client-ts) for remote Turso / libSQL / sqld over HTTP or WebSocket. Driver selection is URL-driven; both SDKs are optional peer dependencies so you install only the one you need. Where this fits SQLite is a **server destination** in the walkerOS flow: Receives events server-side from the collector, serializes them into a canonical row, and inserts them into a SQLite-compatible database. Good fit for single-host deployments (local file), embedded analytics (in-memory), and edge-deployed Turso databases. ## Installation[​](#installation "Direct link to Installation") ``` # Local file / :memory: npm install @walkeros/server-destination-sqlite better-sqlite3 # Remote Turso / libSQL npm install @walkeros/server-destination-sqlite @libsql/client ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationSQLite } from '@walkeros/server-destination-sqlite'; await startFlow({ destinations: { sqlite: { code: destinationSQLite, config: { settings: { sqlite: { url: './events.db', }, }, setup: true, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "sqlite": { "package": "@walkeros/server-destination-sqlite", "config": { "settings": { "sqlite": { "url": "./events.db" } }, "setup": true } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `sqlite*` | `sqlite` | SQLite / libSQL configuration | | | `url*` | `string` | SQLite connection URL. libsql://, http(s)://, ws(s):// route to libSQL/Turso. Anything else is treated as a local file path via better-sqlite3. Use ':memory:' for an ephemeral in-memory database. | | | `authToken` | `string` | libSQL / Turso auth token. Ignored for better-sqlite3 (local) connections. | | | `table` | `string` | Target table name. Defaults to "events". | | | `schema` | `'auto' \| 'manual'` | \[DEPRECATED] Use config.setup instead. "auto" maps to "setup: true" (run \`walkeros setup destination.\\`); "manual" maps to "setup: false". Removed in the next major. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | -------------------------------------------------------------------------------------- | ---- | | `table` | `string` | Override target table name for this rule. Takes precedence over settings.sqlite.table. | | ## Examples ### Custom table A destination-level table setting inserts events into a custom SQLite table with the same column layout. Event ``` { "name": "form submit", "data": { "type": "contact" }, "context": { "dev": [ "test", 1 ] }, "globals": {}, "custom": { "completely": "random" }, "user": { "session": "sess-99", "id": "" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": {}, "id": "evt-2", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/contact" } } ``` Out ``` client.runInsert([ 1700000101, "evt-2", "form submit", "form", "submit", "sess-99", "", "https://example.com/contact", "", "", "{\"type\":\"contact\"}", "{}", "{}" ]) ``` ### Default insert A walker event is inserted into the default events table with canonical columns and JSON-encoded sections. Event ``` { "name": "page view", "data": { "title": "Home" }, "context": { "dev": [ "test", 1 ] }, "globals": { "env": "prod" }, "custom": { "completely": "random" }, "user": { "session": "sess-1", "id": "user-42" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "analytics": true }, "id": "evt-1", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "https://example.com/prev" } } ``` Out ``` client.runInsert([ 1700000100, "evt-1", "page view", "page", "view", "sess-1", "user-42", "https://example.com/", "Home", "https://example.com/prev", "{\"title\":\"Home\"}", "{\"env\":\"prod\"}", "{\"analytics\":true}" ]) ``` ### Order insert An order complete is inserted with numeric data serialized as JSON in the data column. Event ``` { "name": "order complete", "data": { "id": "ORD-1", "total": 99 }, "context": { "shopping": [ "complete", 0 ] }, "globals": {}, "custom": { "completely": "random" }, "user": { "session": "", "id": "" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": {}, "id": "evt-3", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` client.runInsert([ 1700000102, "evt-3", "order complete", "order", "complete", "", "", "", "", "", "{\"id\":\"ORD-1\",\"total\":99}", "{}", "{}" ]) ``` ### Table override A mapping rule overrides the target table so specific events are inserted into a dedicated SQLite table. Event ``` { "name": "order complete", "data": { "id": "ORD-2", "total": 42 }, "context": { "shopping": [ "complete", 0 ] }, "globals": {}, "custom": { "completely": "random" }, "user": { "session": "", "id": "" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": {}, "id": "evt-4", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "table": "orders" } } ``` Out ``` client.runInsert([ 1700000103, "evt-4", "order complete", "order", "complete", "", "", "", "", "", "{\"id\":\"ORD-2\",\"total\":42}", "{}", "{}" ]) ``` The destination opens the connection during `init()` and prepares the INSERT statement once. Schema creation lives in the `setup` lifecycle (see below), not in `init()`. On `destroy()` the connection is closed cleanly. User-provided clients (wired via `env.client` or `settings.sqlite._client`) are left untouched. ## Setup[​](#setup "Direct link to Setup") Create the events table and apply pragmas with one command: ``` walkeros setup destination.sqlite ``` This runs `CREATE TABLE IF NOT EXISTS` with the canonical 15-column walkerOS Event v4 schema and applies four pragmas: * `journal_mode = WAL` (better concurrent reads) * `synchronous = NORMAL` (good durability vs. perf balance) * `foreign_keys = ON` * `temp_store = MEMORY` Setup is idempotent. Re-running against a populated database is a safe no-op. Drift between the declared schema and the actual table is logged as `WARN setup.drift {field, declared, actual}`. Setup never auto-mutates an existing table, no `ALTER TABLE`, no destructive recreates. The default 15-column schema mirrors the canonical walkerOS Event v4 layout. Only `name` is `NOT NULL`. See [Event Model](/docs/getting-started/event-model.md) for the full field reference. ``` CREATE TABLE IF NOT EXISTS events ( name TEXT NOT NULL, data TEXT, context TEXT, globals TEXT, custom TEXT, user TEXT, nested TEXT, consent TEXT, id TEXT, trigger TEXT, entity TEXT, action TEXT, timestamp TEXT, timing INTEGER, source TEXT ) ``` Nested objects (`data`, `context`, `globals`, `custom`, `user`, `nested`, `consent`, `source`) are stored as JSON strings. Override defaults in `config.setup`: ``` "setup": { "pragmas": { "journal_mode": "DELETE" }, "indexes": [{ "name": "idx_events_name", "columns": ["name"] }] } ``` `setup: true` accepts all defaults. `setup: false` (or omitted) means `walkeros setup destination.sqlite` is a no-op for this destination. Use `mapping.settings.table` to route specific events to a dedicated table (for example orders to `orders`, identities to `identities`). ### Migration from `schema`[​](#migration-from-schema "Direct link to migration-from-schema") Deprecated The package-local `settings.sqlite.schema` setting is deprecated. The framework now owns the setup lifecycle through `config.setup`. The deprecated form still works and emits a one-time WARN through the destination logger. | Old (`settings.sqlite.schema`) | New (`config.setup`) | Effect | | ------------------------------ | -------------------- | -------------------------------------------------- | | `'auto'` | `true` | `walkeros setup destination.sqlite` creates table. | | `'manual'` | `false` | Setup is a no-op. Bring your own schema + mapping. | | omitted | omitted | No-op until `setup` is set explicitly. | Remove the `schema` field from `settings.sqlite` and add `setup: true` (or `false`) at the `config` level. ## Drivers[​](#drivers "Direct link to Drivers") ### Local, `better-sqlite3`[​](#local-better-sqlite3 "Direct link to local-better-sqlite3") Sync native driver, fastest option for single-host deployments. The URL is treated as a filesystem path; `:memory:` works too. All four default pragmas are honored. ``` { "sqlite": { "url": "/var/lib/walkeros/events.db" } } ``` ### Remote, `@libsql/client`[​](#remote-libsqlclient "Direct link to remote-libsqlclient") Async HTTP/WSS driver for [Turso](https://turso.tech/), self-hosted [sqld](https://github.com/tursodatabase/libsql), or any libSQL-compatible endpoint. Auth via `authToken`. The remote server controls journaling, so a client-side `journal_mode` pragma is silently ignored. The other pragmas (`synchronous`, `foreign_keys`, `temp_store`) still apply. ``` { "sqlite": { "url": "libsql://my-db.turso.io", "authToken": "$env.TURSO_TOKEN" } } ``` URL prefixes `libsql://`, `http://`, `https://`, `ws://`, `wss://` route to the libSQL driver. Anything else (bare paths, `:memory:`) routes to `better-sqlite3`. ## Limitations[​](#limitations "Direct link to Limitations") * v1 issues one `INSERT` per event. A `pushBatch` path is planned for v2. * Connection death is not auto-retried; a fatal driver error logs and drops events until the flow restarts. --- # TikTok Events API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/tiktok) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/tiktok)[ ](https://www.npmjs.com/package/@walkeros/server-destination-tiktok) [Package](https://www.npmjs.com/package/@walkeros/server-destination-tiktok) Server-side event delivery to TikTok's Events API for enhanced conversion tracking, bypassing browser limitations and improving match quality for ad optimization. Where this fits TikTok Events API is a **server destination** in the walkerOS flow: Sends events server-side to TikTok's Events API using raw HTTP (no SDK). Identity fields (email, phone\_number, external\_id) are SHA-256 hashed before sending. Events include a stable \`event\_id\` for deduplication with the browser pixel. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-tiktok ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationTikTok } from '@walkeros/server-destination-tiktok'; await startFlow({ destinations: { tiktok: { code: destinationTikTok, config: { settings: { accessToken: 'YOUR_ACCESS_TOKEN', pixelCode: 'YOUR_PIXEL_CODE', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "tiktok": { "package": "@walkeros/server-destination-tiktok", "import": "destinationTikTok", "config": { "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelCode": "YOUR_PIXEL_CODE" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------- | ------------------------ | ------------------------------------------------------ | ---- | | `pixelCode*` | `string` | TikTok Pixel Code from Events Manager | | | `accessToken*` | `string` | Events API access token from TikTok Events Manager | | | `url` | `string` | Custom Events API endpoint URL | | | `test_event_code` | `string` | Test event code for debugging in TikTok Events Manager | | | `doNotHash` | `Array` | User data fields to skip hashing | | | `user_data` | `Record` | Mapping for user data fields | | | `partner_name` | `string` | Partner name for TikTok attribution | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add is sent to TikTok as an AddToCart event with value, currency, and product contents. Event ``` { "name": "product add", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "color": "blue" }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "SKU-B2", "name": "Running Shoes", "price": 89.99, "quantity": 1 } } ], "consent": { "functional": true }, "id": "d1e2f3a4b5c60002", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/products/running-shoes" } } ``` Mapping ``` { "name": "AddToCart", "data": { "map": { "content_type": { "value": "product" }, "value": "data.price", "currency": { "value": "EUR" }, "contents": { "loop": [ "nested", { "condition": { "$code": "e=>y(e)&&\"product\"===e.entity" }, "map": { "content_id": "data.id", "content_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 }, "price": "data.price" } } ] } } } } ``` Out ``` sendServer("https://business-api.tiktok.com/open_api/v1.3/event/track/", "{\"pixel_code\":\"PIXEL_CODE\",\"partner_name\":\"walkerOS\",\"data\":[{\"event\":\"AddToCart\",\"event_id\":\"d1e2f3a4b5c60002\",\"timestamp\":\"2023-11-14T22:28:21.000Z\",\"context\":{\"page\":{\"url\":\"https://shop.example.com/products/running-shoes\"}},\"properties\":{\"content_type\":\"product\",\"value\":89.99,\"currency\":\"EUR\",\"contents\":[{\"content_id\":\"SKU-B2\",\"content_name\":\"Running Shoes\",\"quantity\":1,\"price\":89.99}]}}]}", { "headers": { "Access-Token": "s3cr3t", "Content-Type": "application/json" } }) ``` ### Submit form A newsletter form submission is sent to TikTok as a SubmitForm event with the hashed email in user context. Event ``` { "name": "form submit", "data": { "type": "newsletter" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d1e2f3a4b5c60004", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000903000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/contact" } } ``` Mapping ``` { "name": "SubmitForm", "data": { "map": { "user_data": { "map": { "email": "user.email" } } } } } ``` Out ``` sendServer("https://business-api.tiktok.com/open_api/v1.3/event/track/", "{\"pixel_code\":\"PIXEL_CODE\",\"partner_name\":\"walkerOS\",\"data\":[{\"event\":\"SubmitForm\",\"event_id\":\"d1e2f3a4b5c60004\",\"timestamp\":\"2023-11-14T22:28:23.000Z\",\"context\":{\"user\":{\"email\":\"b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514\"},\"page\":{\"url\":\"https://example.com/contact\"}},\"properties\":{}}]}", { "headers": { "Access-Token": "s3cr3t", "Content-Type": "application/json" } }) ``` ### Page view A page view is forwarded to TikTok as a page view event with the source URL in the page context. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "d1e2f3a4b5c60003", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://example.com/docs/" } } ``` Out ``` sendServer("https://business-api.tiktok.com/open_api/v1.3/event/track/", "{\"pixel_code\":\"PIXEL_CODE\",\"partner_name\":\"walkerOS\",\"data\":[{\"event\":\"page view\",\"event_id\":\"d1e2f3a4b5c60003\",\"timestamp\":\"2023-11-14T22:28:22.000Z\",\"context\":{\"page\":{\"url\":\"https://example.com/docs/\"}},\"properties\":{}}]}", { "headers": { "Access-Token": "s3cr3t", "Content-Type": "application/json" } }) ``` ### Complete payment A completed order is sent to the TikTok Events API as a CompletePayment with value, currency, and contents. Event ``` { "name": "order complete", "data": { "id": "ORD-300", "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "user-123", "device": "device-456" }, "nested": [ { "entity": "product", "data": { "id": "SKU-A1", "name": "Widget Pro", "price": 124.99, "quantity": 2 } } ], "consent": { "functional": true }, "id": "d1e2f3a4b5c60001", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "browser", "platform": "web", "url": "https://shop.example.com/checkout/complete" } } ``` Mapping ``` { "name": "CompletePayment", "data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "order_id": "data.id", "content_type": { "value": "product" }, "contents": { "loop": [ "nested", { "condition": { "$code": "e=>y(e)&&\"product\"===e.entity" }, "map": { "content_id": "data.id", "content_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 }, "price": "data.price" } } ] }, "user_data": { "map": { "external_id": "user.id" } } } } } ``` Out ``` sendServer("https://business-api.tiktok.com/open_api/v1.3/event/track/", "{\"pixel_code\":\"PIXEL_CODE\",\"partner_name\":\"walkerOS\",\"data\":[{\"event\":\"CompletePayment\",\"event_id\":\"d1e2f3a4b5c60001\",\"timestamp\":\"2023-11-14T22:28:20.000Z\",\"context\":{\"user\":{\"external_id\":\"fcdec6df4d44dbc637c7c5b58efface52a7f8a88535423430255be0bb89bedd8\"},\"page\":{\"url\":\"https://shop.example.com/checkout/complete\"}},\"properties\":{\"value\":249.99,\"currency\":\"EUR\",\"order_id\":\"ORD-300\",\"content_type\":\"product\",\"contents\":[{\"content_id\":\"SKU-A1\",\"content_name\":\"Widget Pro\",\"quantity\":2,\"price\":124.99}]}}]}", { "headers": { "Access-Token": "s3cr3t", "Content-Type": "application/json" } }) ``` ## Hashing[​](#hashing "Direct link to Hashing") The destination automatically SHA-256 hashes `email`, `phone_number`, and `external_id` before sending to TikTok. Pass raw values and the hashing is handled for you. If your data is already hashed or you want to skip hashing for specific fields, use the `doNotHash` setting: ``` "settings": { "accessToken": "YOUR_ACCESS_TOKEN", "pixelCode": "YOUR_PIXEL_CODE", "doNotHash": ["email"] } ``` ## Deduplication[​](#deduplication "Direct link to Deduplication") Each event is sent with `event_id` set to the walkerOS event `id`. If you also run the [TikTok browser pixel](/docs/destinations/web/tiktok.md), TikTok deduplicates server and browser events that share the same `event_id`, so conversions are not double-counted. --- # X (Twitter) Conversions API [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/twitter) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/destinations/twitter)[ ](https://www.npmjs.com/package/@walkeros/server-destination-twitter) [Package](https://www.npmjs.com/package/@walkeros/server-destination-twitter) Beta Server-side event delivery to X's (Twitter's) Conversions API for enhanced ad attribution and privacy-compliant conversion tracking. Where this fits X Conversions API is a **server destination** in the walkerOS flow: Sends conversion events server-side to X's Conversions API, bypassing browser limitations for improved data quality. Deduplicates with the browser-side X Pixel via \`conversion\_id\`. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-destination-twitter ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationTwitter } from '@walkeros/server-destination-twitter'; await startFlow({ destinations: { twitter: { code: destinationTwitter, config: { settings: { pixelId: 'YOUR_PIXEL_ID', eventId: 'tw-xxxxx-xxxxx', consumerKey: 'YOUR_CONSUMER_KEY', consumerSecret: 'YOUR_CONSUMER_SECRET', accessToken: 'YOUR_ACCESS_TOKEN', accessTokenSecret: 'YOUR_ACCESS_TOKEN_SECRET', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "twitter": { "package": "@walkeros/server-destination-twitter", "import": "destinationTwitter", "config": { "settings": { "pixelId": "YOUR_PIXEL_ID", "eventId": "tw-xxxxx-xxxxx", "consumerKey": "YOUR_CONSUMER_KEY", "consumerSecret": "YOUR_CONSUMER_SECRET", "accessToken": "YOUR_ACCESS_TOKEN", "accessTokenSecret": "YOUR_ACCESS_TOKEN_SECRET" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------------- | ------------------------ | --------------------------------------------------- | ---- | | `pixelId*` | `string` | X Pixel ID used in the Conversions API endpoint URL | | | `eventId*` | `string` | Default pre-registered conversion event ID | | | `consumerKey*` | `string` | OAuth 1.0a API Key (Consumer Key) for X Ads API | | | `consumerSecret*` | `string` | OAuth 1.0a API Key Secret (Consumer Secret) | | | `accessToken*` | `string` | OAuth 1.0a User Access Token | | | `accessTokenSecret*` | `string` | OAuth 1.0a User Access Token Secret | | | `apiVersion` | `string` | X Ads API version number | | | `doNotHash` | `Array` | Array of user data fields that should not be hashed | | | `url` | `string` | Custom base URL for the X Conversions API endpoint | | | `user_data` | `Record` | Mapping configuration for user identifiers | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------------- | ------------------ | ------------------------------------------------------- | ---- | | `eventId` | `string` | Override the default conversion event ID for this event | | | `value` | `string \| number` | Conversion monetary value (sent to X as a string) | | | `currency` | `string` | ISO 4217 currency code | | | `number_items` | `integer` | Number of items in the conversion | | | `description` | `string` | Free-text description of the conversion | | ## Examples ### Lead A form submission is sent to X as a conversion with the SHA-256 hashed email as the identifier. Event ``` { "name": "form submit", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "email": "user@example.com" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000901000", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000901000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Out ``` sendServer("https://ads-api.x.com/12/measurement/conversions/o8z6j", "{\"conversions\":[{\"conversion_time\":\"2023-11-14T22:28:21.000Z\",\"event_id\":\"tw-o8z6j-o8z21\",\"identifiers\":[{\"hashed_email\":\"b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514\"}],\"conversion_id\":\"ev-1700000901000\"}]}", { "headers": { "Authorization": "", "Content-Type": "application/json" } }) ``` ### Purchase A completed order is sent to the X (Twitter) Conversions API with conversion value and the hashed email. Event ``` { "name": "order complete", "data": { "total": 249.99, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "jane@example.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000900000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000900000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } ``` Out ``` sendServer("https://ads-api.x.com/12/measurement/conversions/o8z6j", "{\"conversions\":[{\"conversion_time\":\"2023-11-14T22:28:20.000Z\",\"event_id\":\"tw-o8z6j-o8z21\",\"identifiers\":[{\"hashed_email\":\"8c87b489ce35cf2e2f39f80e282cb2e804932a56a213983eeeb428407d43b52d\"}],\"conversion_id\":\"ev-1700000900000\",\"value\":\"249.99\"}]}", { "headers": { "Authorization": "", "Content-Type": "application/json" } }) ``` ### Purchase with twclid A purchase is sent to X with both the hashed email and the twclid click id for ads attribution. Event ``` { "name": "order complete", "data": { "total": 89.99, "currency": "USD" }, "context": { "twclid": [ "23opevjt88psuo13lu8d020qkn", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "email": "buyer@co.com" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ev-1700000902000", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000902000, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "express", "platform": "server" } } ``` Mapping ``` { "settings": { "value": "data.total", "currency": { "key": "data.currency", "value": "USD" } }, "data": { "map": { "user_data": { "map": { "twclid": "context.twclid" } } } } } ``` Out ``` sendServer("https://ads-api.x.com/12/measurement/conversions/o8z6j", "{\"conversions\":[{\"conversion_time\":\"2023-11-14T22:28:22.000Z\",\"event_id\":\"tw-o8z6j-o8z21\",\"identifiers\":[{\"hashed_email\":\"484c39bfb51212665d9673805c112b5ba04cbf0460b6d3f00bcdc18b92afed66\"},{\"twclid\":\"23opevjt88psuo13lu8d020qkn\"}],\"conversion_id\":\"ev-1700000902000\",\"value\":\"89.99\"}]}", { "headers": { "Authorization": "", "Content-Type": "application/json" } }) ``` ## User identification[​](#user-identification "Direct link to User identification") The destination builds an `identifiers` array. Each identifier is a separate single-key object in the payload. At least one **primary** identifier is required. Events without one are silently skipped. **Primary identifiers:** * **`hashed_email`**: resolved from `event.user.email` or a `user_data.email` mapping. The email is normalized (trimmed, lowercased) and SHA-256 hashed before sending. Set `doNotHash: ['email']` if you are pre-hashing upstream. * **`hashed_phone_number`**: resolved from `event.user.phone` or a `user_data.phone` mapping. SHA-256 hashed. Use `doNotHash: ['phone']` to skip. * **`twclid`**: X click ID (pass-through, **not** hashed). Typically forwarded from the browser session: ``` "settings": { "pixelId": "$env.TWITTER_PIXEL_ID", "eventId": "$env.TWITTER_EVENT_ID", "user_data": { "twclid": "context.twclid" } } ``` **Secondary identifiers** (optional): `ip_address`, `user_agent`, pass-through. ## Per-event overrides[​](#per-event-overrides "Direct link to Per-event overrides") The default `eventId` from settings applies to all events. Override it per event via mapping to route different events to different pre-registered conversion events: ``` "mapping": { "order": { "complete": { "settings": { "eventId": { "value": "tw-o8z6j-purchase01" }, "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "number_items": "data.count" } } } } ``` ## Browser deduplication[​](#browser-deduplication "Direct link to Browser deduplication") When running both the browser-side X Pixel and this server destination, the walkerOS event `id` is sent as `conversion_id` on both sides. X deduplicates conversions that share the same `conversion_id` and conversion event, so only one conversion is counted. --- # Amplitude [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/amplitude) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/amplitude)[ ](https://www.npmjs.com/package/@walkeros/web-destination-amplitude) [Package](https://www.npmjs.com/package/@walkeros/web-destination-amplitude) [Amplitude](https://amplitude.com/) is a product analytics platform with built-in identity, revenue, groups, and optional session replay, feature experiments, and guides & surveys. This destination forwards walkerOS events to Amplitude via the official [`@amplitude/unified`](https://www.npmjs.com/package/@amplitude/unified) SDK, translating events into `amplitude.track(...)`, `identify(...)`, `revenue(...)`, `setGroup(...)`, and `setOptOut(...)` calls. Loads the Amplitude Analytics SDK, forwards every walkerOS event, and lets you layer on identity, revenue, groups, consent toggling, and optional plugins. Where this fits Amplitude is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-amplitude ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationAmplitude } from '@walkeros/web-destination-amplitude'; await startFlow({ destinations: { amplitude: { code: destinationAmplitude, config: { consent: { analytics: true }, settings: { apiKey: 'YOUR_AMPLITUDE_API_KEY', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "amplitude": { "package": "@walkeros/web-destination-amplitude", "config": { "consent": { "analytics": true }, "settings": { "apiKey": "YOUR_AMPLITUDE_API_KEY" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `apiKey*` | `string` | Your Amplitude project API key. Find it in your Amplitude project settings under "General" → "API Keys" (like a1b2c3d4e5f6789012345678abcdef12). | | | `serverZone` | `'US' \| 'EU'` | Amplitude server zone. Default: US. | | | `flushIntervalMillis` | `integer` | How often (in ms) to flush the event queue. Default: 1000. | | | `flushQueueSize` | `integer` | Max queued events before a flush. Default: 30. | | | `flushMaxRetries` | `integer` | Max retries on failed flush. Default: 5. | | | `transport` | `'fetch' \| 'xhr' \| 'beacon'` | HTTP transport. Default: fetch. | | | `useBatch` | `boolean` | Use the Amplitude batch endpoint instead of the standard endpoint. Default: false. | | | `appVersion` | `string` | Application version; tagged onto every event. | | | `identify` | `any` | walkerOS mapping value resolving to an identity object. Keys: user, device, session, set, setOnce, add, append, prepend, preInsert, postInsert, remove, unset, clearAll. | | | `sessionReplay` | `any` | Session Replay options. When present, session replay is enabled via @amplitude/unified with the provided options. | | | `experiment` | `any` | Feature Experiment config. Must include \`deploymentKey\`. When present, the experiment plugin from @amplitude/unified is configured with the provided options. | | | `engagement` | `any` | Guides & Surveys config. Pass \`true\` for defaults, or an options object for custom configuration. Enabled via @amplitude/unified. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | --------------- | ----- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to an object with any of: user, device, session, set, setOnce, add, append, prepend, preInsert, postInsert, remove, unset, clearAll. | | | `revenue` | `any` | Revenue mapping. Resolves to a single object or (via loop) an array of objects, each with: productId, price, quantity, revenueType, currency, revenue, receipt, receiptSig, eventProperties. One amplitude.revenue() call fires per item. | | | `group` | `any` | Group assignment. Resolves to { type, name } → amplitude.setGroup(type, name). | | | `groupIdentify` | `any` | Group properties. Resolves to { type, name, set?, setOnce?, ... } → amplitude.groupIdentify(type, name, identify). | | | `reset` | `any` | Logout trigger. Resolves to a truthy value → amplitude.reset() (clears userId, regenerates deviceId). Typically used with silent: true on a user logout rule. | | ## Examples ### Consent granted A walker consent command with analytics granted opts back into Amplitude tracking via setOptOut(false). Event ``` { "analytics": true } ``` Out ``` amplitude.setOptOut(false) ``` ### Consent revoked A walker consent command with analytics denied opts out of Amplitude tracking via setOptOut(true). Event ``` { "analytics": false } ``` Out ``` amplitude.setOptOut(false); amplitude.setOptOut(true) ``` ### Default track A walker event is forwarded to Amplitude as an amplitude.track call with the event name and empty properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "0fdd63d98b7d08c1", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("product view", {}) ``` ### Destination identify Destination-level identify sets userId, deviceId, and sessionId on the Amplitude client before sending events; setters only re-fire when the resolved values change (cached values are not re-applied on subsequent pushes). Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "603a82f0c9ca2289", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.setUserId("us3r"); amplitude.setDeviceId("c00k13"); amplitude.setSessionId(394324160); amplitude.track("page view", {}) ``` ### Include data Destination-level include flattens the event data section into prefixed event\_properties on every track call. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "7111f1e9cd5e88ab", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` amplitude.track("product view", { "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420 }) ``` ### Group assignment A company update assigns the user to a group and sets group properties via setGroup and groupIdentify. Event ``` { "name": "company update", "data": { "company": "Acme", "industry": "tech", "employee_count": 50, "founded_year": 2020 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "7050697b47fa9ae2", "trigger": "test", "entity": "company", "action": "update", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "group": { "map": { "type": { "value": "company" }, "name": "data.company" } }, "groupIdentify": { "map": { "type": { "value": "company" }, "name": "data.company", "set": { "map": { "industry": "data.industry", "size": "data.employee_count" } }, "setOnce": { "map": { "founded": "data.founded_year" } } } } } } ``` Out ``` amplitude.setGroup("company", "Acme"); amplitude.groupIdentify("company", "Acme", { "set": { "industry": "tech", "size": 50 }, "setOnce": { "founded": 2020 } }) ``` ### Multi-product order An order fires one amplitude.revenue call per nested product plus a single track for the order totals. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "58e306fa378eb849", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "globals" ], "settings": { "revenue": { "loop": [ "nested", { "condition": { "$code": "e=>{var t;return\"number\"==typeof(null==(t=null==e?void 0:e.data)?void 0:t.price)}" }, "map": { "productId": "data.id", "price": "data.price", "quantity": { "key": "data.quantity", "value": 1 }, "revenueType": { "value": "purchase" }, "currency": { "key": "data.currency", "value": "EUR" } } } ] } } } ``` Out ``` amplitude.revenue({ "productId": "ers", "price": 420, "quantity": 1, "revenueType": "purchase", "currency": "EUR" }); amplitude.revenue({ "productId": "cc", "price": 42, "quantity": 1, "revenueType": "purchase", "currency": "EUR" }); amplitude.track("order complete", { "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "globals_pagegroup": "shop" }) ``` ### Rule include overrides A per-rule include replaces the destination-level include for the matched event, here using only globals. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "3168e7e178504b90", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "globals" ] } ``` Out ``` amplitude.track("order complete", { "globals_pagegroup": "shop" }) ``` ### Subscription revenue A subscription renewal fires a single amplitude.revenue call with productId, price, and currency fallback. Event ``` { "name": "subscription renew", "data": { "plan_id": "plan-pro", "amount": 9.99 }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "30e3eed46e05560f", "trigger": "test", "entity": "subscription", "action": "renew", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "revenue": { "map": { "productId": "data.plan_id", "price": "data.amount", "revenueType": { "value": "renewal" }, "currency": { "key": "data.currency", "value": "EUR" } } } } } ``` Out ``` amplitude.revenue({ "productId": "plan-pro", "price": 9.99, "revenueType": "renewal", "currency": "EUR" }) ``` ### User login identify A user login sets the Amplitude userId and runs identify with set, setOnce, and add operations. Event ``` { "name": "user login", "data": { "user_id": "new-user-123", "plan": "premium", "company": "Acme", "email": "user@acme.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "4c97b00228ab5581", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": { "map": { "user": "data.user_id", "set": { "map": { "plan": "data.plan", "company": "data.company", "email": "data.email" } }, "setOnce": { "map": { "first_login": "timestamp" } }, "add": { "map": { "login_count": { "value": 1 } } } } } } } ``` Out ``` amplitude.setUserId("new-user-123"); amplitude.identify({ "set": { "plan": "premium", "company": "Acme", "email": "user@acme.com" }, "setOnce": { "first_login": 1700000105 }, "add": { "login_count": 1 } }) ``` ### User logout reset A user logout calls amplitude.reset to clear the userId and regenerate the deviceId. Event ``` { "name": "user logout", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "c3dcebcdecb19db0", "trigger": "test", "entity": "user", "action": "logout", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "reset": true } } ``` Out ``` amplitude.reset() ``` ## Plugins[​](#plugins "Direct link to Plugins") All three plugin packages are npm-bundled with the destination (\~135 KB for Session Replay plus Experiment and Engagement payloads). Enable them via destination settings: ``` settings: { apiKey: 'YOUR_KEY', sessionReplay: { sampleRate: 1 }, experiment: { deploymentKey: 'DEPLOYMENT_KEY' }, engagement: true, } ``` ## Related[​](#related "Direct link to Related") * [Amplitude Browser SDK](https://amplitude.com/docs/sdks/analytics/browser/browser-sdk-2) * [Amplitude Session Replay](https://amplitude.com/docs/session-replay) * [Amplitude Experiment](https://amplitude.com/docs/experiment) * [Amplitude Guides & Surveys](https://amplitude.com/docs/guides-and-surveys) --- # Microsoft Clarity [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/clarity) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/clarity)[ ](https://www.npmjs.com/package/@walkeros/web-destination-clarity) [Package](https://www.npmjs.com/package/@walkeros/web-destination-clarity) [Microsoft Clarity](https://clarity.microsoft.com/) provides session replays, heatmaps and behavioural insights for web products, free and without sampling caps. This destination forwards walkerOS events to Clarity via the official [`@microsoft/clarity`](https://www.npmjs.com/package/@microsoft/clarity) SDK, translating events into `Clarity.event(...)` calls, resolving custom tags, identities, session priority, and consent state. Where this fits Clarity is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-clarity ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationClarity } from '@walkeros/web-destination-clarity'; await startFlow({ destinations: { clarity: { code: destinationClarity, config: { settings: { apiKey: '3t0wlogvdz', consent: { analytics: 'analytics_Storage', marketing: 'ad_Storage', }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "clarity": { "package": "@walkeros/web-destination-clarity", "config": { "settings": { "apiKey": "3t0wlogvdz", "consent": { "analytics": "analytics_Storage", "marketing": "ad_Storage" } } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `apiKey*` | `string` | Your Microsoft Clarity project ID (e.g. "3t0wlogvdz"). Find it in your Clarity dashboard under Settings → Setup. | | | `consent` | `Record` | Translation table from walkerOS consent keys to Clarity ConsentV2 categories. Example: { "analytics": "analytics\_Storage", "marketing": "ad\_Storage" }. Required to get meaningful consent behavior — Clarity expects its own category names. | | | `identify` | `any` | walkerOS mapping value resolving to positional arguments for Clarity.identify(). Keys: customId (required), customSessionId?, customPageId?, friendlyName?. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ----- | -------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { customId, customSessionId?, customPageId?, friendlyName? } → Clarity.identify(...). | | | `set` | `any` | Explicit custom tag mapping. Resolved object keys become Clarity.setTag(key, value) calls. Array values pass through as string\[] unchanged. | | | `upgrade` | `any` | Session priority reason. Resolves to a string → Clarity.upgrade(reason). Used to flag important sessions for retention beyond sampling. | | ## Examples ### Array tag values Array values such as product tags are passed through to Clarity.setTag preserving the array shape. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420, "tags": [ "sale", "featured" ] }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "73736c3babd44578", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "set": { "map": { "product_tags": "data.tags" } } } } ``` Out ``` clarity.setTag("product_tags", [ "sale", "featured" ]); clarity.event("product view") ``` ### Combined features A purchase identifies the user, sets an order tag, upgrades the session, and fires the Clarity event in order. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ba9d422ae58344d7", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000108, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Purchase", "settings": { "identify": { "map": { "customId": "user.id" } }, "set": { "map": { "order_id": "data.id" } }, "upgrade": { "value": "purchase" } } } ``` Out ``` clarity.identify("us3r"); clarity.setTag("order_id", "0rd3r1d"); clarity.upgrade("purchase"); clarity.event("Purchase") ``` ### Consent granted A walker consent command translates analytics and marketing grants into a Clarity.consentV2 call. Event ``` { "analytics": true, "marketing": true } ``` Out ``` clarity.consentV2({ "analytics_Storage": "granted", "ad_Storage": "granted" }) ``` ### Consent revoked A walker consent command with analytics and marketing denied calls Clarity.consentV2 with denied flags. Event ``` { "analytics": false, "marketing": false } ``` Out ``` clarity.consentV2({ "analytics_Storage": "granted", "ad_Storage": "granted" }); clarity.consentV2({ "analytics_Storage": "denied", "ad_Storage": "denied" }) ``` ### Default event A walker event becomes a Clarity.event call with the event name as the Clarity custom event. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "6386c939cdbbf8f8", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` clarity.event("product view") ``` ### Destination identify Destination-level identify calls Clarity.identify with the user id on every push as Clarity recommends. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "e68333c38261f1f4", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` clarity.identify("us3r"); clarity.event("page view") ``` ### Include data as tags A mapping include flattens the event data section into Clarity.setTag calls before the event fires. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "5ce87b471cd9f05d", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data" ] } ``` Out ``` clarity.setTag("data_id", "0rd3r1d"); clarity.setTag("data_currency", "EUR"); clarity.setTag("data_shipping", "5.22"); clarity.setTag("data_taxes", "73.76"); clarity.setTag("data_total", "555"); clarity.event("order complete") ``` ### Upgrade session A completed order upgrades the Clarity session priority so it is retained beyond the sampling cap. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "6dca14e69ebf9806", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Purchase", "settings": { "upgrade": { "value": "purchase" } } } ``` Out ``` clarity.upgrade("purchase"); clarity.event("Purchase") ``` ### Tags without event A page view sets Clarity tags while silent suppresses the event, letting Clarity handle page tracking itself. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "358b4d9e010aa1ed", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "set": { "map": { "page_id": "data.id" } } } } ``` Out ``` clarity.setTag("page_id", "/docs/") ``` ### Custom tags A product view sets Clarity session tags such as product color and id before firing the event. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "5810c05ce75e449e", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "set": { "map": { "product_color": "data.color", "product_id": "data.id" } } } } ``` Out ``` clarity.setTag("product_color", "black"); clarity.setTag("product_id", "ers"); clarity.event("product view") ``` ### User login identify A user login fires Clarity.identify with custom id, session id, page id, and friendly name then tracks the event. Event ``` { "name": "user login", "data": { "id": "u-123", "name": "Jane Doe" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "shop", "pagetype": "home" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "001c8b9c863bdc14", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "identify": { "map": { "customId": "data.id", "customSessionId": "user.session", "customPageId": "globals.pagetype", "friendlyName": "data.name" } } } } ``` Out ``` clarity.identify("u-123", "s3ss10n", "home", "Jane Doe"); clarity.event("user login") ``` ## Consent translation[​](#consent-translation "Direct link to Consent translation") Clarity expects its own category names (`analytics_Storage`, `ad_Storage`). walkerOS uses arbitrary consent keys, so translation is **explicit**: configure `settings.consent` once and the destination maps every `walker consent` event into `Clarity.consentV2({ ... })`. When all mapped keys are revoked, the destination additionally calls `Clarity.consent(false)` (the legacy v1 API that erases cookies and ends the session). --- # d8a [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/d8a) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/d8a)[ ](https://www.npmjs.com/package/@walkeros/web-destination-d8a) [Package](https://www.npmjs.com/package/@walkeros/web-destination-d8a) [d8a](https://d8a.tech/) is a GA4-compatible, warehouse-native analytics platform. The destination sends walkerOS events to the d8a web tracker using the GA4 gtag-style `d8a()` API, so existing GA4 mappings translate directly. Where this fits d8a is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-d8a ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationD8a } from '@walkeros/web-destination-d8a'; await startFlow({ destinations: { d8a: { code: destinationD8a, config: { settings: { property_id: '80e1d6d0-560d-419f-ac2a-fe9281e93386', server_container_url: 'https://global.t.d8a.tech/80e1d6d0-560d-419f-ac2a-fe9281e93386/d/c', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "d8a": { "package": "@walkeros/web-destination-d8a", "config": { "settings": { "property_id": "80e1d6d0-560d-419f-ac2a-fe9281e93386", "server_container_url": "https://global.t.d8a.tech/80e1d6d0-560d-419f-ac2a-fe9281e93386/d/c" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Event Mapping[​](#event-mapping "Direct link to Event Mapping") By default, walkerOS event names are converted to snake\_case and sent with the configured property ID as `send_to`. Override the name and shape per event the same way you would for GA4: ``` mapping: { order: { complete: { name: 'purchase', data: { map: { transaction_id: 'data.id', value: 'data.total', currency: { key: 'data.currency', value: 'EUR' }, }, }, }, }, } ``` This produces: ``` d8a('event', 'purchase', { transaction_id: '0rd3r1d', value: 555, currency: 'EUR', send_to: '80e1d6d0-560d-419f-ac2a-fe9281e93386', }); ``` ## Consent Mode[​](#consent-mode "Direct link to Consent Mode") The destination supports d8a's gtag-compatible consent mode. By default, walkerOS consent keys map to d8a consent fields: | walkerOS consent key | d8a consent fields | | -------------------- | -------------------------------------------------- | | `marketing` | `ad_storage`, `ad_user_data`, `ad_personalization` | | `functional` | `analytics_storage` | Disable consent mode with `como: false`, or provide a custom mapping via the `como` setting. ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------------------- | ------------------- | ------------------------------------------------------------------------------------ | ---- | | `property_id*` | `string` | d8a property ID | | | `server_container_url*` | `string` | d8a collector URL for the property | | | `como` | `boolean \| object` | Consent mode configuration: false (disabled), true (use defaults), or custom mapping | | | `data` | `any` | Custom data mapping configuration | | | `dataLayerName` | `string` | Name of the d8a command queue (default: d8aLayer) | | | `globalName` | `string` | Name of the global d8a function (default: d8a) | | | `send_page_view` | `boolean` | Enable automatic pageview tracking | | | `snakeCase` | `boolean` | Convert event names to snake\_case | | | `debug_mode` | `boolean` | Enable debug mode | | | `cookie_domain` | `string` | Cookie domain strategy: auto, none, or a specific domain | | | `cookie_path` | `string` | Cookie path | | | `cookie_expires` | `number` | Cookie lifetime in seconds | | | `cookie_flags` | `string` | Raw cookie flags | | | `cookie_prefix` | `string` | Cookie name prefix | | | `cookie_update` | `boolean` | Refresh cookie expirations on activity | | | `session_timeout_ms` | `number` | Session timeout window in milliseconds | | | `session_engagement_time_sec` | `number` | Minimum engaged time in seconds | | | `flush_interval_ms` | `number` | Flush interval in milliseconds | | | `max_batch_size` | `number` | Maximum batch size | | | `user_id` | `string` | GA4-style user ID | | | `client_id` | `string` | Client ID override | | | `campaign_id` | `string` | | | | `campaign_source` | `string` | | | | `campaign_medium` | `string` | | | | `campaign_name` | `string` | | | | `campaign_term` | `string` | | | | `campaign_content` | `string` | | | | `page_location` | `string` | | | | `page_title` | `string` | | | | `page_referrer` | `string` | | | | `content_group` | `string` | | | | `language` | `string` | | | | `screen_resolution` | `string` | | | | `ignore_referrer` | `boolean` | | | | `site_search_enabled` | `boolean` | | | | `site_search_query_params` | `string \| array` | | | | `outbound_clicks_enabled` | `boolean` | | | | `outbound_exclude_domains` | `string \| array` | | | | `file_downloads_enabled` | `boolean` | | | | `file_download_extensions` | `string \| array` | | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A product add event is mapped to the d8a add\_to\_cart event with item details and value. Event ``` { "name": "product add", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "cb6d55efba66a63f", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000301, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "add_to_cart", "include": [ "data" ], "data": { "map": { "currency": { "value": "EUR", "key": "data.currency" }, "value": "data.price", "items": { "loop": [ "this", { "map": { "item_id": "data.id", "item_variant": "data.color", "quantity": { "value": 1, "key": "data.quantity" } } } ] } } } } ``` Out ``` d8a("event", "add_to_cart", { "currency": "EUR", "value": 420, "items": [ { "item_id": "ers", "item_variant": "black", "quantity": 1 } ], "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420, "send_to": "80e1d6d0-560d-419f-ac2a-fe9281e93386" }) ``` ### Consent mode A walker consent command updates d8a using gtag consent mode parameters. Event ``` { "marketing": true, "functional": true } ``` Out ``` d8a("consent", "default", { "ad_storage": "denied", "ad_user_data": "denied", "ad_personalization": "denied", "analytics_storage": "denied" }); d8a("consent", "update", { "ad_storage": "granted", "ad_user_data": "granted", "ad_personalization": "granted", "analytics_storage": "granted" }) ``` ### Initialization The destination installs d8a and configures a property with its server container URL. Event ``` { "settings": { "property_id": "80e1d6d0-560d-419f-ac2a-fe9281e93386", "server_container_url": "https://global.t.d8a.tech/80e1d6d0-560d-419f-ac2a-fe9281e93386/d/c" } } ``` Out ``` d8a("js", {}); d8a("config", "80e1d6d0-560d-419f-ac2a-fe9281e93386", { "server_container_url": "https://global.t.d8a.tech/80e1d6d0-560d-419f-ac2a-fe9281e93386/d/c" }) ``` ### Page view A page view event is forwarded as a d8a page\_view event. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "6e9ab415df068433", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000300, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` d8a("event", "page_view", { "send_to": "80e1d6d0-560d-419f-ac2a-fe9281e93386" }) ``` ### Purchase An order complete event is mapped to the d8a purchase event with transaction details and nested product items. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "0e39f2f697acce9e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000302, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data", "context" ], "data": { "map": { "transaction_id": "data.id", "value": "data.total", "tax": "data.taxes", "shipping": "data.shipping", "currency": { "key": "data.currency", "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>{const a=e;return k(e)&&\"product\"===a.entity}" }, "map": { "item_id": "data.id", "item_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` d8a("event", "purchase", { "transaction_id": "0rd3r1d", "value": 555, "tax": 73.76, "shipping": 5.22, "currency": "EUR", "items": [ { "item_id": "ers", "item_name": "Everyday Ruck Snack", "quantity": 1 }, { "item_id": "cc", "item_name": "Cool Cap", "quantity": 1 } ], "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "context_shopping": "complete", "send_to": "80e1d6d0-560d-419f-ac2a-fe9281e93386" }) ``` --- # FullStory [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/fullstory) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/fullstory)[ ](https://www.npmjs.com/package/@walkeros/web-destination-fullstory) [Package](https://www.npmjs.com/package/@walkeros/web-destination-fullstory) Beta [FullStory](https://www.fullstory.com/) delivers enterprise session replay and digital-experience analytics. This destination forwards walkerOS events to FullStory via the official [`@fullstory/browser`](https://www.npmjs.com/package/@fullstory/browser) SDK v2, calling `FullStory('trackEvent', ...)` for events and supporting identity, user/page properties, and consent gating via `start`/`shutdown`. Where this fits FullStory is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-fullstory ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationFullStory } from '@walkeros/web-destination-fullstory'; await startFlow({ destinations: { fullstory: { code: destinationFullStory, config: { settings: { orgId: 'o-XXXXXX-na1', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "fullstory": { "package": "@walkeros/web-destination-fullstory", "config": { "settings": { "orgId": "o-XXXXXX-na1" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------------------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `orgId*` | `string` | Your FullStory organization ID (e.g. "o-XXXXXX-na1"). Find it in FullStory under Settings > General. | | | `host` | `string` | Recording server host for proxy support. | | | `script` | `string` | Custom script CDN host domain. | | | `cookieDomain` | `string` | Override cookie domain for cross-subdomain tracking. | | | `debug` | `boolean` | Enable browser console debug logging. | | | `devMode` | `boolean` | Disable recording entirely for development environments. | | | `startCaptureManually` | `boolean` | Delay capture until FullStory("start") is called. Recommended for GDPR: init the SDK immediately but wait for consent before recording. | | | `namespace` | `string` | Global FS identifier override (default: "FS"). | | | `recordCrossDomainIFrames` | `boolean` | Enable cross-domain iframe recording. | | | `identify` | `any` | Destination-level identity mapping. Resolves to { uid, properties? } for FullStory setIdentity. Fires on every push. | | | `consent` | `Record` | Translation table from walkerOS consent keys to FullStory consent actions. "capture" controls start/shutdown (recording on/off). "consent" controls setIdentity({ consent }) flag. Example: { "analytics": "capture" }. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { uid, properties? } for FullStory setIdentity. Overrides destination-level identify for this event. | | | `set` | `any` | Property mapping. Resolved object keys become FullStory setProperties() properties. Use with setType to control scope. | | | `setType` | `'user' \| 'page'` | Property scope for the set mapping. "user" persists across sessions; "page" resets on navigation. Default: "user". | | ## Examples ### Combined features A purchase fires FullStory setIdentity, setProperties, and trackEvent in the canonical execution order. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "afe77a361ce0adc1", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Purchase", "settings": { "identify": { "map": { "uid": "user.id" } }, "set": { "map": { "order_id": "data.id" } } } } ``` Out ``` fullstory.setIdentity({ "uid": "us3r" }); fullstory.setProperties({ "type": "user", "properties": { "order_id": "0rd3r1d" } }); fullstory.trackEvent({ "name": "Purchase", "properties": {} }) ``` ### Start capture A walker consent grant for analytics calls FullStory start to resume session recording. Event ``` { "analytics": true } ``` Out ``` fullstory.start() ``` ### Consent flag granted A walker consent grant with action consent sets the FullStory identity consent flag to true. Event ``` { "marketing": true } ``` Out ``` fullstory.setIdentity({ "consent": true }) ``` ### Shutdown capture A walker consent revoke for analytics calls FullStory shutdown to stop session recording. Event ``` { "analytics": false } ``` Out ``` fullstory.start(); fullstory.shutdown() ``` ### Consent flag revoked A walker consent revoke with action consent sets the FullStory identity consent flag to false. Event ``` { "marketing": false } ``` Out ``` fullstory.setIdentity({ "consent": true }); fullstory.setIdentity({ "consent": false }) ``` ### Default event A walker event becomes a FullStory trackEvent call with the event name and empty properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "4fbe9551ec847742", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fullstory.trackEvent({ "name": "product view", "properties": {} }) ``` ### Destination identify Destination-level identify fires FullStory setIdentity with the user id before every track call. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9ee03e1af1593e8b", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` fullstory.setIdentity({ "uid": "us3r" }); fullstory.trackEvent({ "name": "page view", "properties": {} }) ``` ### Renamed event A mapping renames the event so the FullStory trackEvent uses Purchase instead of the walker name. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "45abba4f07995954", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Purchase" } ``` Out ``` fullstory.trackEvent({ "name": "Purchase", "properties": {} }) ``` ### Set page properties A page view sets FullStory page-type properties without firing a track, since FullStory auto-captures navigation. Event ``` { "name": "page view", "data": { "id": "/docs/", "title": "Getting Started" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "638359088adf4d5e", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "set": { "map": { "pageName": "data.title" } }, "setType": "page" } } ``` Out ``` fullstory.setProperties({ "type": "page", "properties": { "pageName": "Getting Started" } }) ``` ### Set user properties A purchase sets user-level FullStory properties such as revenue and currency alongside the tracked event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "8f7c9e4242511a5e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "Purchase", "settings": { "set": { "map": { "revenue": "data.total", "currency": "data.currency" } } } } ``` Out ``` fullstory.setProperties({ "type": "user", "properties": { "revenue": 555, "currency": "EUR" } }); fullstory.trackEvent({ "name": "Purchase", "properties": {} }) ``` ### User login identify A user login fires FullStory setIdentity with uid and profile properties before tracking the event. Event ``` { "name": "user login", "data": { "id": "u-123", "name": "Jane Doe", "email": "jane@example.com" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "05fd1ef163293542", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "identify": { "map": { "uid": "data.id", "properties": { "map": { "displayName": "data.name", "email": "data.email" } } } } } } ``` Out ``` fullstory.setIdentity({ "uid": "u-123", "properties": { "displayName": "Jane Doe", "email": "jane@example.com" } }); fullstory.trackEvent({ "name": "user login", "properties": {} }) ``` ## Consent[​](#consent "Direct link to Consent") FullStory supports runtime consent via `start`/`shutdown` (recording on/off) or the `consent` flag on `setIdentity`. Map walkerOS consent keys to FullStory actions via `settings.consent`, and combine with `startCaptureManually: true` to delay recording until consent is granted: ``` destinations: { fullstory: { code: destinationFullStory, config: { consent: { analytics: true }, settings: { orgId: 'o-XXXXXX-na1', startCaptureManually: true, consent: { analytics: 'capture', }, }, }, }, } ``` * `"capture"` toggles `start`/`shutdown` (recording on/off) * `"consent"` sets the `consent` flag on `setIdentity` --- # Google Gtag [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag)[ ](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Package](https://www.npmjs.com/package/@walkeros/web-destination-gtag) The Google Gtag destination provides a unified interface for sending events to Google Analytics 4 (GA4), Google Ads, and Google Tag Manager (GTM) through a single destination configuration. Loads gtag script dynamically, initializes the gtag function with your config, handles Consent Mode v2 commands automatically, and pushes mapped events to Google services. Consent Mode is on by default via `como` (set `como: false` to disable it). When the destination declares a consent signal (`config.consent` or `como_advanced`), the denied `consent default` is emitted before `config`, as Consent Mode v2 requires. Set `como_advanced: true` (or a number of milliseconds) to opt into advanced mode, emitting the denied default at page load with `wait_for_update`, for non-EU setups. Where this fits Gtag is a **destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-gtag ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationGtag } from '@walkeros/web-destination-gtag'; const { collector, elb } = await startFlow({ destinations: { gtag: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX', }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "ga4": { "measurementId": "G-XXXXXXXXXX" } } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---- | | `como` | `boolean \| object` | Consent mode configuration: false (disabled), true (use defaults), or custom mapping | | | `como_advanced` | `boolean \| number` | Advanced consent mode (non-EU): emit denied default at page load with wait\_for\_update (true=500ms, or a number of ms) | | | `ga4` | `ga4` | GA4-specific configuration settings | | | `measurementId*` | `string` | GA4 Measurement ID from Google Analytics | | | `debug` | `boolean` | Enable debug mode for GA4 | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `server_container_url` | `string` | Server-side GTM container URL | | | `snakeCase` | `boolean` | Convert parameter names to snake\_case | | | `transport_url` | `string` | Custom transport URL for GA4 | | | `data` | `any` | Custom data mapping configuration | | | `ads` | `ads` | Google Ads specific configuration settings | | | `conversionId*` | `string` | Google Ads Conversion ID (required) | | | `currency` | `string` | Currency code (ISO 4217, e.g., USD, EUR) | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | | `enhancedConversions` | `enhancedConversions` | Enhanced conversions: maps event fields to Google user\_data for improved measurement accuracy | | | `email` | `any` | Mapping value for user email | | | `phone_number` | `any` | Mapping value for user phone number | | | `address` | `address` | Address mapping for enhanced conversions | | | `first_name` | `any` | Mapping value for first name | | | `last_name` | `any` | Mapping value for last name | | | `street` | `any` | Mapping value for street address | | | `city` | `any` | Mapping value for city | | | `region` | `any` | Mapping value for region/state | | | `postal_code` | `any` | Mapping value for postal/zip code | | | `country` | `any` | Mapping value for country | | | `gtm` | `gtm` | Google Tag Manager specific configuration settings | | | `containerId*` | `string` | GTM Container ID (required) | | | `dataLayer` | `string` | Custom dataLayer variable name (default: dataLayer) | | | `domain` | `string` | Custom GTM domain for script loading | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | ---------------------------------------- | ---- | | `ga4` | `ga4` | GA4-specific event mapping | | | `ads` | `ads` | Google Ads-specific event mapping | | | `label` | `string` | Conversion label for this specific event | | | `gtm` | `gtm` | GTM-specific event mapping | | ## Examples ### Add to cart A product add event is mapped to the GA4 add\_to\_cart event with item details and value. Event ``` { "name": "product add", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "fbba7c472a9c31eb", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "add_to_cart", "include": [ "data" ], "data": { "map": { "currency": { "value": "EUR", "key": "data.currency" }, "value": "data.price", "items": { "loop": [ "this", { "map": { "item_id": "data.id", "item_variant": "data.color", "quantity": { "value": 1, "key": "data.quantity" } } } ] } } } } ``` Out ``` gtag("event", "add_to_cart", { "currency": "EUR", "value": 420, "items": [ { "item_id": "ers", "item_variant": "black", "quantity": 1 } ], "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420, "send_to": "G-XXXXXX-1" }) ``` ### Enhanced conversions enhancedConversions maps event fields into a gtag set user\_data call sent immediately before the conversion event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555, "customerEmail": "buyer@shop.com", "customerPhone": "+1234567890" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "f453151f820c366e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_EC2", "settings": { "ads": { "label": "PURCHASE_EC2" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("set", "user_data", { "email": "buyer@shop.com", "phone_number": "+1234567890" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_EC2", "currency": "EUR", "value": 555 }) ``` ### Google Ads init The destination bootstraps gtag and configures a Google Ads account via its conversionId. Event ``` { "settings": { "ads": { "conversionId": "AW-123456789", "currency": "EUR" } } } ``` Out ``` gtag("js", {}); gtag("config", "AW-123456789") ``` ### Consent Mode v2 A walker consent command with marketing and functional granted updates gtag Consent Mode v2 parameters. Event ``` { "marketing": true, "functional": true } ``` Out ``` gtag("consent", "default", { "ad_storage": "denied", "ad_user_data": "denied", "ad_personalization": "denied", "analytics_storage": "denied" }); gtag("consent", "update", { "ad_storage": "granted", "ad_user_data": "granted", "ad_personalization": "granted", "analytics_storage": "granted" }) ``` ### GA4 init The destination bootstraps gtag and configures a GA4 property via its measurementId. Event ``` { "settings": { "ga4": { "measurementId": "G-XXXXXX-1" } } } ``` Out ``` gtag("js", {}); gtag("config", "G-XXXXXX-1", {}) ``` ### GA4 include all Include flattens every event section into prefixed GA4 params, exposing data, context, user, source, and event fields. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000106", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "1700000106abcdef1700000106abcdef", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "context", "globals", "user", "source", "event" ] } ``` Out ``` gtag("event", "page_view", { "data_domain": "www.example.com", "data_title": "walkerOS documentation", "data_referrer": "https://www.walkeros.io/", "data_search": "?foo=bar", "data_hash": "#hash", "data_id": "/docs/", "context_dev": "test", "globals_pagegroup": "docs", "user_id": "us3r", "user_device": "c00k13", "user_session": "s3ss10n", "source_type": "collector", "source_schema": "4", "source_count": 1, "source_trace": "1700000106abcdef1700000106abcdef", "source_release_default": "4.3.0", "event_entity": "page", "event_action": "view", "event_trigger": "load", "event_id": "ev-1700000106", "event_name": "page view", "event_timestamp": 1700000106, "event_timing": 3.14, "send_to": "G-XXXXXX-1" }) ``` ### Ads conversion An order complete event is sent as a Google Ads conversion with a configured label and transaction value. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ee11fe65053d4f9d", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_CONV", "settings": { "ads": { "label": "PURCHASE_CONV" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555 }) ``` ### GTM dataLayer push A page view event is pushed to window\.dataLayer for GTM with the mapped event name and parameters. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9486c9004c4cd44d", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "page_view", "settings": { "gtm": {} }, "data": { "map": { "page_title": "data.title", "page_location": "data.domain" } } } ``` Out ``` dataLayer.push({ "event": "page_view", "page_title": "walkerOS documentation", "page_location": "www.example.com" }) ``` ### GTM init The destination initializes the dataLayer and pushes the gtm.js start event for a GTM container. Event ``` { "settings": { "gtm": { "containerId": "GTM-XXXXXXX" } } } ``` Out ``` dataLayer.push({ "gtm.start": 1700000000000, "event": "gtm.js" }) ``` ### Multi-tool push A single order event fans out to GA4, Google Ads, and GTM from one mapping rule with per-tool settings. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1e1a65a4baadca4f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data" ], "settings": { "ga4": {}, "ads": { "label": "PURCHASE_CONV" }, "gtm": {} }, "data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } ``` Out ``` gtag("event", "purchase", { "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "send_to": "G-XXXXXX-1" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555, "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }); dataLayer.push({ "event": "purchase", "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }) ``` ### Page view A page view event is forwarded as a GA4 page\_view event with no additional mapping. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a65af0b9422a99de", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` gtag("event", "page_view", { "send_to": "G-XXXXXX-1" }) ``` ### Purchase An order complete event is mapped to the GA4 purchase event with transaction details and nested product items. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "da5c60c8348abb6a", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data", "context" ], "data": { "map": { "transaction_id": "data.id", "value": "data.total", "tax": "data.taxes", "shipping": "data.shipping", "currency": { "key": "data.currency", "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>U(e)&&\"product\"===e.entity" }, "map": { "item_id": "data.id", "item_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` gtag("event", "purchase", { "transaction_id": "0rd3r1d", "value": 555, "tax": 73.76, "shipping": 5.22, "currency": "EUR", "items": [ { "item_id": "ers", "item_name": "Everyday Ruck Snack", "quantity": 1 }, { "item_id": "cc", "item_name": "Cool Cap", "quantity": 1 } ], "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "context_shopping": "complete", "send_to": "G-XXXXXX-1" }) ``` ## Next steps[​](#next-steps "Direct link to Next steps") * [**Mapping**](/docs/mapping/.md): transform events for Google's expected format * [**GA4 configuration**](/docs/destinations/web/gtag/ga4.md): detailed GA4 setup * [**Google Ads**](/docs/destinations/web/gtag/ads.md): conversion tracking * [**Consent management**](/docs/guides/consent/.md): control when events are sent --- # Google Ads [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag)[ ](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Package](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Google Ads conversion tracking](https://support.google.com/google-ads/answer/1722022?hl=en) enables you to track conversions and measure the effectiveness of your Google Ads campaigns via the unified gtag destination. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-gtag ``` ``` import { startFlow } from '@walkeros/collector'; import { destinationGtag } from '@walkeros/web-destination-gtag'; const { elb } = await startFlow({ destinations: { gtag: { code: destinationGtag, config: { settings: { ads: { conversionId: 'AW-XXXXXXXXX', currency: 'EUR', }, }, mapping: { order: { complete: { settings: { ads: { label: 'purchase_conversion', }, }, }, }, }, }, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---- | | `como` | `boolean \| object` | Consent mode configuration: false (disabled), true (use defaults), or custom mapping | | | `como_advanced` | `boolean \| number` | Advanced consent mode (non-EU): emit denied default at page load with wait\_for\_update (true=500ms, or a number of ms) | | | `ga4` | `ga4` | GA4-specific configuration settings | | | `measurementId*` | `string` | GA4 Measurement ID from Google Analytics | | | `debug` | `boolean` | Enable debug mode for GA4 | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `server_container_url` | `string` | Server-side GTM container URL | | | `snakeCase` | `boolean` | Convert parameter names to snake\_case | | | `transport_url` | `string` | Custom transport URL for GA4 | | | `data` | `any` | Custom data mapping configuration | | | `ads` | `ads` | Google Ads specific configuration settings | | | `conversionId*` | `string` | Google Ads Conversion ID (required) | | | `currency` | `string` | Currency code (ISO 4217, e.g., USD, EUR) | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | | `enhancedConversions` | `enhancedConversions` | Enhanced conversions: maps event fields to Google user\_data for improved measurement accuracy | | | `email` | `any` | Mapping value for user email | | | `phone_number` | `any` | Mapping value for user phone number | | | `address` | `address` | Address mapping for enhanced conversions | | | `first_name` | `any` | Mapping value for first name | | | `last_name` | `any` | Mapping value for last name | | | `street` | `any` | Mapping value for street address | | | `city` | `any` | Mapping value for city | | | `region` | `any` | Mapping value for region/state | | | `postal_code` | `any` | Mapping value for postal/zip code | | | `country` | `any` | Mapping value for country | | | `gtm` | `gtm` | Google Tag Manager specific configuration settings | | | `containerId*` | `string` | GTM Container ID (required) | | | `dataLayer` | `string` | Custom dataLayer variable name (default: dataLayer) | | | `domain` | `string` | Custom GTM domain for script loading | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | ---------------------------------------- | ---- | | `ga4` | `ga4` | GA4-specific event mapping | | | `ads` | `ads` | Google Ads-specific event mapping | | | `label` | `string` | Conversion label for this specific event | | | `gtm` | `gtm` | GTM-specific event mapping | | ## Examples ### Add to cart A product add event is mapped to the GA4 add\_to\_cart event with item details and value. Event ``` { "name": "product add", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "fbba7c472a9c31eb", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "add_to_cart", "include": [ "data" ], "data": { "map": { "currency": { "value": "EUR", "key": "data.currency" }, "value": "data.price", "items": { "loop": [ "this", { "map": { "item_id": "data.id", "item_variant": "data.color", "quantity": { "value": 1, "key": "data.quantity" } } } ] } } } } ``` Out ``` gtag("event", "add_to_cart", { "currency": "EUR", "value": 420, "items": [ { "item_id": "ers", "item_variant": "black", "quantity": 1 } ], "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420, "send_to": "G-XXXXXX-1" }) ``` ### Enhanced conversions enhancedConversions maps event fields into a gtag set user\_data call sent immediately before the conversion event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555, "customerEmail": "buyer@shop.com", "customerPhone": "+1234567890" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "f453151f820c366e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_EC2", "settings": { "ads": { "label": "PURCHASE_EC2" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("set", "user_data", { "email": "buyer@shop.com", "phone_number": "+1234567890" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_EC2", "currency": "EUR", "value": 555 }) ``` ### Google Ads init The destination bootstraps gtag and configures a Google Ads account via its conversionId. Event ``` { "settings": { "ads": { "conversionId": "AW-123456789", "currency": "EUR" } } } ``` Out ``` gtag("js", {}); gtag("config", "AW-123456789") ``` ### Consent Mode v2 A walker consent command with marketing and functional granted updates gtag Consent Mode v2 parameters. Event ``` { "marketing": true, "functional": true } ``` Out ``` gtag("consent", "default", { "ad_storage": "denied", "ad_user_data": "denied", "ad_personalization": "denied", "analytics_storage": "denied" }); gtag("consent", "update", { "ad_storage": "granted", "ad_user_data": "granted", "ad_personalization": "granted", "analytics_storage": "granted" }) ``` ### GA4 init The destination bootstraps gtag and configures a GA4 property via its measurementId. Event ``` { "settings": { "ga4": { "measurementId": "G-XXXXXX-1" } } } ``` Out ``` gtag("js", {}); gtag("config", "G-XXXXXX-1", {}) ``` ### GA4 include all Include flattens every event section into prefixed GA4 params, exposing data, context, user, source, and event fields. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000106", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "1700000106abcdef1700000106abcdef", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "context", "globals", "user", "source", "event" ] } ``` Out ``` gtag("event", "page_view", { "data_domain": "www.example.com", "data_title": "walkerOS documentation", "data_referrer": "https://www.walkeros.io/", "data_search": "?foo=bar", "data_hash": "#hash", "data_id": "/docs/", "context_dev": "test", "globals_pagegroup": "docs", "user_id": "us3r", "user_device": "c00k13", "user_session": "s3ss10n", "source_type": "collector", "source_schema": "4", "source_count": 1, "source_trace": "1700000106abcdef1700000106abcdef", "source_release_default": "4.3.0", "event_entity": "page", "event_action": "view", "event_trigger": "load", "event_id": "ev-1700000106", "event_name": "page view", "event_timestamp": 1700000106, "event_timing": 3.14, "send_to": "G-XXXXXX-1" }) ``` ### Ads conversion An order complete event is sent as a Google Ads conversion with a configured label and transaction value. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ee11fe65053d4f9d", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_CONV", "settings": { "ads": { "label": "PURCHASE_CONV" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555 }) ``` ### GTM dataLayer push A page view event is pushed to window\.dataLayer for GTM with the mapped event name and parameters. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9486c9004c4cd44d", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "page_view", "settings": { "gtm": {} }, "data": { "map": { "page_title": "data.title", "page_location": "data.domain" } } } ``` Out ``` dataLayer.push({ "event": "page_view", "page_title": "walkerOS documentation", "page_location": "www.example.com" }) ``` ### GTM init The destination initializes the dataLayer and pushes the gtm.js start event for a GTM container. Event ``` { "settings": { "gtm": { "containerId": "GTM-XXXXXXX" } } } ``` Out ``` dataLayer.push({ "gtm.start": 1700000000000, "event": "gtm.js" }) ``` ### Multi-tool push A single order event fans out to GA4, Google Ads, and GTM from one mapping rule with per-tool settings. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1e1a65a4baadca4f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data" ], "settings": { "ga4": {}, "ads": { "label": "PURCHASE_CONV" }, "gtm": {} }, "data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } ``` Out ``` gtag("event", "purchase", { "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "send_to": "G-XXXXXX-1" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555, "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }); dataLayer.push({ "event": "purchase", "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }) ``` ### Page view A page view event is forwarded as a GA4 page\_view event with no additional mapping. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a65af0b9422a99de", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` gtag("event", "page_view", { "send_to": "G-XXXXXX-1" }) ``` ### Purchase An order complete event is mapped to the GA4 purchase event with transaction details and nested product items. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "da5c60c8348abb6a", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data", "context" ], "data": { "map": { "transaction_id": "data.id", "value": "data.total", "tax": "data.taxes", "shipping": "data.shipping", "currency": { "key": "data.currency", "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>U(e)&&\"product\"===e.entity" }, "map": { "item_id": "data.id", "item_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` gtag("event", "purchase", { "transaction_id": "0rd3r1d", "value": 555, "tax": 73.76, "shipping": 5.22, "currency": "EUR", "items": [ { "item_id": "ers", "item_name": "Everyday Ruck Snack", "quantity": 1 }, { "item_id": "cc", "item_name": "Cool Cap", "quantity": 1 } ], "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "context_shopping": "complete", "send_to": "G-XXXXXX-1" }) ``` ## Enhanced conversions[​](#enhanced-conversions "Direct link to Enhanced conversions") [Enhanced conversions](https://support.google.com/google-ads/answer/9888656) improve measurement accuracy by sending first-party customer data (email, phone, address) alongside a conversion. The destination calls `gtag('set', 'user_data', {...})` immediately before the conversion event, and passes `allow_enhanced_conversions: true` at init. Google hashes the data, the destination sends it in plain form. Set `enhancedConversions` on the `ads` settings. It maps event fields to Google's `user_data`. Each value is a standard mapping value, so you can pull from `event.user`, `event.data`, or anywhere a mapping reaches. ``` { settings: { ads: { conversionId: 'AW-XXXXXXXXX', currency: 'EUR', enhancedConversions: { email: 'data.customerEmail', phone_number: 'data.customerPhone', address: { city: 'data.city', country: 'data.country', }, }, }, }, } ``` The `address` object accepts `first_name`, `last_name`, `street`, `city`, `region`, `postal_code`, and `country`. When no user data resolves, the `set user_data` call is skipped and the conversion is sent normally. --- # Google Analytics 4 (GA4) [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag)[ ](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Package](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Google Analytics 4 (GA4)](https://support.google.com/analytics/answer/10089681?hl=en\&ref_topic=14089939\&sjid=11701302784514925217-EU) is configured within the unified gtag destination. walkerOS loads the gtag script and forwards mapped events to GA4. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-gtag ``` ``` import { startFlow } from '@walkeros/collector'; import { destinationGtag } from '@walkeros/web-destination-gtag'; await startFlow({ destinations: { ga4: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX', }, }, }, }, }, }); ``` ## A page view needs no mapping[​](#a-page-view-needs-no-mapping "Direct link to A page view needs no mapping") The gtag destination snake-cases an `entity action` event name into a GA4 event name automatically, so `page view` arrives in GA4 as `page_view` with no mapping at all. An unmapped `product add` would arrive as `product_add` carrying only `send_to`, which is why the ecommerce event below needs an explicit mapping. ## Add the ecommerce mapping[​](#add-the-ecommerce-mapping "Direct link to Add the ecommerce mapping") Nest the mapping under the gtag destination to turn a `product add` event into the GA4 `add_to_cart` event. The product price becomes the GA4 `value`, the currency falls back to `EUR`, and the product is shaped into a GA4 `items` array via a `this` loop. ``` mapping: { product: { add: { name: 'add_to_cart', data: { map: { value: 'data.price', currency: { value: 'EUR', key: 'data.currency' }, items: { loop: [ 'this', { map: { item_id: 'data.id', item_name: 'data.name', quantity: { value: 1, key: 'data.quantity' }, }, }, ], }, }, }, }, }, } ``` Pushing a `product add` event: ``` elb('product add', { id: 'ers', name: 'Everyday Ruck Snack', price: 420 }); ``` produces the GA4 `add_to_cart` event: ``` gtag('event', 'add_to_cart', { value: 420, currency: 'EUR', items: [{ item_id: 'ers', item_name: 'Everyday Ruck Snack', quantity: 1 }], send_to: 'G-XXXXXXXXXX', }); ``` ## Verify in GA4[​](#verify-in-ga4 "Direct link to Verify in GA4") Copy your Measurement ID from GA4 > Admin > Data Streams and open your web stream. Watch Reports > Realtime to confirm the `page_view` event, then open Admin > DebugView to inspect the `add_to_cart` event and its parameters as they arrive. ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---- | | `como` | `boolean \| object` | Consent mode configuration: false (disabled), true (use defaults), or custom mapping | | | `como_advanced` | `boolean \| number` | Advanced consent mode (non-EU): emit denied default at page load with wait\_for\_update (true=500ms, or a number of ms) | | | `ga4` | `ga4` | GA4-specific configuration settings | | | `measurementId*` | `string` | GA4 Measurement ID from Google Analytics | | | `debug` | `boolean` | Enable debug mode for GA4 | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `server_container_url` | `string` | Server-side GTM container URL | | | `snakeCase` | `boolean` | Convert parameter names to snake\_case | | | `transport_url` | `string` | Custom transport URL for GA4 | | | `data` | `any` | Custom data mapping configuration | | | `ads` | `ads` | Google Ads specific configuration settings | | | `conversionId*` | `string` | Google Ads Conversion ID (required) | | | `currency` | `string` | Currency code (ISO 4217, e.g., USD, EUR) | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | | `enhancedConversions` | `enhancedConversions` | Enhanced conversions: maps event fields to Google user\_data for improved measurement accuracy | | | `email` | `any` | Mapping value for user email | | | `phone_number` | `any` | Mapping value for user phone number | | | `address` | `address` | Address mapping for enhanced conversions | | | `first_name` | `any` | Mapping value for first name | | | `last_name` | `any` | Mapping value for last name | | | `street` | `any` | Mapping value for street address | | | `city` | `any` | Mapping value for city | | | `region` | `any` | Mapping value for region/state | | | `postal_code` | `any` | Mapping value for postal/zip code | | | `country` | `any` | Mapping value for country | | | `gtm` | `gtm` | Google Tag Manager specific configuration settings | | | `containerId*` | `string` | GTM Container ID (required) | | | `dataLayer` | `string` | Custom dataLayer variable name (default: dataLayer) | | | `domain` | `string` | Custom GTM domain for script loading | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | ---------------------------------------- | ---- | | `ga4` | `ga4` | GA4-specific event mapping | | | `ads` | `ads` | Google Ads-specific event mapping | | | `label` | `string` | Conversion label for this specific event | | | `gtm` | `gtm` | GTM-specific event mapping | | ## Examples ### Add to cart A product add event is mapped to the GA4 add\_to\_cart event with item details and value. Event ``` { "name": "product add", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "fbba7c472a9c31eb", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "add_to_cart", "include": [ "data" ], "data": { "map": { "currency": { "value": "EUR", "key": "data.currency" }, "value": "data.price", "items": { "loop": [ "this", { "map": { "item_id": "data.id", "item_variant": "data.color", "quantity": { "value": 1, "key": "data.quantity" } } } ] } } } } ``` Out ``` gtag("event", "add_to_cart", { "currency": "EUR", "value": 420, "items": [ { "item_id": "ers", "item_variant": "black", "quantity": 1 } ], "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420, "send_to": "G-XXXXXX-1" }) ``` ### Enhanced conversions enhancedConversions maps event fields into a gtag set user\_data call sent immediately before the conversion event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555, "customerEmail": "buyer@shop.com", "customerPhone": "+1234567890" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "f453151f820c366e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_EC2", "settings": { "ads": { "label": "PURCHASE_EC2" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("set", "user_data", { "email": "buyer@shop.com", "phone_number": "+1234567890" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_EC2", "currency": "EUR", "value": 555 }) ``` ### Google Ads init The destination bootstraps gtag and configures a Google Ads account via its conversionId. Event ``` { "settings": { "ads": { "conversionId": "AW-123456789", "currency": "EUR" } } } ``` Out ``` gtag("js", {}); gtag("config", "AW-123456789") ``` ### Consent Mode v2 A walker consent command with marketing and functional granted updates gtag Consent Mode v2 parameters. Event ``` { "marketing": true, "functional": true } ``` Out ``` gtag("consent", "default", { "ad_storage": "denied", "ad_user_data": "denied", "ad_personalization": "denied", "analytics_storage": "denied" }); gtag("consent", "update", { "ad_storage": "granted", "ad_user_data": "granted", "ad_personalization": "granted", "analytics_storage": "granted" }) ``` ### GA4 init The destination bootstraps gtag and configures a GA4 property via its measurementId. Event ``` { "settings": { "ga4": { "measurementId": "G-XXXXXX-1" } } } ``` Out ``` gtag("js", {}); gtag("config", "G-XXXXXX-1", {}) ``` ### GA4 include all Include flattens every event section into prefixed GA4 params, exposing data, context, user, source, and event fields. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000106", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "1700000106abcdef1700000106abcdef", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "context", "globals", "user", "source", "event" ] } ``` Out ``` gtag("event", "page_view", { "data_domain": "www.example.com", "data_title": "walkerOS documentation", "data_referrer": "https://www.walkeros.io/", "data_search": "?foo=bar", "data_hash": "#hash", "data_id": "/docs/", "context_dev": "test", "globals_pagegroup": "docs", "user_id": "us3r", "user_device": "c00k13", "user_session": "s3ss10n", "source_type": "collector", "source_schema": "4", "source_count": 1, "source_trace": "1700000106abcdef1700000106abcdef", "source_release_default": "4.3.0", "event_entity": "page", "event_action": "view", "event_trigger": "load", "event_id": "ev-1700000106", "event_name": "page view", "event_timestamp": 1700000106, "event_timing": 3.14, "send_to": "G-XXXXXX-1" }) ``` ### Ads conversion An order complete event is sent as a Google Ads conversion with a configured label and transaction value. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ee11fe65053d4f9d", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_CONV", "settings": { "ads": { "label": "PURCHASE_CONV" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555 }) ``` ### GTM dataLayer push A page view event is pushed to window\.dataLayer for GTM with the mapped event name and parameters. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9486c9004c4cd44d", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "page_view", "settings": { "gtm": {} }, "data": { "map": { "page_title": "data.title", "page_location": "data.domain" } } } ``` Out ``` dataLayer.push({ "event": "page_view", "page_title": "walkerOS documentation", "page_location": "www.example.com" }) ``` ### GTM init The destination initializes the dataLayer and pushes the gtm.js start event for a GTM container. Event ``` { "settings": { "gtm": { "containerId": "GTM-XXXXXXX" } } } ``` Out ``` dataLayer.push({ "gtm.start": 1700000000000, "event": "gtm.js" }) ``` ### Multi-tool push A single order event fans out to GA4, Google Ads, and GTM from one mapping rule with per-tool settings. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1e1a65a4baadca4f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data" ], "settings": { "ga4": {}, "ads": { "label": "PURCHASE_CONV" }, "gtm": {} }, "data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } ``` Out ``` gtag("event", "purchase", { "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "send_to": "G-XXXXXX-1" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555, "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }); dataLayer.push({ "event": "purchase", "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }) ``` ### Page view A page view event is forwarded as a GA4 page\_view event with no additional mapping. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a65af0b9422a99de", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` gtag("event", "page_view", { "send_to": "G-XXXXXX-1" }) ``` ### Purchase An order complete event is mapped to the GA4 purchase event with transaction details and nested product items. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "da5c60c8348abb6a", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data", "context" ], "data": { "map": { "transaction_id": "data.id", "value": "data.total", "tax": "data.taxes", "shipping": "data.shipping", "currency": { "key": "data.currency", "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>U(e)&&\"product\"===e.entity" }, "map": { "item_id": "data.id", "item_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` gtag("event", "purchase", { "transaction_id": "0rd3r1d", "value": 555, "tax": 73.76, "shipping": 5.22, "currency": "EUR", "items": [ { "item_id": "ers", "item_name": "Everyday Ruck Snack", "quantity": 1 }, { "item_id": "cc", "item_name": "Cool Cap", "quantity": 1 } ], "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "context_shopping": "complete", "send_to": "G-XXXXXX-1" }) ``` --- # Google Tag Manager (GTM) [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/gtag)[ ](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Package](https://www.npmjs.com/package/@walkeros/web-destination-gtag) [Google Tag Manager (GTM)](https://tagmanager.google.com/) is a tool to manage and add third-party integrations to your website, configured within the unified gtag destination. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-gtag ``` ``` import { startFlow } from '@walkeros/collector'; import { destinationGtag } from '@walkeros/web-destination-gtag'; await startFlow({ destinations: { gtag: { code: destinationGtag, config: { settings: { gtm: { containerId: 'GTM-XXXXXXX', dataLayer: 'dataLayer', }, }, }, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---- | | `como` | `boolean \| object` | Consent mode configuration: false (disabled), true (use defaults), or custom mapping | | | `como_advanced` | `boolean \| number` | Advanced consent mode (non-EU): emit denied default at page load with wait\_for\_update (true=500ms, or a number of ms) | | | `ga4` | `ga4` | GA4-specific configuration settings | | | `measurementId*` | `string` | GA4 Measurement ID from Google Analytics | | | `debug` | `boolean` | Enable debug mode for GA4 | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `server_container_url` | `string` | Server-side GTM container URL | | | `snakeCase` | `boolean` | Convert parameter names to snake\_case | | | `transport_url` | `string` | Custom transport URL for GA4 | | | `data` | `any` | Custom data mapping configuration | | | `ads` | `ads` | Google Ads specific configuration settings | | | `conversionId*` | `string` | Google Ads Conversion ID (required) | | | `currency` | `string` | Currency code (ISO 4217, e.g., USD, EUR) | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | | `enhancedConversions` | `enhancedConversions` | Enhanced conversions: maps event fields to Google user\_data for improved measurement accuracy | | | `email` | `any` | Mapping value for user email | | | `phone_number` | `any` | Mapping value for user phone number | | | `address` | `address` | Address mapping for enhanced conversions | | | `first_name` | `any` | Mapping value for first name | | | `last_name` | `any` | Mapping value for last name | | | `street` | `any` | Mapping value for street address | | | `city` | `any` | Mapping value for city | | | `region` | `any` | Mapping value for region/state | | | `postal_code` | `any` | Mapping value for postal/zip code | | | `country` | `any` | Mapping value for country | | | `gtm` | `gtm` | Google Tag Manager specific configuration settings | | | `containerId*` | `string` | GTM Container ID (required) | | | `dataLayer` | `string` | Custom dataLayer variable name (default: dataLayer) | | | `domain` | `string` | Custom GTM domain for script loading | | | `data` | `any` | Custom data mapping (WalkerOS.Mapping.Value \| Values) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | -------- | -------- | ---------------------------------------- | ---- | | `ga4` | `ga4` | GA4-specific event mapping | | | `ads` | `ads` | Google Ads-specific event mapping | | | `label` | `string` | Conversion label for this specific event | | | `gtm` | `gtm` | GTM-specific event mapping | | ## Examples ### Add to cart A product add event is mapped to the GA4 add\_to\_cart event with item details and value. Event ``` { "name": "product add", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "intent", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "fbba7c472a9c31eb", "trigger": "click", "entity": "product", "action": "add", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "add_to_cart", "include": [ "data" ], "data": { "map": { "currency": { "value": "EUR", "key": "data.currency" }, "value": "data.price", "items": { "loop": [ "this", { "map": { "item_id": "data.id", "item_variant": "data.color", "quantity": { "value": 1, "key": "data.quantity" } } } ] } } } } ``` Out ``` gtag("event", "add_to_cart", { "currency": "EUR", "value": 420, "items": [ { "item_id": "ers", "item_variant": "black", "quantity": 1 } ], "data_id": "ers", "data_name": "Everyday Ruck Snack", "data_color": "black", "data_size": "l", "data_price": 420, "send_to": "G-XXXXXX-1" }) ``` ### Enhanced conversions enhancedConversions maps event fields into a gtag set user\_data call sent immediately before the conversion event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555, "customerEmail": "buyer@shop.com", "customerPhone": "+1234567890" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "f453151f820c366e", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000109, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_EC2", "settings": { "ads": { "label": "PURCHASE_EC2" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("set", "user_data", { "email": "buyer@shop.com", "phone_number": "+1234567890" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_EC2", "currency": "EUR", "value": 555 }) ``` ### Google Ads init The destination bootstraps gtag and configures a Google Ads account via its conversionId. Event ``` { "settings": { "ads": { "conversionId": "AW-123456789", "currency": "EUR" } } } ``` Out ``` gtag("js", {}); gtag("config", "AW-123456789") ``` ### Consent Mode v2 A walker consent command with marketing and functional granted updates gtag Consent Mode v2 parameters. Event ``` { "marketing": true, "functional": true } ``` Out ``` gtag("consent", "default", { "ad_storage": "denied", "ad_user_data": "denied", "ad_personalization": "denied", "analytics_storage": "denied" }); gtag("consent", "update", { "ad_storage": "granted", "ad_user_data": "granted", "ad_personalization": "granted", "analytics_storage": "granted" }) ``` ### GA4 init The destination bootstraps gtag and configures a GA4 property via its measurementId. Event ``` { "settings": { "ga4": { "measurementId": "G-XXXXXX-1" } } } ``` Out ``` gtag("js", {}); gtag("config", "G-XXXXXX-1", {}) ``` ### GA4 include all Include flattens every event section into prefixed GA4 params, exposing data, context, user, source, and event fields. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "ev-1700000106", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "1700000106abcdef1700000106abcdef", "type": "collector", "schema": "4" } } ``` Mapping ``` { "include": [ "data", "context", "globals", "user", "source", "event" ] } ``` Out ``` gtag("event", "page_view", { "data_domain": "www.example.com", "data_title": "walkerOS documentation", "data_referrer": "https://www.walkeros.io/", "data_search": "?foo=bar", "data_hash": "#hash", "data_id": "/docs/", "context_dev": "test", "globals_pagegroup": "docs", "user_id": "us3r", "user_device": "c00k13", "user_session": "s3ss10n", "source_type": "collector", "source_schema": "4", "source_count": 1, "source_trace": "1700000106abcdef1700000106abcdef", "source_release_default": "4.3.0", "event_entity": "page", "event_action": "view", "event_trigger": "load", "event_id": "ev-1700000106", "event_name": "page view", "event_timestamp": 1700000106, "event_timing": 3.14, "send_to": "G-XXXXXX-1" }) ``` ### Ads conversion An order complete event is sent as a Google Ads conversion with a configured label and transaction value. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "ee11fe65053d4f9d", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "PURCHASE_CONV", "settings": { "ads": { "label": "PURCHASE_CONV" } }, "data": { "map": { "value": "data.total" } } } ``` Out ``` gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555 }) ``` ### GTM dataLayer push A page view event is pushed to window\.dataLayer for GTM with the mapped event name and parameters. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "9486c9004c4cd44d", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "page_view", "settings": { "gtm": {} }, "data": { "map": { "page_title": "data.title", "page_location": "data.domain" } } } ``` Out ``` dataLayer.push({ "event": "page_view", "page_title": "walkerOS documentation", "page_location": "www.example.com" }) ``` ### GTM init The destination initializes the dataLayer and pushes the gtm.js start event for a GTM container. Event ``` { "settings": { "gtm": { "containerId": "GTM-XXXXXXX" } } } ``` Out ``` dataLayer.push({ "gtm.start": 1700000000000, "event": "gtm.js" }) ``` ### Multi-tool push A single order event fans out to GA4, Google Ads, and GTM from one mapping rule with per-tool settings. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1e1a65a4baadca4f", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000107, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data" ], "settings": { "ga4": {}, "ads": { "label": "PURCHASE_CONV" }, "gtm": {} }, "data": { "map": { "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } ``` Out ``` gtag("event", "purchase", { "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "send_to": "G-XXXXXX-1" }); gtag("event", "conversion", { "send_to": "AW-123456789/PURCHASE_CONV", "currency": "EUR", "value": 555, "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }); dataLayer.push({ "event": "purchase", "value": 555, "currency": "EUR", "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555 }) ``` ### Page view A page view event is forwarded as a GA4 page\_view event with no additional mapping. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "a65af0b9422a99de", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` gtag("event", "page_view", { "send_to": "G-XXXXXX-1" }) ``` ### Purchase An order complete event is mapped to the GA4 purchase event with transaction details and nested product items. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "da5c60c8348abb6a", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "include": [ "data", "context" ], "data": { "map": { "transaction_id": "data.id", "value": "data.total", "tax": "data.taxes", "shipping": "data.shipping", "currency": { "key": "data.currency", "value": "EUR" }, "items": { "loop": [ "nested", { "condition": { "$code": "e=>U(e)&&\"product\"===e.entity" }, "map": { "item_id": "data.id", "item_name": "data.name", "quantity": { "key": "data.quantity", "value": 1 } } } ] } } } } ``` Out ``` gtag("event", "purchase", { "transaction_id": "0rd3r1d", "value": 555, "tax": 73.76, "shipping": 5.22, "currency": "EUR", "items": [ { "item_id": "ers", "item_name": "Everyday Ruck Snack", "quantity": 1 }, { "item_id": "cc", "item_name": "Cool Cap", "quantity": 1 } ], "data_id": "0rd3r1d", "data_currency": "EUR", "data_shipping": 5.22, "data_taxes": 73.76, "data_total": 555, "context_shopping": "complete", "send_to": "G-XXXXXX-1" }) ``` --- # Heap [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/heap) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/heap)[ ](https://www.npmjs.com/package/@walkeros/web-destination-heap) [Package](https://www.npmjs.com/package/@walkeros/web-destination-heap) Beta [Heap](https://www.heap.io/) (now part of Contentsquare) is a product analytics platform with auto-capture and retroactive event definitions. This destination loads the Heap snippet from Heap's CDN and forwards walkerOS events through `heap.track(name, properties)`, plus identity, user properties, persistent event properties, and runtime consent. Where this fits Heap is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-heap ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationHeap } from '@walkeros/web-destination-heap'; await startFlow({ destinations: { heap: { code: destinationHeap, config: { loadScript: true, settings: { appId: 'YOUR_HEAP_APP_ID', }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "heap": { "package": "@walkeros/web-destination-heap", "config": { "loadScript": true, "settings": { "appId": "YOUR_HEAP_APP_ID" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `appId*` | `string` | Heap App ID. Find it in your Heap project under Settings > App ID. | | | `disableTextCapture` | `boolean` | Disable Heap auto text capture. Default: true. | | | `disablePageviewAutocapture` | `boolean` | Disable Heap automatic pageview tracking. Default: true (walkerOS sources handle pageviews). | | | `disableSessionReplay` | `boolean` | Disable Heap session replay. | | | `secureCookie` | `boolean` | SSL-only cookies. | | | `ingestServer` | `string` | Custom server endpoint for proxying Heap data. | | | `identify` | `any` | Destination-level identity mapping. Resolves to a string for heap.identify(). Example: { "key": "user.id" }. | | | `userProperties` | `any` | Destination-level user properties mapping. Resolves to object for heap.addUserProperties(). Example: { "map": { "plan": "data.plan" } }. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ---------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------- | ---- | | `identify` | `any` | Per-event identity. Resolves to a string for heap.identify(). Example: { "key": "data.email" }. | | | `reset` | `any` | Reset Heap identity on this event. Set to true to call heap.resetIdentity(). | | | `userProperties` | `any` | Per-event user properties. Resolves to object for heap.addUserProperties(). | | | `eventProperties` | `any` | Per-event persistent event properties. Resolves to object for heap.addEventProperties() (persisted across page loads). | | | `clearEventProperties` | `any` | Clear all persistent event properties. Set to true to call heap.clearEventProperties(). | | ## Examples ### Consent granted A walker consent grant for analytics calls heap.startTracking to resume event capture. Event ``` { "analytics": true } ``` Out ``` heap.startTracking() ``` ### Consent revoked After analytics consent is granted (Heap loads and starts tracking), revoking it calls heap.stopTracking to pause event capture. Event ``` { "analytics": false } ``` Out ``` heap.startTracking(); heap.stopTracking() ``` ### Default track A walker event becomes a Heap track call with the event name and empty properties. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "18cf5fe34c191ea9", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` heap.track("product view", {}) ``` ### Destination identify Destination-level identify calls heap.identify with the user id before firing the default track. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "0d98b7c4868fb919", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` heap.identify("us3r"); heap.track("page view", {}) ``` ### Renamed purchase An order complete is renamed to purchase and mapped to Heap track properties such as order\_id, total, and currency. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "c6ea4a5fcb7b112a", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "purchase", "data": { "map": { "order_id": "data.id", "total": "data.total", "currency": { "key": "data.currency", "value": "EUR" } } } } ``` Out ``` heap.track("purchase", { "order_id": "0rd3r1d", "total": 555, "currency": "EUR" }) ``` ### User properties on event An order fires Heap addUserProperties with last-order fields and then tracks the event. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "total": 555, "currency": "EUR" }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "7903105f97553c86", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "userProperties": { "map": { "last_order_value": "data.total", "last_order_currency": "data.currency" } } } } ``` Out ``` heap.addUserProperties({ "last_order_value": 555, "last_order_currency": "EUR" }); heap.track("order complete", {}) ``` ### Global event properties A page view sets persistent Heap event properties so all subsequent events include the page category. Event ``` { "name": "page view", "data": { "category": "docs" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "645a5ea53cf0bd49", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "eventProperties": { "map": { "page_category": "data.category" } } } } ``` Out ``` heap.addEventProperties({ "page_category": "docs" }) ``` ### User login identify A user login identifies the Heap user by email and adds user properties while skipping the track. Event ``` { "name": "user login", "data": { "email": "user@example.com", "plan": "premium", "company": "Acme" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "61b108e0bfe0908f", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "identify": "data.email", "userProperties": { "map": { "plan": "data.plan", "company": "data.company" } } } } ``` Out ``` heap.identify("user@example.com"); heap.addUserProperties({ "plan": "premium", "company": "Acme" }) ``` ### User logout reset A user logout calls heap.resetIdentity to clear the identified user from the Heap client. Event ``` { "name": "user logout", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "48453f5ecaf83e33", "trigger": "test", "entity": "user", "action": "logout", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "reset": true } } ``` Out ``` heap.resetIdentity() ``` ## Consent[​](#consent "Direct link to Consent") The destination wires walkerOS `config.consent` to Heap's runtime consent API. `heap.startTracking()` fires when all required consent keys are granted; `heap.stopTracking()` fires otherwise. ``` destinations: { heap: { code: destinationHeap, config: { consent: { analytics: true }, settings: { appId: 'YOUR_HEAP_APP_ID' }, }, }, } ``` --- # Hotjar [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/hotjar) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/hotjar)[ ](https://www.npmjs.com/package/@walkeros/web-destination-hotjar) [Package](https://www.npmjs.com/package/@walkeros/web-destination-hotjar) Beta [Hotjar](https://www.hotjar.com/) provides session recordings, heatmaps, and on-page feedback for web products. This destination forwards walkerOS events to Hotjar via the official [`@hotjar/browser`](https://www.npmjs.com/package/@hotjar/browser) SDK, translating events into `Hotjar.event(...)` calls and handling identity and SPA state changes. Where this fits Hotjar is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-hotjar ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationHotjar } from '@walkeros/web-destination-hotjar'; await startFlow({ destinations: { hotjar: { code: destinationHotjar, config: { settings: { siteId: 1234567, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "hotjar": { "package": "@walkeros/web-destination-hotjar", "config": { "settings": { "siteId": 1234567 } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `siteId*` | `integer` | Your Hotjar site ID (e.g. 1234567). Find it in your Hotjar dashboard under Settings. | | | `hotjarVersion` | `integer` | Hotjar snippet version. Defaults to 6 (current). Override only if Hotjar releases a new version. | | | `debug` | `boolean` | Enable Hotjar debug mode for development troubleshooting. | | | `nonce` | `string` | CSP nonce for the injected Hotjar script tag. Required when using strict Content-Security-Policy. | | | `identify` | `any` | walkerOS mapping value resolving to { userId, ...attributes } for Hotjar.identify(). userId is extracted as the first argument; remaining keys become user attributes. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `identify` | `any` | Per-event identity mapping. Resolves to { userId, ...attributes } -> Hotjar.identify(userId, attributes). | | | `stateChange` | `any` | SPA route change notification. Resolves to a relative path string -> Hotjar.stateChange(path). Used for accurate heatmaps on virtual page views. | | ## Examples ### Combined features A purchase fires Hotjar.identify then the renamed custom event in the canonical execution order. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "e1929c016f614377", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000106, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "completed_purchase", "settings": { "identify": { "map": { "userId": "user.id" } } } } ``` Out ``` hotjar.identify("us3r", {}); hotjar.event("completed_purchase") ``` ### Default event A walker event becomes a Hotjar.event call with the event name as the custom Hotjar event. Event ``` { "name": "product view", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "detail", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [], "consent": { "functional": true }, "id": "cad44796507f04b9", "trigger": "load", "entity": "product", "action": "view", "timestamp": 1700000100, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` hotjar.event("product view") ``` ### Destination identify Destination-level identify calls Hotjar.identify with the user id on every push as Hotjar recommends. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "88aed530cffb9b04", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Out ``` hotjar.identify("us3r", {}); hotjar.event("page view") ``` ### State change An SPA navigation fires Hotjar.stateChange with the new path instead of a Hotjar custom event. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "278a59cee40e47bc", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000105, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "silent": true, "settings": { "stateChange": "data.id" } } ``` Out ``` hotjar.stateChange("/docs/") ``` ### Renamed event A mapping renames the event so the Hotjar custom event uses a canonical name like completed\_purchase. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "fbcd519da02ed83c", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "name": "completed_purchase" } ``` Out ``` hotjar.event("completed_purchase") ``` ### User login identify A user login fires Hotjar.identify with userId and attributes before firing the event. Event ``` { "name": "user login", "data": { "id": "u-123", "email": "jane@example.com", "plan": "premium" }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "2e97bcae83eaee35", "trigger": "test", "entity": "user", "action": "login", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "identify": { "map": { "userId": "data.id", "email": "data.email", "plan": "data.plan" } } } } ``` Out ``` hotjar.identify("u-123", { "email": "jane@example.com", "plan": "premium" }); hotjar.event("user login") ``` ## Consent[​](#consent "Direct link to Consent") Hotjar has no runtime consent API. Consent is gated at the walkerOS level via `config.consent`: ``` destinations: { hotjar: { code: destinationHotjar, config: { consent: { marketing: true }, settings: { siteId: 1234567 }, }, }, } ``` Unconsented events are blocked by the collector before reaching the destination. --- # LinkedIn Insight Tag [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/linkedin) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/destinations/linkedin)[ ](https://www.npmjs.com/package/@walkeros/web-destination-linkedin) [Package](https://www.npmjs.com/package/@walkeros/web-destination-linkedin) The [LinkedIn Insight Tag](https://www.linkedin.com/help/lms/answer/a427660) forwards browser conversions to LinkedIn Campaign Manager for ad optimization, retargeting, and demographic insights. This destination wraps the standard Insight Tag snippet (loaded from `snap.licdn.com/li.lms-analytics/insight.min.js`); there is no npm SDK for the browser tag. Mapped events become `window.lintrk('track', { conversion_id, conversion_value?, currency?, event_id? })` calls. Where this fits LinkedIn Insight Tag is a **web destination** in the walkerOS flow: ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-destination-linkedin ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { destinationLinkedIn } from '@walkeros/web-destination-linkedin'; await startFlow({ consent: { marketing: false }, destinations: { linkedin: { code: destinationLinkedIn, config: { consent: { marketing: true }, loadScript: true, settings: { apiKey: '123456', }, mapping: { order: { complete: { settings: { conversion: { map: { id: { value: 67890 }, value: 'data.total', currency: { key: 'data.currency', value: 'EUR' }, eventId: 'id', }, }, }, }, }, }, }, }, }, }); ``` Add to your `flow.json` destinations: ``` "destinations": { "linkedin": { "package": "@walkeros/web-destination-linkedin", "config": { "consent": { "marketing": true }, "loadScript": true, "settings": { "apiKey": "123456" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see [destination configuration](/docs/destinations/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `apiKey*` | `string` | Your LinkedIn Partner ID (numeric string, typically 6 to 7 digits). Find it in Campaign Manager under Insight Tag. Assigned to window.\_linkedin\_partner\_id before the Insight Tag script loads (like 1234567). | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | ------------ | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `conversion` | `any` | Resolves to a LinkedIn conversion object with short keys: { id (required number — the Campaign Manager conversion\_id), value? (number), currency? (ISO code), eventId? (string for deduplication with a future server/Conversions API destination) }. Events without a resolved id are silently ignored. | | ## Examples ### Lead conversion A demo request fires a LinkedIn lintrk track for a lead conversion with id and event\_id only. Event ``` { "name": "demo request", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "1700000104abcdef", "trigger": "test", "entity": "demo", "action": "request", "timestamp": 1700000104, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "id": { "value": 44444 }, "eventId": "id" } } } } ``` Out ``` lintrk("track", { "conversion_id": 44444, "event_id": "1700000104abcdef" }) ``` ### Ecommerce conversion A completed order fires a LinkedIn lintrk track with conversion\_id, value, currency, and event\_id for deduplication. Event ``` { "name": "order complete", "data": { "id": "0rd3r1d", "currency": "EUR", "shipping": 5.22, "taxes": 73.76, "total": 555 }, "context": { "shopping": [ "complete", 0 ] }, "globals": { "pagegroup": "shop" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "product", "data": { "id": "ers", "name": "Everyday Ruck Snack", "color": "black", "size": "l", "price": 420 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "product", "data": { "id": "cc", "name": "Cool Cap", "size": "one size", "price": 42 }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] }, { "entity": "gift", "data": { "name": "Surprise" }, "context": { "shopping": [ "complete", 0 ] }, "nested": [] } ], "consent": { "functional": true }, "id": "1700000102abcdef", "trigger": "load", "entity": "order", "action": "complete", "timestamp": 1700000102, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "id": { "value": 67890 }, "value": "data.total", "currency": { "key": "data.currency", "value": "EUR" }, "eventId": "id" } } } } ``` Out ``` lintrk("track", { "conversion_id": 67890, "conversion_value": 555, "currency": "EUR", "event_id": "1700000102abcdef" }) ``` ### Key page view A page view fires an explicit lintrk track mapped to a LinkedIn KEY\_PAGE\_VIEW conversion rule. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "walkerOS documentation", "referrer": "https://www.walkeros.io/", "search": "?foo=bar", "hash": "#hash", "id": "/docs/" }, "context": { "dev": [ "test", 1 ] }, "globals": { "pagegroup": "docs" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "1700000103abcdef", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000103, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "id": { "value": 11111 }, "eventId": "id" } } } } ``` Out ``` lintrk("track", { "conversion_id": 11111, "event_id": "1700000103abcdef" }) ``` ### Simple conversion A form submit fires a LinkedIn lintrk track with only a conversion\_id from Campaign Manager. Event ``` { "name": "form submit", "data": { "string": "foo", "number": 1, "boolean": true, "array": [ 0, "text", false ] }, "context": { "dev": [ "test", 1 ] }, "globals": { "lang": "elb" }, "custom": { "completely": "random" }, "user": { "id": "us3r", "device": "c00k13", "session": "s3ss10n" }, "nested": [ { "entity": "child", "data": { "is": "subordinated" } } ], "consent": { "functional": true }, "id": "228dfe5872eaab6a", "trigger": "test", "entity": "form", "action": "submit", "timestamp": 1700000101, "timing": 3.14, "source": { "count": 1, "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9", "type": "collector", "schema": "4" } } ``` Mapping ``` { "settings": { "conversion": { "map": { "id": { "value": 12345 } } } } } ``` Out ``` lintrk("track", { "conversion_id": 12345 }) ``` ## Opt-in conversion model[​](#opt-in-conversion-model "Direct link to Opt-in conversion model") LinkedIn is an **opt-in conversion platform**. Every tracked event must reference a pre-created Conversion Rule in Campaign Manager via its numeric `conversion_id`. Events without an explicit `mapping.settings.conversion` produce zero `lintrk` calls. This is the opposite of analytics destinations (Amplitude, Clarity, PostHog) which forward every event by default. Map only the events that matter for attribution. ## Falsy guards[​](#falsy-guards "Direct link to Falsy guards") The destination protects against misconfigured mappings: * If `id` resolves to falsy or `0` the entire `lintrk` call is skipped (no silent error or partial conversion). * If `value` / `currency` / `eventId` resolve to falsy, those individual fields are dropped from the call. `id` is always required. ## Consent[​](#consent "Direct link to Consent") LinkedIn is an advertising platform. The walkerOS consent key is **`marketing`** (not `analytics`): ``` config: { consent: { marketing: true }, loadScript: true, settings: { apiKey: '123456' }, } ``` The walkerOS `config.consent` gate blocks unconsented events from reaching the destination in the first place. The Insight Tag itself has no vendor opt-out API. When consent is later revoked, walkerOS stops sending events, but the loaded tag stays in the page. ### Deferred script load[​](#deferred-script-load "Direct link to Deferred script load") If you want the Insight Tag to load **only** after consent is granted, set `loadScript: true` and leave `config.consent.marketing` un-granted at init time. The destination will skip `addScript()` during init and inject the script the first time an `on('consent')` event grants `marketing`. The handler is idempotent. If a ` ``` Run directly: ``` node ./dist/bundle.mjs ``` Or with Docker: ``` walkeros run flow.json ``` *** ## Adding destinations[​](#adding-destinations "Direct link to Adding destinations") Add destinations to your `flow.json`: ``` { "version": 4, "flows": { "default": { "config": { "platform": "web", "bundle": { "packages": { "@walkeros/collector": {}, "@walkeros/web-source-browser": {}, "@walkeros/web-destination-api": {}, "@walkeros/web-destination-gtag": {} } } }, "sources": { "browser": { "package": "@walkeros/web-source-browser", "config": { "settings": { "pageview": true } } } }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "https://your-api.com/events" } } }, "ga4": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "ga4": { "measurementId": "G-XXXXXXXXXX" } } } } }, "collector": { "run": true } } } } ``` Rebuild: `walkeros bundle flow.json` *** ## Adding consent[​](#adding-consent "Direct link to Adding consent") Add consent requirements in the destination config: ``` "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "https://your-api.com/events" }, "consent": { "functional": true } } }, "ga4": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "ga4": { "measurementId": "G-XXXXXXXXXX" } }, "consent": { "analytics": true } } } } ``` Set consent at runtime: ``` // After loading walker.js elb('walker consent', { functional: true, analytics: true }); ``` *** ## Key concepts[​](#key-concepts "Direct link to Key concepts") ### The `package:` property[​](#the-package-property "Direct link to the-package-property") In Bundled mode, you reference packages by name: ``` "sources": { "browser": { "package": "@walkeros/web-source-browser" } } ``` The CLI downloads and bundles the package. This differs from Integrated mode where you use `code:` with a direct import. ### Variables and environment[​](#variables-and-environment "Direct link to Variables and environment") Use variables for environment-specific values: ``` { "version": 4, "variables": { "GA_ID": "$env.GA_MEASUREMENT_ID:G-DEFAULT", "API_URL": "$env.API_ENDPOINT:https://api.example.com" }, "flows": { "default": { "destinations": { "ga4": { "config": { "settings": { "ga4": { "measurementId": "$var.GA_ID" } } } } } } } } ``` Environment variables (`$env.NAME:default`) are resolved at build time. Config variables (`$var.name`) reference values from the `variables` section. ### Multiple flows[​](#multiple-flows "Direct link to Multiple flows") Define different flows for different environments: ``` { "version": 4, "flows": { "development": { "config": { "platform": "web" }, "destinations": { "console": { "package": "@walkeros/destination-demo" } } }, "production": { "config": { "platform": "web" }, "destinations": { "ga4": { "package": "@walkeros/web-destination-gtag" } } } } } ``` Build specific flow: `walkeros bundle flow.json --flow production` *** ## CLI commands[​](#cli-commands "Direct link to CLI commands") | Command | Purpose | | -------------------------- | ---------------------------- | | `walkeros bundle` | Build production bundle | | `walkeros push --simulate` | Test with mocked events | | `walkeros push` | Test with real API calls | | `walkeros run` | Start HTTP collection server | See [CLI documentation](/docs/apps/cli.md) for full details. *** ## Docker deployment[​](#docker-deployment "Direct link to Docker deployment") For production server deployments: ``` # Build the bundle walkeros bundle flow.json # Run with Docker docker run -d -p 8080:8080 \ -v $(pwd)/dist/bundle.mjs:/app/flow.mjs \ -e BUNDLE=/app/flow.mjs \ walkeros/flow:latest ``` See [Docker documentation](/docs/apps/docker.md) for complete deployment guides. *** ## See your event[​](#see-your-event "Direct link to See your event") Before you build and deploy, run the event through the flow offline with the named destination mocked. * Web (browser) * Server (Node.js) Using the flow\.json with destinations you created above, `push` builds the flow and runs the event through it with the named destination mocked. Nothing is sent to a vendor. ``` npx walkeros push flow.json \ -e '{"name":"page view"}' \ --simulate destination.ga4 ``` Using the **server** `flow.json` above (which defines the `console` destination), `push` builds and simulates it: ``` npx walkeros push flow.json \ -e '{"name":"page view"}' \ --simulate destination.console ``` `push --simulate` builds the flow, runs the event through it with `destination.console` replaced by a mock, and prints the resolved outcome (`success` and `Duration`); add `--json` to get the same result as JSON. Nothing is sent to a vendor. Event names use a space (`page view`). *** ## Next steps[​](#next-steps "Direct link to Next steps") * [**CLI documentation**](/docs/apps/cli.md): All CLI commands and options * [**Docker deployment**](/docs/apps/docker.md): Production container deployment * [**Mapping**](/docs/mapping/.md): Transform events for destinations *** ## See also[​](#see-also "Direct link to See also") * [**Integrated mode**](/docs/getting-started/modes/integrated.md): Build into your app with TypeScript * [**Collector reference**](/docs/collector/.md): Understanding the collector engine --- # Integrated mode In Integrated mode, walkerOS lives inside your application code. You configure it with TypeScript, and it deploys as part of your app bundle. ## Quickstart[​](#quickstart "Direct link to Quickstart") ### 1. Install the collector[​](#1-install-the-collector "Direct link to 1. Install the collector") ``` npm install @walkeros/collector ``` ### 2. Send your first event[​](#2-send-your-first-event "Direct link to 2. Send your first event") ``` import { startFlow } from '@walkeros/collector'; const { elb } = await startFlow({ destinations: { console: { code: { type: 'console', config: {}, push: (event) => console.log('Event:', event.name), }, }, }, }); await elb('page view', { title: 'Home' }); // -> logs: Event: page view ``` That's it. You just sent your first event and saw it in the console. *** ## Adding destinations[​](#adding-destinations "Direct link to Adding destinations") Install destination packages and add them to your config: ``` npm install @walkeros/web-destination-api @walkeros/web-destination-gtag ``` ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { destinationAPI } from '@walkeros/web-destination-api'; import { destinationGtag } from '@walkeros/web-destination-gtag'; const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true }, }, }, }, destinations: { // Send to your API api: { code: destinationAPI, config: { settings: { url: 'https://your-api.com/events' }, }, }, // Send to Google Analytics 4 ga4: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' }, }, }, }, }, }); ``` *** ## Adding consent[​](#adding-consent "Direct link to Adding consent") Add consent requirements to control which destinations receive events: ``` const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true } }, }, }, destinations: { api: { code: destinationAPI, config: { settings: { url: 'https://your-api.com/events' }, consent: { functional: true }, // Requires functional consent }, }, ga4: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' } }, consent: { analytics: true }, // Requires analytics consent }, }, }, }); // When user accepts consent elb('walker consent', { functional: true, analytics: true }); ``` *** ## Key concepts[​](#key-concepts "Direct link to Key concepts") ### The `code:` Property[​](#the-code-property "Direct link to the-code-property") In Integrated mode, you pass actual code references: ``` sources: { browser: { code: sourceBrowser, // Direct import, not a string }, }, ``` This differs from Bundled mode where you use `package:` with a string reference. ### The `elb` Function[​](#the-elb-function "Direct link to the-elb-function") `startFlow()` returns an `elb` function for tracking events: ``` const { elb } = await startFlow({ ... }); // Track events with entity-action format elb('page view', { title: 'Home' }); elb('product add', { id: 'abc', name: 'Widget', price: 29.99 }); elb('order complete', { total: 99.99, currency: 'USD' }); ``` ### Type safety[​](#type-safety "Direct link to Type safety") Integrated mode gives you full TypeScript support: ``` import type { WalkerOS } from '@walkeros/core'; const { elb } = await startFlow({ // Full autocomplete and type checking }); ``` *** ## Framework examples[​](#framework-examples "Direct link to Framework examples") * React * Next.js ``` // hooks/useWalker.ts import { useEffect, useState } from 'react'; import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; export function useWalker() { const [elb, setElb] = useState(null); useEffect(() => { startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true } } }, }, }).then(({ elb }) => setElb(() => elb)); }, []); return elb; } ``` ``` // app/providers.tsx 'use client'; import { useEffect } from 'react'; import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; export function WalkerProvider({ children }: { children: React.ReactNode }) { useEffect(() => { startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true } } }, }, }); }, []); return <>{children}; } ``` *** ## See your event[​](#see-your-event "Direct link to See your event") The basic setup above logs every event it receives. Run it (Node, or your app's dev server) and push an event: ``` await elb('page view', { title: 'Home' }); // -> logs: Event: page view ``` Integrated mode verifies where it runs: in your console. For an offline CLI test loop with mocked destinations, use [bundled mode](/docs/getting-started/modes/bundled.md#see-your-event). *** ## Next steps[​](#next-steps "Direct link to Next steps") * [**Event Model**](/docs/getting-started/event-model.md): How events are structured * [**Mapping**](/docs/mapping/.md): Transform events for destinations * [**Browser Source**](/docs/sources/web/browser/.md): DOM-based automatic tracking * [**Destinations**](/docs/destinations/.md): Available destination packages *** ## See also[​](#see-also "Direct link to See also") * [**Bundled mode**](/docs/getting-started/modes/bundled.md): Configure with JSON, build with CLI * [**Collector reference**](/docs/collector/.md): Full `startFlow()` API documentation --- # Quickstart Get started with walkerOS in minutes. These examples provide working code that you can copy, paste, and run immediately. All examples are maintained in our [quickstart app](https://github.com/elbwalker/walkerOS/tree/main/apps/quickstart) and tested with every release. This quickstart uses [integrated mode](/docs/getting-started/modes/integrated.md): the collector runs inside your own code. You can switch how you package it later, see [operating modes](/docs/getting-started/modes/.md). Working with an AI assistant? `@walkeros/mcp` runs locally without an account. Install the plugin and describe the flow you want: [With an AI assistant](/docs/apps/mcp.md). ## 1. Install packages[​](#1-install-packages "Direct link to 1. Install packages") Install the collector package from npm: ``` npm install @walkeros/collector @walkeros/web-source-browser ``` ## 2. Create basic setup[​](#2-create-basic-setup "Direct link to 2. Create basic setup") Send your first event with a collector and a single console destination. This first snippet stays DOM-free so it runs in Node; the browser source for automatic DOM capture comes in the next step. ``` import { startFlow } from '@walkeros/collector'; const { elb } = await startFlow({ destinations: { console: { code: { type: 'console', config: {}, push: (event) => console.log('Event:', event.name), }, }, }, }); await elb('page view', { title: 'Home' }); // -> logs: Event: page view ``` ## 3. Add destinations[​](#3-add-destinations "Direct link to 3. Add destinations") Install destination packages and add them to your setup: ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { destinationGtag } from '@walkeros/web-destination-gtag'; import { destinationAPI } from '@walkeros/web-destination-api'; export async function initializeWalker(): Promise { const { collector, elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, elb: 'elb', }, }, }, }, destinations: { // Send events to your API api: { code: destinationAPI, config: { settings: { url: 'https://your-api.com/events', }, }, }, // Send to Google Analytics 4 ga4: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' }, }, }, }, }, }); window.walker = collector; } ``` ## 4. Add consent management[​](#4-add-consent-management "Direct link to 4. Add consent management") Add consent requirements to your destinations: ``` export async function initializeWalker(): Promise { const { collector, elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, elb: 'elb', }, }, }, }, destinations: { // API destination - requires functional consent api: { code: destinationAPI, config: { settings: { url: 'https://your-api.com/events', }, consent: { functional: true, }, }, }, // GA4 - requires analytics consent ga4: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' }, }, consent: { analytics: true, }, }, }, }, }); window.walker = collector; // Set consent when user accepts/denies // This would typically come from your consent banner function handleConsentAccept() { elb('walker consent', { functional: true, analytics: true, marketing: true, }); } function handleConsentDeny() { elb('walker consent', { functional: true, analytics: false, marketing: false, }); } } ``` ## See your event[​](#see-your-event "Direct link to See your event") The step 2 setup runs directly in Node. Save it as `index.mjs` (the import and top-level `await` need ES modules) and run: ``` node index.mjs ``` You should see: ``` Event: page view ``` That is the whole loop: an event pushed with `elb` reaching a destination. The browser setups in steps 3 and 4 verify the same way in your browser console once your app runs. To test flows offline with the CLI instead, see [bundled mode](/docs/getting-started/modes/bundled.md#see-your-event). ## Next steps[​](#next-steps "Direct link to Next steps") * **[Send it to GA4](/docs/getting-started/ga4-ecommerce.md)**: the same event arriving live in GA4 DebugView * **[DOM-based tagging](/docs/sources/web/browser/tagging/html-attributes.md)**: automatic tracking via data-elb attributes * **[Event mapping](/docs/mapping/.md)**: transform events before sending * **[Operating modes](/docs/getting-started/modes/.md)**: choose how to package and deploy --- # Docker ## Try the demo[​](#try-the-demo "Direct link to Try the demo") ``` docker run -p 8080:8080 -e BUNDLE=/app/demos/demo-collect.mjs walkeros/flow:latest ``` Output: ``` INFO [runner] Loading flow from /app/demos/demo-collect.mjs INFO [runner] Flow running INFO [runner] Port: 8080 ``` Test it: ``` curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{"name":"page view","data":{"title":"Test Page"}}' ``` ## HTTP event collection[​](#http-event-collection "Direct link to HTTP event collection") ``` docker run -d -p 8080:8080 \ -e BUNDLE=/app/demos/demo-collect.mjs \ walkeros/flow # Send event curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{"name":"page view","data":{"title":"Home"}}' ``` ## Built-in demos[​](#built-in-demos "Direct link to Built-in demos") The Docker image includes demo bundles at `/app/demos/`: * `demo-collect.mjs` - Server collector demo * `demo-serve.js` - Web bundle demo ## Custom flow[​](#custom-flow "Direct link to Custom flow") ### Build your flow[​](#build-your-flow "Direct link to Build your flow") Docker runtime requires **pre-built bundles**, not JSON configs. **Workflow:** 1. Create a [flow.json](/docs/getting-started/flow/.md) config 2. Bundle with CLI: `walkeros bundle flow.json --output flow.mjs` 3. Run in Docker: `docker run -e BUNDLE=/app/flow.mjs` ### Build custom image[​](#build-custom-image "Direct link to Build custom image") ``` FROM walkeros/flow:latest COPY flow.mjs /app/flow.mjs ENV BUNDLE=/app/flow.mjs ``` ``` docker build -t my-walker . docker run -p 8080:8080 my-walker ``` ## Docker compose[​](#docker-compose "Direct link to Docker compose") ``` version: '3.8' services: walkeros: image: walkeros/flow:latest environment: BUNDLE: /app/demos/demo-collect.mjs ports: - '8080:8080' restart: unless-stopped ``` ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Description | Default | | ------------ | ----------------------------- | -------------- | | `BUNDLE` | Path to pre-bundled flow file | Required | | `PORT` | Server port | 8080 | | `HOST` | Server host | 0.0.0.0 | | `SERVE_NAME` | Filename in URL (serve mode) | walker.js | | `SERVE_PATH` | URL directory (serve mode) | (empty = root) | ## Next steps[​](#next-steps "Direct link to Next steps") * [Available sources](/docs/sources/.md) * [Available destinations](/docs/destinations/.md) * [Docker Package README](https://github.com/elbwalker/walkerOS/tree/main/packages/docker) --- # Next.js Quickstart This guide shows you how to integrate walkerOS into your Next.js application. Most setup is identical to React - check the [React quickstart guide](/docs/getting-started/quickstart/react.md) for more details. ## Installation[​](#installation "Direct link to Installation") Install the required packages: ``` npm install @walkeros/collector @walkeros/web-source-browser ``` ## Setup[​](#setup "Direct link to Setup") ### 1. Walker initialization[​](#1-walker-initialization "Direct link to 1. Walker initialization") Create the walker initialization file: ``` // walker/index.ts import type { Collector, WalkerOS } from '@walkeros/core'; import { startFlow } from '@walkeros/collector'; import { createTagger, sourceBrowser } from '@walkeros/web-source-browser'; // Global type declarations declare global { interface Window { elb: WalkerOS.Elb; walker: Collector.Instance; } } export async function initializeWalker(): Promise { // Skip initialization if already done if (window.walker) return; // Create collector with run: false for manual pageview control const { collector, elb } = await startFlow({ run: false, consent: { functional: true }, sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, elb: 'elb', }, }, }, }, destinations: { // Your destinations - add console for testing console: { code: { type: 'console', config: {}, push(event, data) { console.log('Event:', event, data); }, }, }, }, }); // Set global window object window.walker = collector; } // Tagger helper for easy component tagging const taggerInstance = createTagger(); export function tagger(entity?: string) { return taggerInstance(entity); } ``` ### 2. App integration[​](#2-app-integration "Direct link to 2. App integration") For Next.js applications, use the Next.js router events: ``` // pages/_app.tsx import { useRouter } from 'next/router'; import { useEffect, useRef } from 'react'; import { initializeWalker } from '../walker'; export default function App({ Component, pageProps }) { const router = useRouter(); const hasInitialized = useRef(false); const firstRun = useRef(true); useEffect(() => { // Prevent double execution if (!hasInitialized.current) { initializeWalker(); hasInitialized.current = true; } }, []); useEffect(() => { const handleRouteChange = () => { if (firstRun.current) { firstRun.current = false; return; } window.elb('walker run'); }; // Next.js route events router.events.on('routeChangeComplete', handleRouteChange); // Call on initial load handleRouteChange(); return () => { router.events.off('routeChangeComplete', handleRouteChange); }; }, [router.events]); return ; } ``` For Next.js 13+ with App Router: ``` // app/layout.tsx 'use client'; import { usePathname } from 'next/navigation'; import { useEffect, useRef } from 'react'; import { initializeWalker } from './walker'; export default function RootLayout({ children }) { const pathname = usePathname(); const hasInitialized = useRef(false); const firstRun = useRef(true); useEffect(() => { // Prevent double execution if (!hasInitialized.current) { initializeWalker(); hasInitialized.current = true; } }, []); useEffect(() => { if (firstRun.current) { firstRun.current = false; return; } window.elb('walker run'); }, [pathname]); return ( {children} ); } ``` ## Component usage[​](#component-usage "Direct link to Component usage") Component usage is identical to React. Use the tagger helper and manual events: ``` import { tagger } from '../walker'; function ProductPage({ product }) { // Manual event tracking useEffect(() => { window.elb('product view', { id: product.id, name: product.name, price: product.price, }); }, [product]); return (

{product.name}

); } ``` info When using the `tagger` helper, make sure to call .get() at the end of the chain. ## Testing[​](#testing "Direct link to Testing") During walker initialization, we added a console destination. This will output events to the console. Open your browser's developer console to see tracked events: ``` Event: "product view", { id: 123, name: "Premium Chocolate", price: 12.99 } ``` ## Best practices[​](#best-practices "Direct link to Best practices") 1. **Use tagger**: Clean component tagging with the tagger helper 2. **Minimal Next.js code**: Keep Next.js integration as simple as possible 3. **Trust walkerOS**: No need for custom error handling or state management 4. **Direct integration**: No providers needed, integrate directly in \_app or layout 5. **Test with console**: Use console destination during development ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Events not firing[​](#events-not-firing "Direct link to Events not firing") 1. Check that `window.walker` is set in the browser console 2. Verify data attributes are correctly formatted (use `data-elb` prefix) 3. Ensure walker initialization completed (window\.walker.allowed must be `true`) 4. Check browser console for any error messages ### Route changes not tracked[​](#route-changes-not-tracked "Direct link to Route changes not tracked") 1. Check that `walker run` is called on route changes 2. Ensure firstRun logic is preventing double execution on initial load 3. Verify router event listeners are working correctly ### TypeScript errors[​](#typescript-errors "Direct link to TypeScript errors") Make sure you've added the global type declarations in your walker initialization file. ## Next steps[​](#next-steps "Direct link to Next steps") * Configure [destinations](/docs/destinations/.md) for your analytics tools * Set up [consent management](/docs/guides/consent/.md) * **[Send it to GA4](/docs/getting-started/ga4-ecommerce.md)**: the same event arriving live in GA4 DebugView * **[Operating modes](/docs/getting-started/modes/.md)**: choose how to package and deploy --- # React Quickstart This guide shows you how to integrate walkerOS into your React application with automatic pageview tracking and component-level event tracking. ## Key concepts[​](#key-concepts "Direct link to Key concepts") * **Single file setup**: One initialization file, direct usage in App.tsx * **No providers needed**: Direct integration in your main App component * **StrictMode safe**: Handles React's double execution correctly * **Data attributes**: Use the tagger helper for clean component tracking ## Installation[​](#installation "Direct link to Installation") First, install the required walkerOS packages: ``` npm install @walkeros/core @walkeros/collector @walkeros/web-source-browser ``` ## Setup[​](#setup "Direct link to Setup") ### 1. walker initialization[​](#1-walker-initialization "Direct link to 1. walker initialization") ``` // apps/demos/react/src/walker/index.ts import type { Collector, WalkerOS } from '@walkeros/core'; import { startFlow } from '@walkeros/collector'; import { createTagger, sourceBrowser } from '@walkeros/web-source-browser'; // Global type declarations declare global { interface Window { elb: WalkerOS.Elb; walker: Collector.Instance; } } export async function initializeWalker(): Promise { // Skip initialization if already done if (window.walker) return; // Create collector with run: false for manual pageview control const { collector } = await startFlow({ run: false, consent: { functional: true }, sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, elb: 'elb', }, }, }, }, destinations: { console: { code: { type: 'console', config: {}, push: (event) => console.log('Event:', event.name), }, }, }, }); // Set global window object window.walker = collector; } // Tagger helper for easy component tagging const taggerInstance = createTagger(); export function tagger(entity?: string) { return taggerInstance(entity); } ``` ### 2. App integration[​](#2-app-integration "Direct link to 2. App integration") Integrate directly in your main App component: ``` // apps/demos/react/src/App.tsx import { Routes, Route, useLocation } from 'react-router-dom'; import { useEffect, useRef } from 'react'; import { initializeWalker } from './walker'; function App() { const location = useLocation(); const hasInitialized = useRef(false); const firstRun = useRef(true); useEffect(() => { // Prevent React StrictMode double execution if (!hasInitialized.current) { initializeWalker(); hasInitialized.current = true; } }, []); useEffect(() => { // Skip first run to prevent double page views if (firstRun.current) { firstRun.current = false; return; } window.elb('walker run'); }, [location]); return (
} /> } /> } />
); } export default App; ``` ## Component tagging[​](#component-tagging "Direct link to Component tagging") Use the tagger helper for clean component tracking: ``` // apps/demos/react/src/components/ProductDetail.tsx import { tagger } from '../walker'; function ProductDetail({ productId }: { productId: string }) { const product = getProductById(parseInt(productId)); return (

{product.name}

€{product.price}

); } ``` ``` // Without tagger helper (verbose)
// With tagger helper (clean)
``` info When using the `tagger` helper, make sure to call .get() at the end of the chain. ## Manual event tracking[​](#manual-event-tracking "Direct link to Manual event tracking") Use the `elb` function for custom events: ``` function CheckoutForm() { const handleSubmit = (formData: any) => { window.elb('checkout complete', { total: formData.total, items: formData.items.length, }); }; return
{/* Form fields */}
; } ``` ## Testing[​](#testing "Direct link to Testing") During #1-walker-initialization, we added a console destination. This will output events to the console. Open your browser's developer console to see tracked events: ``` Event: "product view", { id: 123, name: "Premium Chocolate", price: 12.99 } ``` ## Best practices[​](#best-practices "Direct link to Best practices") 1. **Use tagger**: Clean component tagging with the tagger helper 2. **Minimal React code**: Keep React integration as simple as possible 3. **Trust walkerOS**: No need for custom error handling or state management 4. **Direct integration**: No providers needed, integrate directly in App component 5. **Test with console**: Use console destination during development ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Events not firing[​](#events-not-firing "Direct link to Events not firing") 1. Check that `window.walker` is set in the browser console 2. Verify data attributes are correctly formatted (use `data-elb` prefix) 3. Ensure walker initialization completed (window\.walker.allowed must be `true`) 4. Check browser console for any error messages ### Route changes not tracked[​](#route-changes-not-tracked "Direct link to Route changes not tracked") 1. Check that `walker run` is called on route changes in `window.elbLayer` 2. Ensure firstRun logic is preventing double execution on initial load 3. Verify location/pathname dependency is working correctly ## Next steps[​](#next-steps "Direct link to Next steps") * Configure [destinations](/docs/destinations/.md) for your analytics tools * Set up [consent management](/docs/guides/consent/.md) * **[Send it to GA4](/docs/getting-started/ga4-ecommerce.md)**: the same event arriving live in GA4 DebugView * **[Operating modes](/docs/getting-started/modes/.md)**: choose how to package and deploy --- # Stores Stores are the fourth component type in walkerOS, alongside sources, transformers, and destinations. They provide **passive key-value storage** that other components consume, with no event chains and no push functions. ## The Store interface[​](#the-store-interface "Direct link to The Store interface") Every store implements four operations: ``` interface Store.Instance { type: string; config: Store.Config; get(key: string): StoreValue | undefined | Promise; set(key: string, value: StoreValue, ttl?: number): void | Promise; delete(key: string): void | Promise; destroy?(): void | Promise; } ``` Stores can be synchronous or asynchronous. The interface supports both. Stores hold one canonical value type: structured data (`StoreValue`), with binary (`Uint8Array`) as a first-class leaf. A shared core codec serializes that value to and from each backing. By default a store is a structured key-value store. Set `file: true` on a byte-native backend (filesystem, S3, GCS) to make it persist raw bytes byte-exact instead, the mode for serving assets such as walker.js. The Sheets store is structured-only and rejects `file: true`. ## Available implementations[​](#available-implementations "Direct link to Available implementations") | Package | Environment | Storage | Features | | ------------------------------- | ----------- | -------------------- | ----------------------------------- | | `@walkeros/server-store-fs` | Node.js | Filesystem | Read/write files, directory listing | | `@walkeros/server-store-s3` | Node.js | S3-compatible | Object storage via AWS SDK | | `@walkeros/server-store-gcs` | Node.js | Google Cloud Storage | Object storage via GCS JSON API | | `@walkeros/server-store-sheets` | Node.js | Google Sheets | Row-per-key storage via Sheets API | The collector ships a built-in in-memory cache tier (`__cache`). Wire it on any store via `Flow.Store.cache` instead of declaring a separate memory store. ## Configuration[​](#configuration "Direct link to Configuration") ### Bundled mode (flow\.json)[​](#bundled-mode-flowjson "Direct link to Bundled mode (flow.json)") Define stores in the `stores` section of a flow config. Use `cache` to enable the built-in in-memory cache tier on top of the backing store: ``` { "stores": { "files": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./public" } }, "cache": { "rules": [{ "ttl": 60 }] } } } } ``` ### Integrated mode (TypeScript)[​](#integrated-mode-typescript "Direct link to Integrated mode (TypeScript)") ``` import { startFlow } from '@walkeros/collector'; import { storeFsInit } from '@walkeros/server-store-fs'; const { collector } = await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './public' } }, cache: { rules: [{ ttl: 60 }] }, }, }, }); // Access stores on the collector collector.stores.files.set('key', 'value'); const val = await collector.stores.files.get('key'); ``` ## Connecting stores to components[​](#connecting-stores-to-components "Direct link to Connecting stores to components") Use the `$store.` prefix in a component's `env` to inject a store instance at runtime: ``` { "stores": { "files": { "package": "@walkeros/server-store-fs" } }, "transformers": { "fingerprint": { "package": "@walkeros/server-transformer-fingerprint", "env": { "store": "$store.files" } } } } ``` The bundler resolves `$store.cache` to the initialized store instance. Invalid references (pointing to undefined stores) are caught at build time. ## Lifecycle[​](#lifecycle "Direct link to Lifecycle") Stores follow a strict lifecycle relative to other components: 1. **Stores initialize first**: before sources, transformers, and destinations 2. **Stores destroy last**: after all other components have shut down This guarantees stores are available when any component starts and remain available while they gracefully shut down. ### Setup lifecycle (optional)[​](#setup-lifecycle-optional "Direct link to Setup lifecycle (optional)") Stores may also implement an optional `setup()` lifecycle for one-time, operator-time provisioning, for example creating SQLite tables or an S3 bucket. Setup runs only when an operator explicitly invokes `walkeros setup store.`; the runtime never auto-invokes it. Opt in via `config.setup` in the flow config (`true` or an object). See [Setup lifecycle](/docs/destinations/create-your-own.md#setup-lifecycle-optional) on the destinations page for the full concept, and [walkeros setup CLI command](/docs/apps/cli.md#setup-command) for the operator-facing command. ## Next steps[​](#next-steps "Direct link to Next steps") * **[Flow configuration](/docs/getting-started/modes/bundled.md)** - Stores in flow\.json * **[File transformer](/docs/transformers/file.md)** - Serve files from a store * **[Event cache](/docs/collector/cache.md)** - Response caching with stores * **[Docker: Including files](/docs/apps/docker.md#including-files)** - Bake store data into Docker images --- # Guides Practical guides and best practices to help you implement and optimize walkerOS in real-world scenarios. From consent management to user identification, these guides provide step-by-step instructions and proven patterns for common use cases. ## Available guides[​](#available-guides "Direct link to Available guides") * **[Consent management](/docs/guides/consent/.md)** - handle user consent across different CMP providers * **[User stitching](/docs/guides/user-stitching.md)** - connect user sessions and identities across touchpoints * **[Migration](/docs/guides/migration.md)** - how to migrate to the latest walkerOS packages --- # Consent management Managing user consent is critical for legal compliance and user privacy. WalkerOS follows a **privacy-by-design** approach and offers multiple ways to handle consent and manage data processing to ensure the user's privacy is respected. warning We don't give any legal advice. Make sure to consult a legal expert to ensure that your implementation is compliant with the laws and regulations that apply to your business. Typically, a Consent Management Platform (CMP) handles the consent. This is an asynchronous process. The CMP has to be loaded, a user has to make a choice, and/or the state has to be updated. walkerOS provides [CMP source packages](/docs/sources/web/cmps/.md) that automate this. They listen for CMP events and call the consent command for you. To set/update the consent state, the CMP should actively push the `consent command` with a group or an individual tool's permission state (`true/false`). If only one condition applies, consent is granted. Updating only one value won't override existing states. ``` elb('walker consent', { functional: true, marketing: false }); ``` Consent keys can be defined arbitrarily (typically known as *functional*, *analytics*, and *marketing*). However, you can also use individual names for each vendor. The walker handles the **race conditions**: Previously pushed events get processed in the correct order after granting the consent state with the destinations as well as new ones. ### Destinations[​](#destinations "Direct link to Destinations") With each event, the consent states get checked. Every event will be added to an ordered queue if consent isn't granted yet and the destination's queue isn't disabled. The queue resets with each `walker run` command. And will be (re-)processed with each `walker consent` update. The key must match the key used in each `destination.config.consent`. note The queued events properties `consent`, `globals`, and `user` will be **updated** to the current state before being processed. note A destination only requires one granted consent state to process events. To revoke consent and stop sharing events with a destination, all matching rules have to be set to `false`: ``` elb('walker consent', { marketing: false }); ``` ## on consent[​](#on-consent "Direct link to on consent") To start walker.js only after a consent choice, use the `on consent` command: ``` // Wait for functional consent to run elb('walker on', { type: 'consent', rules: { functional: () => { elb('walker run'); }, }, }); // Later, a CMP has to update the consent state elb('walker consent', { functional: true }); ``` Another example of how to use the `on consent` command might be to update [Googles consent mode v2](https://developers.google.com/tag-platform/security/guides/consent?consentmode=basic): ``` elb('walker on', { type: 'consent', rules: (consent, context) => { // Check if marketing consent is granted if (consent.marketing) { gtag('consent', 'update', { ad_user_data: 'granted', ad_personalization: 'granted', ad_storage: 'granted', analytics_storage: 'granted', }); } }, }); ``` The consent mode will be updated via `gtag` as soon as marketing consent is granted. --- # My events aren't arriving When a destination stays silent, you do not need a server or a login to find out why. Run your flow against a single event offline and read the result. ``` npx walkeros push flow.json -e '{"name":"product add","data":{"id":"ers","price":420}}' --simulate destination.gtag ``` The simulation reports whether the destination accepted the event: ``` Simulating destination: gtag success: true Duration: 4302ms ``` Add `--json` to get the same result as machine-readable output (`{ "success": true, "duration": 1104 }`). Your `flow.json` must carry `config.platform` (`"web"` or `"server"`), otherwise bundling throws before the simulation runs. ## Symptom to cause[​](#symptom-to-cause "Direct link to Symptom to cause") | Symptom | Likely cause | | ------------------------ | ---------------------------- | | No events at all | Source not initialized | | A destination is silent | Mapping mismatch | | Partial or wrong data | Path does not exist | | A destination is skipped | Required consent not granted | | A destination throws | Vendor SDK or API error | ## Log every event at the collector[​](#log-every-event-at-the-collector "Direct link to Log every event at the collector") To confirm events are flowing before you blame a vendor, add an inline console destination. It logs the name of every event the collector processes, so you can see what arrives and when. ``` const { elb } = await startFlow({ destinations: { debug: { code: { type: 'console', config: {}, push: (event) => console.log('Event:', event.name), }, }, // ...your other destinations }, }); ``` If you see event names in the console but the real destination stays silent, the problem is downstream of the collector: a mapping mismatch, a missing consent, or a vendor error. If you see nothing, the source is not capturing or the collector never started. ## Checklist[​](#checklist "Direct link to Checklist") 1. Any errors in the browser console? 2. Is the event pushed? Add the console destination above to confirm. 3. Does the destination mapping match the entity and action names exactly? Names are case-sensitive (`product add`, not `Product Add`). 4. Do the mapped data paths exist on the event? Log the full event and check. 5. Is the required consent granted for the destination? 6. Run `npx walkeros validate flow.json` to catch a malformed config. ## Zero-install equivalent (AI assistant via MCP)[​](#zero-install-equivalent-ai-assistant-via-mcp "Direct link to Zero-install equivalent (AI assistant via MCP)") If you use the walkerOS MCP server with an AI assistant, the same offline simulation is one tool call. `step` is required and uses the `type.name` form (for example `destination.gtag`). ``` flow_simulate({ configPath: 'flow.json', step: 'destination.gtag', event: { name: 'product add', data: { id: 'ers', price: 420 } }, }) ``` The response includes `_hints.warnings`, which flags when no destination received the event. --- # Deploy walkerOS to Bunny Magic Containers This guide deploys a walkerOS event collector to [Bunny Magic Containers](https://bunny.net/magic-containers/), forwarding events to BigQuery. We use GitHub Container Registry (GHCR) to store the Docker image privately. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [GitHub account](https://github.com/) (for private container registry) * [Bunny.net account](https://bunny.net/) with Magic Containers access * [Docker](https://docs.docker.com/get-docker/) installed * [Node.js 18+](https://nodejs.org/) installed ## 1. Set Up BigQuery[​](#1-set-up-bigquery "Direct link to 1. Set Up BigQuery") Follow the [GCP BigQuery setup](/docs/destinations/server/gcp.md#gcp-setup) to: * Create a BigQuery dataset * Create a service account with write permissions * [Download a service account key](/docs/destinations/server/gcp.md#service-account-key) (`sa-bigquery.json`) caution The key file will be baked into your Docker image. Keep the image in a **private** registry only. ## 2. Bundle the Flow[​](#2-bundle-the-flow "Direct link to 2. Bundle the Flow") Install the CLI and bundle the flow: ``` npm install -g @walkeros/cli # Set your environment variables export GCP_PROJECT_ID="your-project-id" export BQ_DATASET="walkerOS" export BQ_TABLE="events" export BQ_LOCATION="EU" # Use the container path - variables are substituted at bundle time export SA_KEY_PATH="/app/sa-bigquery.json" # Bundle the flow walkeros bundle https://www.walkeros.io/flows/gcp-bigquery.json ``` This creates `dist/bundle.mjs` with the credentials path baked in. Quick Local Test Test your flow before building Docker using `push --simulate` with a local credentials path: ``` export SA_KEY_PATH="./sa-bigquery.json" walkeros push https://www.walkeros.io/flows/gcp-bigquery.json \ -e '{"name": "test", "data": {}}' --simulate destination.bigquery ``` ## 3. Build Docker Image[​](#3-build-docker-image "Direct link to 3. Build Docker Image") Create a `Dockerfile`: ``` FROM walkeros/flow:latest # Copy pre-built flow bundle COPY dist/bundle.mjs /app/flow/bundle.mjs # Copy service account credentials COPY sa-bigquery.json /app/sa-bigquery.json # Configure runtime ENV FLOW=/app/flow/bundle.mjs EXPOSE 8080 ``` Build the image with your GitHub Container Registry tag: ``` # Replace YOUR_GITHUB_USERNAME with your GitHub username (lowercase) docker build -t ghcr.io/YOUR_GITHUB_USERNAME/walkeros-collector:latest . ``` ## 4. Test Locally[​](#4-test-locally "Direct link to 4. Test Locally") Run the container locally to verify everything works: ``` docker run --rm -p 8080:8080 ghcr.io/YOUR_GITHUB_USERNAME/walkeros-collector:latest ``` In another terminal, send a test event: ``` curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Local Test", "path": "/test" } }' ``` Expected response: ``` { "success": true, "timestamp": 1234567890 } ``` Verify in BigQuery: ``` bq query --use_legacy_sql=false \ "SELECT * FROM walkerOS.events ORDER BY timestamp DESC LIMIT 5" ``` ## 5. Push to Private GHCR[​](#5-push-to-private-ghcr "Direct link to 5. Push to Private GHCR") ### Create GitHub Personal Access Token[​](#create-github-personal-access-token "Direct link to Create GitHub Personal Access Token") 1. Go to [GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)](https://github.com/settings/tokens) 2. Click **Generate new token (classic)** 3. Name it `ghcr-walkeros` 4. Select scopes: * `write:packages` (to push images) * `read:packages` (to pull images) * `delete:packages` (optional, for cleanup) 5. Click **Generate token** 6. Copy the token (you won't see it again) ### Login and Push[​](#login-and-push "Direct link to Login and Push") ``` # Login to GHCR echo YOUR_GITHUB_TOKEN | docker login ghcr.io -u YOUR_GITHUB_USERNAME --password-stdin # Push the image docker push ghcr.io/YOUR_GITHUB_USERNAME/walkeros-collector:latest ``` ### Verify Image is Private[​](#verify-image-is-private "Direct link to Verify Image is Private") 1. Go to `https://github.com/YOUR_GITHUB_USERNAME?tab=packages` 2. Find `walkeros-collector` 3. Confirm visibility is **Private** (default for new packages) tip If the package is public, click **Package settings** → **Change visibility** → **Private**. ## 6. Connect GHCR to Bunny[​](#6-connect-ghcr-to-bunny "Direct link to 6. Connect GHCR to Bunny") ### Create Read-Only Token for Bunny[​](#create-read-only-token-for-bunny "Direct link to Create Read-Only Token for Bunny") For security, create a separate token with read-only access: 1. Go to [GitHub Settings → Personal access tokens](https://github.com/settings/tokens) 2. Generate new token with only `read:packages` scope 3. Name it `bunny-ghcr-readonly` ### Add Registry to Bunny[​](#add-registry-to-bunny "Direct link to Add Registry to Bunny") 1. Log in to [bunny.net dashboard](https://dash.bunny.net/) 2. Navigate to **Magic Containers** → **Image Registries** 3. Click **Add Image Registry** 4. Select **GitHub** from the dropdown 5. Enter: * **Username**: Your GitHub username * **Personal Access Token**: The read-only token from above 6. Click **Save** ## 7. Deploy to Bunny Magic Containers[​](#7-deploy-to-bunny-magic-containers "Direct link to 7. Deploy to Bunny Magic Containers") ### Create New App[​](#create-new-app "Direct link to Create New App") 1. In Bunny dashboard, go to **Magic Containers** 2. Click **Add App** 3. Enter app name: `walkeros-collector` 4. Select deployment type: * **Magic** (recommended) - AI auto-scales across regions * **Single Region** - Deploy to one region 5. Click **Next** ### Add Container[​](#add-container "Direct link to Add Container") 1. Click **Add Container** 2. Enter container name: `collector` 3. Select your GitHub registry 4. Select image: `walkeros-collector` 5. Select tag: `latest` 6. Click **Next** ### Configure Endpoint[​](#configure-endpoint "Direct link to Configure Endpoint") 1. Click **Add New Endpoint** 2. Enter endpoint name: `collect` 3. Select exposure method: * **CDN** - Routes through Bunny CDN (recommended for caching static responses) * **Anycast** - Direct routing to container ($2/month for anycast IP) 4. Set container port: `8080` 5. Enable SSL 6. Click **Next** ### Deploy[​](#deploy "Direct link to Deploy") 1. Review your configuration 2. Click **Confirm and Create** 3. Wait for deployment (typically 1-2 minutes) 4. Note the endpoint URL (e.g., `https://walkeros-collector-xxxxx.b-cdn.net`) ## 8. Verify Deployment[​](#8-verify-deployment "Direct link to 8. Verify Deployment") ### Health Check[​](#health-check "Direct link to Health Check") ``` curl https://YOUR_BUNNY_ENDPOINT/health ``` ### Send Test Event[​](#send-test-event "Direct link to Send Test Event") ``` curl -X POST https://YOUR_BUNNY_ENDPOINT/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Bunny Production Test", "path": "/" } }' ``` ### Query BigQuery[​](#query-bigquery "Direct link to Query BigQuery") ``` bq query --use_legacy_sql=false \ "SELECT name, data, timestamp FROM walkerOS.events ORDER BY timestamp DESC LIMIT 5" ``` ## Updating Your Deployment[​](#updating-your-deployment "Direct link to Updating Your Deployment") When you need to update your flow or configuration: ``` # 1. Make changes and rebuild docker build -t ghcr.io/YOUR_GITHUB_USERNAME/walkeros-collector:latest . # 2. Push new image docker push ghcr.io/YOUR_GITHUB_USERNAME/walkeros-collector:latest # 3. In Bunny dashboard: Magic Containers → Your App → Redeploy ``` ## Next Steps[​](#next-steps "Direct link to Next Steps") * [Configure event mapping](/docs/mapping/.md) to transform events before sending to BigQuery * [Add more destinations](/docs/destinations/.md) to forward events to multiple services * Set up [Bunny monitoring](https://docs.bunny.net/docs/magic-containers-overview) for your deployment ## Cleanup[​](#cleanup "Direct link to Cleanup") To remove all created resources: * Delete the Bunny Magic Container app from the dashboard * Delete the GHCR image from your GitHub packages * [Clean up BigQuery resources](/docs/destinations/server/gcp.md#cleanup) * Delete local files (`dist/bundle.mjs`, `sa-bigquery.json`, `Dockerfile`) --- # Deploy walkerOS to GCP Cloud Run This guide deploys a walkerOS flow to Google Cloud Run, forwarding events to BigQuery. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [Docker](https://docs.docker.com/get-docker/) installed (for local testing) * [Node.js 18+](https://nodejs.org/) installed ## 1. Set Up BigQuery[​](#1-set-up-bigquery "Direct link to 1. Set Up BigQuery") Follow the [GCP BigQuery setup](/docs/destinations/server/gcp.md#gcp-setup) to: * Create a BigQuery dataset * [Create a service account](/docs/destinations/server/gcp.md#create-service-account) with write permissions * [Create the events table](/docs/destinations/server/gcp.md#create-table-query) For local testing, [download a service account key](/docs/destinations/server/gcp.md#service-account-key). Production uses [Workload Identity](/docs/destinations/server/gcp.md#workload-identity). ### Enable Cloud Run API[​](#enable-cloud-run-api "Direct link to Enable Cloud Run API") ``` gcloud services enable run.googleapis.com ``` ## 2. Bundle the Flow[​](#2-bundle-the-flow "Direct link to 2. Bundle the Flow") Install the CLI and bundle the flow with a Dockerfile: ``` npm install -g @walkeros/cli # Set your environment variables export GCP_PROJECT_ID="your-project-id" export BQ_DATASET="walkerOS" export BQ_TABLE="events" export BQ_LOCATION="EU" # Bundle the flow and generate Dockerfile walkeros bundle https://www.walkeros.io/flows/gcp-bigquery.json --dockerfile ``` This creates `dist/bundle.mjs` and `dist/Dockerfile`. Variable substitution The flow uses `${VARIABLE}` syntax. During bundling, these are replaced with your environment variable values. Required: `GCP_PROJECT_ID`. Optional with defaults: `BQ_DATASET` (walkerOS), `BQ_TABLE` (events), `BQ_LOCATION` (EU). ## 3. Test Locally with Docker[​](#3-test-locally-with-docker "Direct link to 3. Test Locally with Docker") Before deploying, verify everything works locally: ``` # Run locally (uses key file for auth) docker run --rm \ -p 8080:8080 \ -v $(pwd)/dist/bundle.mjs:/app/flow/bundle.mjs:ro \ -v $(pwd)/sa-bigquery.json:/app/sa-bigquery.json:ro \ -e GOOGLE_APPLICATION_CREDENTIALS=/app/sa-bigquery.json \ walkeros/flow:next ``` In another terminal, send a test event: ``` curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Test Page", "path": "/test" } }' ``` Expected response: ``` { "success": true, "timestamp": 1234567890 } ``` Check BigQuery for the event: ``` bq query --use_legacy_sql=false \ "SELECT * FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5" ``` ## 4. Deploy to Cloud Run[​](#4-deploy-to-cloud-run "Direct link to 4. Deploy to Cloud Run") The Dockerfile was generated by `--dockerfile`. Cloud Run uses Workload Identity, so no credentials file is needed in the image. Deploy with the service account attached: ``` cd dist gcloud run deploy walkeros-flow \ --source . \ --port 8080 \ --region europe-west1 \ --allow-unauthenticated \ --service-account=walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com \ --memory 512Mi \ --cpu 1 \ --min-instances 0 \ --max-instances 10 ``` Note the service URL from the output (e.g., `https://walkeros-flow-xxx-ew.a.run.app`). ## 5. Verify Deployment[​](#5-verify-deployment "Direct link to 5. Verify Deployment") ### Health Check[​](#health-check "Direct link to Health Check") ``` curl https://walkeros-flow-xxx-ew.a.run.app/health ``` ### Send Test Event[​](#send-test-event "Direct link to Send Test Event") ``` curl -X POST https://walkeros-flow-xxx-ew.a.run.app/collect \ -H "Content-Type: application/json" \ -d '{ "name": "page view", "data": { "title": "Production Test", "path": "/" } }' ``` ### Query BigQuery[​](#query-bigquery "Direct link to Query BigQuery") ``` bq query --use_legacy_sql=false \ "SELECT name, data, timestamp FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5" ``` ## Next Steps[​](#next-steps "Direct link to Next Steps") * [Configure event mapping](/docs/mapping/.md) to transform events before sending to BigQuery * [Add more destinations](/docs/destinations/.md) to forward events to multiple services ## Cleanup[​](#cleanup "Direct link to Cleanup") To remove all created resources: * Delete the Cloud Run service (`gcloud run services delete walkeros-flow --region europe-west1`) * [Clean up BigQuery resources](/docs/destinations/server/gcp.md#cleanup) * Delete local files (`dist/`, `sa-bigquery.json`) --- # Migration guide: from @elbwalker to @walkerOS This guide helps you migrate from the old elbwalker packages to the new walkerOS packages. ## Quick reference[​](#quick-reference "Direct link to Quick reference") ### Core package changes[​](#core-package-changes "Direct link to Core package changes") * `@elbwalker/walker.js` → `@walkeros/collector` + `@walkeros/web-source-browser` * `@elbwalker/types` + `@elbwalker/utils` → `@walkeros/core` * `@elbwalker/destination-web-*` → `@walkeros/web-destination-*` * `@elbwalker/destination-node-*` → `@walkeros/server-destination-*` ## Step-by-step migration[​](#step-by-step-migration "Direct link to Step-by-step migration") ### 1. Update dependencies[​](#1-update-dependencies "Direct link to 1. Update dependencies") **Before:** ``` { "dependencies": { "@elbwalker/walker.js": "^2.0.0", "@elbwalker/destination-web-google-ga4": "^2.0.0" } } ``` **After:** ``` { "dependencies": { "@walkeros/collector": "^0.0.7", "@walkeros/web-source-browser": "^0.0.7", "@walkeros/web-destination-gtag": "^0.0.7" } } ``` ### 2. Update imports[​](#2-update-imports "Direct link to 2. Update imports") **Before:** ``` import { Walkerjs } from '@elbwalker/walker.js'; import { destinationGoogleGA4 } from '@elbwalker/destination-web-google-ga4'; import type { WalkerOS } from '@elbwalker/types'; ``` **After:** ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { destinationGtag } from '@walkeros/web-destination-gtag'; import type { WalkerOS } from '@walkeros/core'; ``` ### 3. Update function calls[​](#3-update-function-calls "Direct link to 3. Update function calls") **Before:** ``` const walker = Walkerjs({ destinations: [destinationGoogleGA4], }); ``` **After:** ``` const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { scope: document.body }, }, }, }, destinations: { gtag: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' }, }, }, }, }, }); ``` ## Breaking changes[​](#breaking-changes "Direct link to Breaking changes") ### 1. Unified collector[​](#1-unified-collector "Direct link to 1. Unified collector") * Single `@walkeros/collector` package works across all platforms * Use `startFlow()` instead of `Walkerjs()` or `createSourceNode()` ### 2. Modular sources[​](#2-modular-sources "Direct link to 2. Modular sources") * DOM tracking is now `@walkeros/web-source-browser` * Import and initialize sources separately ### 3. Core package merger[​](#3-core-package-merger "Direct link to 3. Core package merger") * All types and utilities now come from `@walkeros/core` * Update all import statements accordingly ### 4. Google destinations unified[​](#4-google-destinations-unified "Direct link to 4. Google destinations unified") * GA4, Google Ads, and GTM are now unified in `@walkeros/web-destination-gtag` * Configure all Google services through a single destination ## Destination configuration patterns[​](#destination-configuration-patterns "Direct link to Destination configuration patterns") ### Unified configuration approach[​](#unified-configuration-approach "Direct link to Unified configuration approach") Configure all sources and destinations in a single `startFlow` call: ``` const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { scope: document.body, session: true }, }, }, }, destinations: { gtag: { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXXXXXXXXX' }, }, }, }, }, }); ``` **Benefits:** * Single configuration describing entire tracking setup * Full type safety with proper config linking * No manual destination registration needed * Clean separation between code and configuration ## Migration checklist[​](#migration-checklist "Direct link to Migration checklist") ### Package dependencies[​](#package-dependencies "Direct link to Package dependencies") * \[ ] Remove old `@elbwalker/*` packages * \[ ] Add `@walkeros/collector` * \[ ] Add required sources and destinations * \[ ] Add `@walkeros/core` for custom implementations ### Code updates[​](#code-updates "Direct link to Code updates") * \[ ] Update all imports to new packages * \[ ] Replace collector initialization with `startFlow()` * \[ ] Update type imports to use `@walkeros/core` * \[ ] Test functionality after migration ## Common issues[​](#common-issues "Direct link to Common issues") **"Cannot find module @walkeros/web-collector"**
Solution: Use `@walkeros/collector` + `@walkeros/web-source-browser` **"elb is not defined"**
Solution: `elb` is returned by `startFlow()` function **TypeScript cannot find types**
Solution: Import types from `@walkeros/core` ## data-elbaction vs data-elbactions[​](#data-elbaction-vs-data-elbactions "Direct link to data-elbaction vs data-elbactions") ### Breaking change in @walkeros packages[​](#breaking-change-in-walkeros-packages "Direct link to Breaking change in @walkeros packages") The behavior of `data-elbaction` has changed to improve performance and provide more precise entity targeting: * **@elbwalker behavior**: `data-elbaction` applied to **all entities** in the DOM hierarchy * **@walkeros behavior**: `data-elbaction` applies to **nearest entity only** ### Migration strategy[​](#migration-strategy "Direct link to Migration strategy") #### Option 1: keep current behavior (recommended for migration)[​](#option-1-keep-current-behavior-recommended-for-migration "Direct link to Option 1: keep current behavior (recommended for migration)") Replace `data-elbaction` with `data-elbactions` to maintain the same behavior: ```
``` #### Option 2: use new behavior (recommended for new projects)[​](#option-2-use-new-behavior-recommended-for-new-projects "Direct link to Option 2: use new behavior (recommended for new projects)") Use `data-elbaction` for more precise, performance-optimized tracking: ```
``` ### When to use each[​](#when-to-use-each "Direct link to When to use each") **Use `data-elbaction` (nearest entity) when:** * You want precise tracking of only the specific entity * Performance is critical (fewer events) * Implementing new features **Use `data-elbactions` (all entities) when:** * Migrating from @elbwalker and need same behavior * You need context from parent entities * Analytics requires full DOM hierarchy ### Tagger API[​](#tagger-api "Direct link to Tagger API") The tagger also provides both methods: ``` // Nearest entity only (data-elbaction) tagger().action('click', 'select').get(); // All entities (data-elbactions) tagger().actions('click', 'select').get(); ``` ## visible vs impression triggers[​](#visible-vs-impression-triggers "Direct link to visible vs impression triggers") ### Breaking change in @walkeros packages[​](#breaking-change-in-walkeros-packages-1 "Direct link to Breaking change in @walkeros packages") The visibility trigger names have been updated to better reflect their behavior: * **`visible` trigger**: Now fires **multiple times** when element re-enters viewport (was `visibles`) * **`impression` trigger**: Fires **once only** when element first becomes visible (was `visible`) ### Migration strategy[​](#migration-strategy-1 "Direct link to Migration strategy") #### Option 1: update trigger names (recommended)[​](#option-1-update-trigger-names-recommended "Direct link to Option 1: update trigger names (recommended)") Update your HTML to use the new trigger names: ```
Single fire
Multiple fires
Single fire
Multiple fires
``` #### Option 2: understand the behavior change[​](#option-2-understand-the-behavior-change "Direct link to Option 2: understand the behavior change") If you keep using the old names, understand the behavior has changed: * Old `visible` behavior (single-fire) → Now use `impression` * Old `visibles` behavior (multiple-fire) → Now use `visible` ### When to use each[​](#when-to-use-each-1 "Direct link to When to use each") **Use `impression` trigger when:** * You want to track when content is first seen * Measuring ad impressions or content views * One-time engagement metrics **Use `visible` trigger when:** * You want to track repeated interactions * Measuring scroll behavior or re-engagement * Analytics requires multiple visibility events ### Tagger API[​](#tagger-api-1 "Direct link to Tagger API") The tagger supports both trigger types: ``` // Single impression (fires once) tagger().action('impression', 'view').get(); // Multiple visibility (fires each time visible) tagger().action('visible', 'track').get(); ``` --- # Reference Syntax walkerOS flow configs (flow\.json) support dynamic references inside string values. They let you pull in variables, environment values, contract fragments, store instances, secrets, and inline JavaScript without duplicating configuration. ## The one rule[​](#the-one-rule "Direct link to The one rule") walkerOS uses **two separator characters**, each with a clear meaning: | Separator | Meaning | | --------- | ----------------------------------------------------------------------------------------------- | | **`.`** | A key or path. The resolver looks up what follows in a map or walks it as a dotted path. | | **`:`** | A literal value or raw-code payload. The resolver uses what follows verbatim, it is not a name. | Every reference follows that rule. There are no exceptions. ## All references[​](#all-references "Direct link to All references") ``` $var.name look up variables[name] (whole-string returns native type) $var.name.deep.path walk a dotted path inside the variable $env.NAME read process.env[NAME] $env.NAME:default read process.env[NAME] with a literal fallback $contract.name inject a whole named contract entry $contract.name.events walk the contract path $store.id inject a store instance (resolved at bundle time) $secret.NAME inject a secret value (resolved at deploy time) $code:(event) => ... inline JavaScript (resolved at bundle time) ``` ## `$var.` — variables[​](#var--variables "Direct link to var--variables") Look up a value in the `variables` map at config, flow, or step level. Step overrides flow overrides config. Variables hold any JSON value, scalars (string, number, boolean), objects, arrays, or whole mapping templates. The reference may include a deep path that walks the value. ``` { "variables": { "measurementId": "G-XXXXXX", "ga4Items": { "loop": ["nested", { "map": { "item_id": "data.id" } }] }, "api": { "version": "v2", "url": "https://api.example.com" } }, "flows": { "web": { "destinations": { "gtag": { "config": { "settings": { "ga4": { "measurementId": "$var.measurementId" } }, "mapping": { "order": { "complete": { "data": { "map": { "items": "$var.ga4Items" } } } } } } }, "api": { "config": { "settings": { "endpoint": "$var.api.url", "version": "$var.api.version" } } } } } } } ``` **Resolution rules:** * A whole-string reference (`"$var.api.url"`) replaces the value with the variable's native type. Object, array, number, and boolean values are preserved. * An inline reference (`"Bearer $var.token"`) substitutes a scalar mid-string. Object or array values throw, scalars only. * Variables may reference other variables. Resolution is recursive with cycle detection. **Naming convention:** when variables hold structures (mapping templates, matchers, consent objects), pick a name that signals the shape, for example `ga4ItemMapping`, `ga4ItemsLoop`, `consentMarketing`, `getMastertagJs`. Keep the convention consistent within a single config; we don't enforce a global prefix or suffix rule. ## `$env.` — environment variables[​](#env--environment-variables "Direct link to env--environment-variables") Read from `process.env` at runtime. Optionally provide a literal fallback with the `:default` suffix. ``` { "variables": { "apiUrl": "$env.API_URL:http://localhost:8080/collect", "pixelId": "$env.META_PIXEL_ID" } } ``` The `:` here is the **default-value separator**, not a name delimiter. The token after `:` is used verbatim. Use `$env.` for non-secret config (IDs, hostnames, regions). For credentials, tokens, and private keys managed by the secret manager, use `$secret.`: the deploy pipeline only injects values referenced as `$secret.NAME`. ## `$contract.` — contract references[​](#contract--contract-references "Direct link to contract--contract-references") Inject a fragment of a named contract entry. Same whole-string and deep-path rules as `$var.`. The contract is fully resolved (extend + wildcards) before path access, so the value you get is the merged shape. See [Contract](/docs/getting-started/flow/contract.md) for the full structure. ``` { "contract": { "default": { "schema": { "properties": { "globals": { "required": ["country"] } } }, "events": { "product": { "add": { "properties": { "data": { "required": ["id", "name"] } } } } } } }, "flows": { "web": { "destinations": { "ga4": { "validate": { "events": "$contract.default.events", "schema": "$contract.default.schema" } } } } } } ``` ## `$store.` — store wiring[​](#store--store-wiring "Direct link to store--store-wiring") Inject a store instance declared under `stores`. Wired through a component's `env` so the bundler can resolve to the real store at build time. ``` { "flows": { "server": { "stores": { "cache": { "package": "@walkeros/server-store-fs" } }, "sources": { "http": { "package": "@walkeros/server-source-http", "env": { "store": "$store.cache" } } } } } } ``` Store IDs are plain identifiers (camelCase). The `.` is the separator, not a path walker, store IDs do not contain dots. ## `$secret.` — secrets[​](#secret--secrets "Direct link to secret--secrets") Inject a secret resolved at deploy time. Secrets are stored outside the config and never written into customer databases alongside the flow. ``` { "flows": { "web": { "destinations": { "api": { "config": { "settings": { "headers": { "Authorization": "Bearer $secret.API_TOKEN" } } } } } } } } ``` Secret names are uppercase: `[A-Z0-9_]+`. The `.` is the separator, secrets have no internal structure. Use `$secret.` for credentials, tokens, and private keys (for example a service account JSON on `config.credentials`). The app's deploy pipeline detects `$secret.NAME` references and injects the managed secret value into the server runner's environment at runtime. A managed secret referenced as `$env.NAME` is **not** injected, so reach for `$secret.` whenever the value comes from the secret manager. Secrets resolve on server flows only. ## `$code:` — inline code[​](#code--inline-code "Direct link to code--inline-code") Embed a JavaScript function as a string. The bundler compiles the payload into real JS at build time. Everything after `$code:` is the raw function body. ``` { "variables": { "onlyProducts": { "condition": "$code:(entity) => entity.entity === 'product'" } } } ``` The `:` here marks the **payload**. Nothing after the colon is looked up, it is compiled as-is. ## Cascade and resolution order[​](#cascade-and-resolution-order "Direct link to Cascade and resolution order") `variables` can live at three levels. Priority (highest wins): 1. Step level (`sources.*`, `transformers.*`, `destinations.*`, `stores.*`) 2. Flow level (`flows..variables`) 3. Config root (`variables`) Resolution runs at `getFlowSettings()` in `@walkeros/core`, the runtime replaces every reference before the bundler or collector sees the config. ## What else to read[​](#what-else-to-read "Direct link to What else to read") * [`@walkeros/core` reference exports](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/references.ts): `REF_VAR_FULL`, `REF_VAR_INLINE`, `REF_ENV`, `REF_CONTRACT`, `REF_FLOW`, `REF_STORE`, `REF_SECRET`, `REF_CODE_PREFIX`. The single source of truth for the regex patterns. Also exports `scanFlowRefs(value)` which walks any value (string, array, object) and returns every `$flow.` reference found, including refs nested inside `$code:` snippets. * [Stores guide](/docs/stores/.md): wiring stores via `$store.`. * [Mapping guide](/docs/mapping/.md): `$code:` and mapping-value shortcuts. --- # Session [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/core/src/session)[ ](https://www.npmjs.com/package/@walkeros/web-core) [Package](https://www.npmjs.com/package/@walkeros/web-core) There are multiple ways to define and measure a session. Sessions represent a period when a user actively engages with a website, often used for attribution and conversion tracking. Different tools may define and detect sessions in different ways. The `sessionStart` util helps to detect a new session independently and triggers a `session start` event or [executes custom code](#callback). It works client-side and is cookieless by default. The Util returns [Session Data](#session-data) information. There are [Config Parameters](#config-parameters) to customize the session detection. In the cookieless mode (default), only the [sessionWindow](#sessionwindow) Util is used. With config parameter `storage: true`, the [sessionStorage](#sessionstorage) gets called before the `sessionWindow`. note Working with storage usually requires consent and is not activated by default. Use the [consent](#consent) option to control the storage access permissions. ## Session data[​](#session-data "Direct link to Session data") Depending on the `storage` and `consent` parameters, the `sessionStart` function returns an object with several properties. If a new session is detected, `isStart` is set to `true`, otherwise `false`. | Property | Type | Description | | --------- | ------- | --------------------------------------------------- | | isStart | boolean | If this is a new session or a known one | | storage | boolean | If the storage was used to determine the session | | id | string | Randomly generated or previously stored session id | | start | number | Timestamp of session start | | marketing | boolean | If the session was started by a marketing parameter | | referrer | string | Hostname of the referring site if available | With `storage: true` and granted `consent`, the returning object will be extended with the following: | Property | Type | Description | | -------- | ------- | ------------------------------------------------- | | updated | number | Timestamp of last update | | isNew | boolean | If this is the first visit on a device | | device | string | Randomly generated or previously stored device id | | count | number | Total number of sessions | | runs | number | Total number of runs (like page views) | ## sessionStart[​](#sessionstart "Direct link to sessionStart") Example of calling `sessionStart` on a user's first visit: ``` // On page https://www.walkeros.io/docs/session?utm_campaign=docs sessionStart({ storage: true }); // will automatically create the event { event: "session start", data: { isStart: true, storage: true, id: 'r4nd0m1d', start: 1711715862000, marketing: true, campaign: 'docs', // Additionally in storage mode updated: 1711715862000, isNew: true, device: 'd3v1c31d', count: 1, runs: 1, }, // ... } ``` info In addition, with `storage: true` and optionally granted `consent`, the `id` and `device` values are set automatically as `user.session` and `user.device` ids. ### Config parameters[​](#config-parameters "Direct link to Config parameters") The `sessionStart` function is designed to work out of the box. All parameters are optional for customization: | Parameter | Type | Description | | ------------------- | ---------------------- | ------------------------------------------------------------------------------------ | | [consent](#consent) | Array\ | The consent state to permit or deny storage access | | [storage](#storage) | boolean | If the storage should be used | | [cb](#callback) | false or
function | Callback function called after detection, or `false` to disable the default callback | info There are additional config parameters [for storage](#sessionstorage) and [for window](#sessionwindow) available. #### Consent[​](#consent "Direct link to Consent") Setting a consent state to wait for before detecting a new session is used to decide if storage access is allowed or not. If set, it registers an [on consent event](/docs/collector/commands.md#on) and won't start until a consent choice is available. If at least one permission was granted, the `sessionStorage` detects a new session; otherwise, the `sessionWindow`. ``` sessionStart({ consent: 'marketing' }); // Won't start automatically // User makes a consent choice and CMP calls: elb('walker consent', { marketing: true }); // Triggers the session detection // Returns a session with storage access ``` #### Storage[​](#storage "Direct link to Storage") Option to enable the [sessionStorage](#sessionstorage) util to detect a new session with more accuracy and enhanced data. ``` sessionStart({ storage: true }); ``` #### Callback[​](#callback "Direct link to Callback") The `cb` parameter can be used to disable the default callback or to define a custom one. The default callback triggers a `session start` event if `isStart` is `true`. And additionally, with `storage: true`, the user's `session` and `device` ids are also set via `elb('walker user', user);`. The default callback function is passed as the third parameter. ``` const session = sessionStart({ cb: false }); // Disables the default callback sessionStart({ cb: (session, instance, defaultCb) => { console.log(session); defaultCb(session, instance); // Call the default callback }, }); ``` ### Detection process[​](#detection-process "Direct link to Detection process") Based on the [storage](#storage) option either the [sessionStorage](#sessionstorage) or the [sessionWindow](#sessionwindow) is used to detect a new session. If a [consent](#consent) state is set, the session detection gets scheduled via an [on-consent](/docs/collector/commands.md#on) command. It will only run once per `run`. ## sessionStorage[​](#sessionstorage "Direct link to sessionStorage") ### Config parameters[​](#config-parameters-1 "Direct link to Config parameters") Additional config parameters for storage-based session detection: | Property | Type | Description | Default | | -------------- | ------- | ------------------------------------------------------------ | ------------ | | deviceKey | string | The key to store the device ID in the storage | elbDeviceId | | deviceStorage | string | The storage type to use for the device id | local | | deviceAge | number | The age in days to consider the device ID as expired | 30 | | sessionKey | string | The key to store the session ID in the storage | elbSessionId | | sessionStorage | string | The storage type to use for the session id | local | | length | number | Minutes after the last update to consider session as expired | 30 | | pulse | boolean | Update the current session to stay active | false | info There are additional config parameters for [sessionWindow](#sessionwindow) available. ### Detection process[​](#detection-process-1 "Direct link to Detection process") Basic rules to detect a new session: ## sessionWindow[​](#sessionwindow "Direct link to sessionWindow") ### Config parameters[​](#config-parameters-2 "Direct link to Config parameters") | Parameter | Type | Description | | -------------------------------------- | ------- | ------------------------------------------------------------------------ | | [data](#custom-data) | object | Custom data to enhance the default `data` properties | | [domains](#internal-domains) | array | Internal domains to prevent new sessions from triggering when navigating | | [isStart](#manual-new-session-control) | boolean | Manual new session control | | [parameters](#marketing-parameters) | object | Marketing parameters to enhance the default and support custom ones | | [referrer](#referrer-customization) | string | Referrer customization | | [url](#url-customization) | string | URL customization | #### Custom data[​](#custom-data "Direct link to Custom data") Enhance the default `data` properties with custom information, like a session count: ``` const count = 2; const session = sessionStart({ data: { count } }); ``` This will return a `data` object like `{ id: "r4nd0m", count: 2 }`. #### Internal domains[​](#internal-domains "Direct link to Internal domains") Define internal domains to prevent new sessions from triggering when navigating between them: ``` const session = sessionStart({ domains: ['subdomain.walkeros.io', 'example.com'], }); ``` A user coming from `subdomain.walkeros.io` or `example.com` to e.g. `www.walkeros.io`, will no longer trigger a new session. #### Manual new session control[​](#manual-new-session-control "Direct link to Manual new session control") Determine if it's a new session using the `isStart` parameter. This might require consent for storage access, which isn't implemented by default. Usually, the `sessionId` written to the storage is set up to expire and should be updated with each page view. If the `sessionId` is missing, it may be an expired but at least a new session. ``` if (!storageRead('sessionId')) { const session = sessionStart({ isStart: true }); storageWrite('sessionId', session.id); } ``` > For more information on storage and expiration, check the storage configuration options in this guide. #### Marketing parameters[​](#marketing-parameters "Direct link to Marketing parameters") The helper util `getMarketingParameters` is used to extract common parameters like all `utm variants`, typical clickIds like `fbclid`, `gclid`, and others. To enhance the default and support custom ones add `parameters`, like `{ elb_campaign: 'campaign' }` to add `campaign: "docs"` to `data` for a url with `?elb_campaign=docs`. A session with marketing parameters will be flagged with `data.marketing = true` automatically. ``` interface MarketingParameters { [key: string]: string; } sessionStart({ parameters: { elb_campaign: 'docs' }, }); ``` ### Click IDs and platform resolution[​](#click-ids-and-platform-resolution "Direct link to Click IDs and platform resolution") When the URL contains a known ad-platform click ID, `getMarketingParameters` adds two fields to its result: * `clickId`: the URL parameter name that was found (e.g. `gclid`). * `platform`: the canonical platform identifier (e.g. `google`). The raw value is also stored under the parameter name itself, so a URL like `https://example.com/?gclid=abc` produces: ``` { clickId: 'gclid', gclid: 'abc', platform: 'google', } ``` If a URL contains multiple click IDs, every raw value is preserved, but `clickId` and `platform` reference the **highest-priority** match. See [`clickIds.ts`](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/clickIds.ts) for the full list and priority order. #### Extending the registry from flow\.json[​](#extending-the-registry-from-flowjson "Direct link to Extending the registry from flow.json") Pass a `clickIds` array in the session source settings. Entries with a `param` matching a default override the platform name in place; new params append to the end of the priority list. ``` { "version": 4, "flows": { "default": { "config": { "platform": "web" }, "sources": { "session": { "package": "@walkeros/web-source-session", "config": { "settings": { "clickIds": [ { "param": "xyzclid", "platform": "xyz" }, { "param": "fbclid", "platform": "facebook" } ] } } } } } } } ``` #### Referrer customization[​](#referrer-customization "Direct link to Referrer customization") By default the `document.referrer` is used, but it can be overwritten with the `referrer` parameter. ``` sessionStart({ referrer: 'https://example.com' }); ``` Domains can be extended, e.g., internal sub-domains. Data can be pre-defined, e.g., to use your ID. #### URL customization[​](#url-customization "Direct link to URL customization") By default, the `window.location.href` is used, but it can be overwritten with the `url` parameter. ``` sessionStart({ url: 'https://example.com' }); ``` ### Detection process[​](#detection-process-2 "Direct link to Detection process") Basic rules to detect a new session: 1. **Storage Check** (*Optional*): First, check for an existing sessionId in storage. If none is found, consider it a new session. This usually requires consent. 2. **Page reload**: If the entry type is a page reload, it's not a new session. 3. **Marketing Parameters**: The presence of marketing parameters in the URL indicates a new session. 4. **Referrer Check**: A different referrer from the current domain signals a new session. note Be aware of potential multiple unintended events for the same user due to referrer hiding. For more details, learn about [Referrer Hiding](https://en.wikipedia.org/wiki/HTTP_referer#Referrer_hiding). --- # User stitching To combine website and mobile app data, to unify customer interaction data across different platforms, or to generally track a user's journey through various touchpoints, **user stitching** is required. It is a technique to combine data from multiple sources or sessions to create a unified view of a user. This is especially useful in scenarios where a user interacts with a system across multiple devices or platforms, and the goal is to consolidate these interactions into a single, coherent user profile. In general, when working with user behavior data, there's always a high chance of incomplete data. Especially in such a volatile environment like the web, where users can easily **delete cookies**, **change devices**, or simply use a **different browser**. Understanding user identification to stitch correctly is key. Working with persistent identifiers requires a **user's consent**. Consent is always bound to specific purposes, which have to be documented and communicated properly. tip Learn more about [consent management](/docs/guides/consent/.md) with walkerOS. Relying on third-party tools and cookies might have been a common practice with benefits in the past. Support for third-party cookies is phasing out, and there always has been and will be a dependence on others if data isn't owned and collected first-party. First-party data collection is fundamental for **attribution modeling**, creating a **holistic view** of your users and gaining **data ownership**. ## User identification levels[​](#user-identification-levels "Direct link to User identification levels") **User identification** is the process of recognizing the identity of a user. This typically involves associating a user with a unique identifier. There are three levels of user identification in walkerOS: * **id**: Represents a unique identifier for a user, typically drawn from internal systems like a CRM. * **device**: Used as a device-specific identifier, saved in the storage. * **session**: Refers to a session-specific identifier to track user activities within a single session. A single user can have different devices with multiple sessions. The `id` is used to distinctly identify an individual user across various sessions and devices. The `device` helps to recognize a user across multiple sessions on the same device, providing continuity in tracking user interactions over time, while `session `is for understanding user behavior and interactions in a confined timeframe. The diagram represents one user with two devices across three sessions. warning These techniques may require a user's consent and documentation to respect privacy and stay compliant. Please request your data protection officer for legal advice. ## Example[​](#example "Direct link to Example") Events are individually measured user interactions that need to be stitched together. All event-related data is additional information to help identifying a user. This is necessary to create user journeys. The following table represents a minified event log over several days. It shows how user activities can be tracked over time, across different sessions, with varying levels of user identification and consent categories. A `group` id gets generated with each run, representing a page load. A `hash` is a temporary fingerprint. `session`, `device`, and `user` are all simplified unique identifiers. | Step | Day | Event | Consent | Group | Hash | Session | Device | User | | ---- | ------------------- | -------------- | --------- | ----- | ---- | ------- | ------ | ---- | | 1 | Monday,
1st | session start | analytics | a | foo | | | | | 2 | Monday,
1st | page view | analytics | a | foo | | | | | 3 | Monday,
1st | consent deny | analytics | a | foo | | | | | 4 | Monday,
1st | page view | analytics | b | foo | | | | | 5 | Tuesday,
2nd | session start | marketing | c | bar | 1 | phone | | | 6 | Tuesday,
2nd | page view | marketing | c | bar | 1 | phone | | | 7 | Tuesday,
2nd | consent accept | marketing | c | bar | 1 | phone | | | 8 | Tuesday,
2nd | session start | marketing | d | bar | 2 | phone | | | 9 | Tuesday,
2nd | page view | marketing | d | bar | 2 | phone | | | 11 | Wednesday,
3rd | session start | marketing | e | baz | 3 | phone | olli | | 12 | Wednesday,
3rd | page view | marketing | e | baz | 3 | phone | olli | | 13 | Wednesday,
3rd | user login | marketing | e | baz | 3 | phone | olli | | 14 | Friday,
5th | session start | analytics | f | qux | | | | | 15 | Friday,
5th | page view | analytics | f | qux | | | | | 16 | Friday,
5th | page view | analytics | g | qux | | | olli | | 17 | Friday,
5th | consent deny | analytics | g | qux | | | olli | | 18 | Friday,
5th | user login | analytics | g | qux | | | olli | | 19 | Thursday,
11th | session start | marketing | h | lol | 4 | ph0n3 | | Based on the data, we can at least summarize the following: * 19 tracked `events` in total * 8 different `group` ids ('a', 'b', 'c', 'd', 'e', 'f', 'g', 'h') * 6 individual `session start` events * 5 temporary `hashes` ('foo', 'bar', 'baz', 'qux', 'lol') * 4 reliable `sessions` (1, 2, 3, 4) * 2 known `devices` ('phone' and 'ph0n3') * 1 unique `user` ('olli') The example illustrates the complexity and richness of user data captured through user stitching. --- # Mapping Mapping is walkerOS's code-free way to reshape events as they move through the flow. You describe the transformation as data (JSON or object literal), and the collector applies it at two stations: **source push** (clean and normalize incoming data) and **destination transform** (adapt to each tool's expected format). ## Two concepts[​](#two-concepts "Direct link to Two concepts") Everything in walkerOS mapping is built from two building blocks: * **[Mapping.Value](/docs/mapping/value.md)**: the polymorphic primitive. Whenever a config field accepts a value (`data`, `map.*`, `policy.*`, `set[]`, …), you write a `Mapping.Value`. There are eight forms (path, constant, map, loop, set, fn, condition, consent). * **[Mapping.Rule](/docs/mapping/rule.md)**: the object on a destination's `mapping[entity][action]`. It composes values into a complete rule (`name`, `data`, `condition`, `consent`, `policy`, `batch`, `ignore`, `skip`, `settings`). When a destination doc shows `data: any`, the value you put there is a `Mapping.Value`. The whole rule around it is a `Mapping.Rule`. ## Minimal example[​](#minimal-example "Direct link to Minimal example") ``` // Destination rule: transform "product view" for GA4 { product: { view: { name: 'view_item', data: { map: { item_id: 'data.id', value: 'data.price', currency: { value: 'USD' }, }, }, }, }, } ``` `product.view` is the entity-action key. `name` renames the event. `data` is a `Mapping.Value` (specifically the `map` form) that builds the destination payload. ## Try it[​](#try-it "Direct link to Try it") Map events by their entity-action structure, live: Configuration Loading... Result Loading... ## Where mapping runs[​](#where-mapping-runs "Direct link to Where mapping runs") **Source mapping** cleans, filters, or renames incoming events before they reach the collector: ``` await startFlow({ sources: { browser: { code: sourceBrowser, config: { mapping: { product: { click: { name: 'product view' } }, test: { '*': { ignore: true } }, }, }, }, }, }); ``` **Destination mapping** adapts events to each tool's API shape: ``` await startFlow({ destinations: { gtag: { code: destinationGtag, config: { mapping: { product: { view: { name: 'view_item', data: { map: { item_id: 'data.id' } }, }, }, }, }, }, }, }); ``` Both mappings are independent. The same event can be transformed differently at each stage. ## See also[​](#see-also "Direct link to See also") * [Mapping.Value](/docs/mapping/value.md): the eight value forms * [Mapping.Rule](/docs/mapping/rule.md): rule-level fields and API reference * [Transformers](/docs/transformers/.md): middleware alternative for validation, enrichment, redaction * [Consent](/docs/guides/consent/.md): consent-gated mapping --- # Mapping.Rule A `Mapping.Rule` is the object that sits at `config.mapping[entity][action]` on a source or destination. It decides whether an event applies, renames it, reshapes its data, gates it by consent, and can emit side effects (batching, custom settings). The shape is defined in [`packages/core/src/types/mapping.ts`](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/mapping.ts). ## Configuration[​](#configuration "Direct link to Configuration") | Property | Type | Description | More | | ----------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `name` | `string` | Custom event name override (e.g., "view\_item" for "product view") | | | `data` | `any \| any` | Data transformation rules for event | | | `settings` | `any` | Destination-specific settings for this event mapping | | | `condition` | `string` | Condition function as string: return true to process event | | | `consent` | `WalkerOS.Consent` | Required consent states to process this event | | | `policy` | `Mapping.Policy` | Event-level policy overrides (applied after config-level policy) | | | `batch` | `number \| object` | Batch scheduling: bare number is the debounce wait window (legacy); object form supports wait/size/age. | | | `include` | `Array` | Event sections (e.g. \["context", "globals"]) flattened into context.data | | | `ignore` | `boolean` | Skip the event entirely. No push, no side effects. Use for suppression. | | | `silent` | `boolean` | Run side effects (settings.identify, ...) but suppress the destination default push call. | | | `extend` | `Mapping.RulePatch` | Merge mode: a partial rule deep-merged onto the package-shipped default at this key (instead of replacing it). A null value clears an inherited field. | | | `remove` | `Array` | Dotted paths stripped from the produced data payload after evaluation (applied last). | | Every value-shaped field (`data`, entries in `policy`, entries in `settings` that take values) accepts any form documented in [Mapping.Value](/docs/mapping/value.md). ## How destinations consume a rule[​](#how-destinations-consume-a-rule "Direct link to How destinations consume a rule") When an event reaches a destination, the collector looks up `mapping[event.entity][event.action]`. An array means "try each in order, first matching rule wins". A wildcard `'*'` on either entity or action acts as a fallback. Mapping wildcards differ from contract wildcards Mapping wildcards are **first-match**: once `entity.action` (or its `entity.*` / `*.action` / `*.*` fallback) hits, no other rule is consulted. [Contract wildcards](/docs/getting-started/flow/contract.md#wildcard-inheritance) are **additive**: every matching level merges in. The two systems intentionally pick different defaults — mapping selects a single transformation target, contract accumulates requirements. Configuration Loading... Result Loading... Once a rule matches, the collector runs `policy` (event-level), resolves `data`, checks `consent` and `condition`, then hands the result to the destination's `push` (or buffers it when `batch` is set). ### Batch scheduling[​](#batch-scheduling "Direct link to Batch scheduling") `batch` accepts a number (debounce `wait` window in ms, legacy form) or an object `{ wait?, size?, age? }` for finer control: * `wait` (ms): debounce window. The timer resets on every push. * `size`: hard count cap. Flushes immediately at this many events. * `age` (ms): hard age cap since the first entry of the current window. A rule-level `batch` splits this entity-action into its own buffer. When the destination also sets `config.batch`, the rule's values override destination-level per field (`rule ?? config ?? default`); fields the rule omits fall back to `config.batch`. Setting `config.batch` alone batches **all** of the destination's events into one shared buffer with no rule needed. Defaults when batching is enabled: `size: 1000`, `age: 30000`. `wait` is a hint, not a hard delay: when events keep arriving faster than `wait`, the `size` and `age` caps are what force the flush. ## Conditional rules[​](#conditional-rules "Direct link to Conditional rules") Use an array with conditions to branch by event contents; the first matching rule wins. Configuration Loading... Result Loading... ## Ignore vs skip[​](#ignore-vs-skip "Direct link to Ignore vs skip") Two flags control destination behaviour and they are not interchangeable: * `ignore: true`: the rule matched but **nothing happens**. No data transform, no destination call, no side effects. Use for suppression. * `skip: true`: the rule matched and the destination's `push()` **is** called. `settings.identify`, `settings.revenue`, `settings.group`, and other side effects still run. Only the destination's default forwarding call (e.g. `track()`, `capture()`, `event()`) is suppressed. Use for "identify without an event" style flows. If both are set on the same rule, `ignore` wins. Configuration Loading... Result Loading... ``` { user: { login: { skip: true, settings: { identify: { map: { user: 'data.id' } } }, }, }, } ``` The example runs the destination's `identify()` side effect on `user login` events but skips the default forwarding call. ## API reference[​](#api-reference "Direct link to API reference") The two functions below power mapping resolution. Destinations and sources call them through the collector, but they are exported for direct use in custom transformers or tests. ### getMappingEvent[​](#getmappingevent "Direct link to getMappingEvent") ``` getMappingEvent( event: WalkerOS.PartialEvent, mapping?: Mapping.Rules, ): Promise ``` Resolves the matching rule for an event based on `event.name` (the `"entity action"` string). Returns `{ eventMapping, mappingKey }` where `eventMapping` is the matched `Mapping.Rule` and `mappingKey` is the lookup key that matched (including wildcards). ### getMappingValue[​](#getmappingvalue "Direct link to getMappingValue") ``` getMappingValue( value: unknown, mapping: Mapping.Data, context?: Partial, ): Promise ``` Resolves a `Mapping.Value` against an input. Used internally for every `data`, `policy`, and `set` resolution. The companion reference for each value form lives in [Mapping.Value](/docs/mapping/value.md). ## Rule-level examples[​](#rule-level-examples "Direct link to Rule-level examples") ### Source mapping[​](#source-mapping "Direct link to Source mapping") Normalize events before they reach the collector: ``` await startFlow({ sources: { browser: { code: sourceBrowser, config: { mapping: { product: { click: { name: 'product view', policy: { 'data.id': { fn: (e) => e.data?.id?.toUpperCase() }, }, }, }, test: { '*': { ignore: true } }, }, policy: { 'user.email': { fn: (e) => e.user?.email?.toLowerCase() }, }, }, }, }, }); ``` ### Destination mapping[​](#destination-mapping "Direct link to Destination mapping") Transform for a specific destination API: ``` await startFlow({ destinations: { gtag: { code: destinationGtag, config: { mapping: { product: { view: { name: 'view_item', data: { map: { item_id: 'data.id', value: 'data.price', currency: { value: 'USD' }, }, }, }, }, }, }, }, }, }); ``` ### Combined flow[​](#combined-flow "Direct link to Combined flow") The same event can be transformed at both stations: ``` 1. Browser sends: "product click" 2. Source mapping: "product click" → "product view" 3. Destination mapping: "product view" → "view_item" 4. GA4 receives: "view_item" ``` ## Patching package-shipped rules[​](#patching-package-shipped-rules "Direct link to Patching package-shipped rules") Some packages (such as `@walkeros/transformer-ga4`) ship their own default mapping rules. Normally, a user rule at the same key replaces the default in full. Two keywords change that: ### `extend` (config layer)[​](#extend-config-layer "Direct link to extend-config-layer") `extend` holds a **partial rule** that is deep-merged onto the package-shipped default at the same key instead of replacing it. The merge happens once at init, before any event is evaluated. A `null` value clears an inherited field. Use `extend` when you want to add or override one field of a default rule while inheriting the rest unchanged. ### `remove` (output layer)[​](#remove-output-layer "Direct link to remove-output-layer") `remove` is a list of **dotted paths** stripped from the produced data payload after evaluation, regardless of how each field was produced (`map`, `fn`, `include`, etc.). Applied last, so it always wins. Use `remove` when you must not emit a field (PII, a vendor-reserved parameter) but do not want to rewrite the full rule. ### Mode switching[​](#mode-switching "Direct link to Mode switching") A rule that has **neither** `extend` nor `remove` keeps today's replace behavior. Presence of either keyword activates merge mode for that rule. ### Example: patch a GA4 decoder rule[​](#example-patch-a-ga4-decoder-rule "Direct link to Example: patch a GA4 decoder rule") The transformer-ga4 `purchase` default ships `id`, `currency`, `total`, `tax`, `shipping`, and `coupon`. The config below adds `affiliation` from a GA4 event parameter and drops `currency` from the output, keeping all other default fields: ``` { "settings": { "mapping": { "purchase": { "extend": { "data": { "map": { "affiliation": "params.ep.affiliation" } } }, "remove": ["currency"] } } } } ``` `extend.data.map` is deep-merged onto the default `data.map`, so `affiliation` is added alongside the existing keys. `remove: ["currency"]` then strips that field from the final payload. ## See also[​](#see-also "Direct link to See also") * [Mapping.Value](/docs/mapping/value.md): the eight value forms used in every rule field * [Consent guide](/docs/guides/consent/.md): consent model and gating * [Destinations reference](/docs/destinations/.md): destination-specific `settings` * [`mapping.ts` source](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/mapping.ts) --- # Mapping.Value `Mapping.Value` is what you put into any mapping field that expects a value: `data`, every entry under `map`, entries in `set[]`, keys under `policy`, and anywhere a destination rule field accepts `any`. It is polymorphic: a value can be a simple string path, an object with one of several transforming keys, or an array of those. The shape is defined in [`packages/core/src/types/mapping.ts`](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/mapping.ts) as `Value = ValueType | Array`, where `ValueType` is either a string (path) or a `ValueConfig` object. ## Options[​](#options "Direct link to Options") | Form | Shape | What / When | | ----------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------- | | [Path](#path) | `string` | Reads a dot-path from the event. Use for copying raw fields straight into the output. | | [Constant](#constant) | `{ value }` | Emits a fixed value. Use when the destination needs a literal like a currency, ID, or label. | | [Map](#map) | `{ map: { key: Value, … } }` | Builds an object from multiple paths. Use when the destination expects a structured payload. | | [Loop](#loop) | `{ loop: [source, Value] }` | Iterates an array source and transforms each item. Use for nested product, cart, or item lists. | | [Set](#set) | `{ set: [Value, Value, …] }` | Picks the first defined value from a list. Use as a fallback chain like email then user\_id. | | [Fn](#fn) | `{ fn: (value, context) => … }` | Runs a custom function on the event. Use only when no declarative form fits. | | [Condition](#condition) | `{ condition, value }` | Gates the value on a predicate. Use for branching logic inside a single rule. | | [Consent](#consent) | `{ consent: {…}, value }` | Gates the value on consent states. Use to make individual fields privacy-aware. | All `ValueConfig` keys can be combined on the same object: `key`, `condition`, `consent`, `validate` act as filters; `value`, `fn`, `map`, `loop`, `set` produce output. ## Path[​](#path "Direct link to Path") Extract a value by dot-path from the event. Configuration Loading... Result Loading... Paths also index into arrays: Configuration Loading... Result Loading... ## Constant[​](#constant "Direct link to Constant") Return a fixed value, independent of the event. Configuration Loading... Result Loading... ## Map[​](#map "Direct link to Map") Build an object by mapping keys to their own `Mapping.Value`. Configuration Loading... Result Loading... ## Loop[​](#loop "Direct link to Loop") Iterate an array source and transform each element. `loop: [source, value]`: `source` selects the array (a path, or `'this'` for the event itself), `value` transforms each item. Configuration Loading... Result Loading... ## Set[​](#set "Direct link to Set") Try each value in order and return the first defined result. Useful as a fallback chain. ``` { set: ['data.email', 'user.email', { value: 'anonymous' }] } ``` ## Fn[​](#fn "Direct link to Fn") Run a custom function. The handler receives `(value, context)` and may return a promise. All three callbacks (`fn`, `condition`, `validate`) share the same shape. ### Context object[​](#context-object "Direct link to Context object") | Field | Type | Required | Description | | ----------- | ------------------------------- | -------- | ------------------------------------------ | | `event` | `WalkerOS.DeepPartialEvent` | yes | The root event being mapped | | `mapping` | `Mapping.Value \| Mapping.Rule` | yes | The surrounding mapping config (or rule) | | `collector` | `Collector.Instance` | yes | Active collector, use for `push` and queue | | `logger` | `Logger.Instance` | yes | Use for `info`/`warn`/`error`/`debug` | | `consent` | `WalkerOS.Consent` (optional) | no | Resolved consent at this evaluation point | One-arg callbacks like `(value) => value.toUpperCase()` continue to work, TypeScript ignores the unused `context` argument. Configuration Loading... Result Loading... ## Condition[​](#condition "Direct link to Condition") Only produce output when `condition(event)` returns truthy. ``` { condition: (event) => event.data?.value > 50, value: 'high_value', } ``` ## Consent[​](#consent "Direct link to Consent") Only produce output when the required consent states are granted on the event. Configuration Loading... Result Loading... ## Validate[​](#validate "Direct link to Validate") `validate` is a filter, not a form. Combine it with any value-producing key. The function runs on the produced value; if it returns falsy the result is dropped (`undefined`). Configuration Loading... Result Loading... ## Policy[​](#policy "Direct link to Policy") A policy is a record of `Mapping.Value`s applied to the event before the rule runs. The key is the path the result writes into; the value is any form above. **Config-level policy** applies to every event handled by the source or destination: ``` { config: { policy: { 'user.email': { fn: (e) => e.user?.email?.toLowerCase() }, 'data.timestamp': { fn: () => Date.now() }, }, }, } ``` **Event-level policy** applies only inside the matched rule: ``` { config: { mapping: { product: { view: { name: 'view_item', policy: { 'data.id': { fn: (e) => `PRODUCT_${e.data?.id}` }, }, }, }, }, }, } ``` Processing order: config policy → event matching → event policy → data transformation. ## See also[​](#see-also "Direct link to See also") * [Mapping.Rule](/docs/mapping/rule.md): the rule object that composes values * [`getMappingValue` API](/docs/mapping/rule.md#getmappingvalue): programmatic value resolution * [Consent guide](/docs/guides/consent/.md): consent model in depth --- # Migrating to @walkeros/cli 4.x The bundler now uses [`@vercel/nft`](https://github.com/vercel/nft) to trace server flow dependencies and ship only the files actually used. flow\.json field locations are unchanged. The migration is a small, mechanical edit to your Dockerfile and (if you used it) the `walkerOS.bundle.external` annotation. ## flow\.json schema is unchanged[​](#flowjson-schema-is-unchanged "Direct link to flow.json schema is unchanged") `version: 4` stays. `flow..config.bundle.packages`, `flow..config.bundle.overrides`, and `flow..config.bundle.traceInclude` remain the canonical locations for build-time fields. ## Remove `flow..config.bundle.external` entirely[​](#remove-flownameconfigbundleexternal-entirely "Direct link to remove-flownameconfigbundleexternal-entirely") nft handles externalization automatically. Remove the field; it is no longer recognized. ``` "config": { "platform": "server", "bundle": { "packages": {...}, - "external": ["@google-cloud/bigquery-storage", "@grpc/grpc-js"] } } ``` The CLI prints a deprecation warning if it sees a v3-era `config.bundle.external` block. The same applies to step-package manifests carrying `walkerOS.bundle.external` (also no longer recognized). ## Update the Dockerfile[​](#update-the-dockerfile "Direct link to Update the Dockerfile") Server bundles are always a directory now: `dist/{flow.mjs, package.json, node_modules/}`. If your Dockerfile copied just `bundle.mjs`, change it to copy the whole directory and bump the runtime image: ``` FROM node:22.23.0-alpine AS builder WORKDIR /build RUN npm init -y && npm install --save-dev @walkeros/cli COPY flow.json ./ -RUN npx walkeros bundle ./flow.json --output dist/bundle.mjs +RUN npx walkeros bundle flow.json -o dist/ -FROM walkeros/flow:3.3.0 -COPY --from=builder /build/dist/bundle.mjs /app/flow/bundle.mjs -ENV BUNDLE=/app/flow/bundle.mjs -ENV FILES_PATH=/app/flow +FROM walkeros/flow:4 +WORKDIR /app/flow +COPY --from=builder /build/dist/ ./ ENV PORT=8080 EXPOSE 8080 ``` The `BUNDLE` and `FILES_PATH` env vars are no longer needed: the runtime image's defaults match `/app/flow/flow.mjs`. ## You no longer need `npm install` for step packages[​](#you-no-longer-need-npm-install-for-step-packages "Direct link to you-no-longer-need-npm-install-for-step-packages") Pacote drives the install layer based on `flow..config.bundle.packages`. Your project's `package.json` only needs `@walkeros/cli` as a devDependency (plus any local code your project ships, of course). You can remove pinned `@walkeros/...` deps from `package.json` if you previously listed them just to satisfy the bundler. ## Escape hatch: `traceInclude`[​](#escape-hatch-traceinclude "Direct link to escape-hatch-traceinclude") If nft cannot statically reach a runtime asset (rare: `require(somePathFromEnv)`), declare it explicitly under the per-flow bundle block: ``` "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-destination-gcp": {} }, "traceInclude": [ "node_modules/some-pkg/data/*.json" ] } } } } ``` Paths and globs both work. They resolve against the install root (where pacote put files), not your project directory. ## Output filename change[​](#output-filename-change "Direct link to Output filename change") Default output filename is now `flow.mjs` (was `bundle.mjs`). The runtime image expects `/app/flow/flow.mjs`. If you have scripts that read `dist/bundle.mjs`, update them. --- # Migrating flow\.json from v3 to v4 walkerOS Flow v4 reshapes `flow.json` so each flow has a dedicated `config` block (platform, url, settings, bundle), and adds the `$flow.X.Y` reference syntax for cross-flow lookups inside the same file. The v4 CLI rejects v3 input outright, so every existing `flow.json` needs a one-time edit. This guide walks through the four mechanical JSON transforms, then covers the matching TypeScript type renames for code that imports `Flow.*` from `@walkeros/core`. ## What changed at a glance[​](#what-changed-at-a-glance "Direct link to What changed at a glance") * **Type renames in `@walkeros/core`:** `Flow.Settings` (whole flow) becomes `Flow`, `Flow.Config` (root file) becomes `Flow.Json`, and the new `Flow.Settings` is now the small key-value bag inside `Flow.Config.settings`. * **New per-flow `config` block** with `platform`, `url`, `settings`, and `bundle`. * **`web` and `server` keys are gone.** Their fields move into `config.platform` and `config.settings`. * **`bundle` is no longer a top-level flow key.** It moves into `config.bundle`. * **New `$flow.X.Y` reference syntax** lets one flow read another flow's resolved `config` (e.g., `$flow.server.url`). * **Hard cut:** the v4 CLI refuses v3 input. There is no compat shim. * **No automated migration command.** Edits are mechanical, see below. ## Step-by-step JSON migration[​](#step-by-step-json-migration "Direct link to Step-by-step JSON migration") The transforms are independent; apply them in any order. ### Transform 1: bump `version`[​](#transform-1-bump-version "Direct link to transform-1-bump-version") ``` { "version": 4 } ``` Was `"version": 3`. The v4 CLI fails fast on any other value with a clear `unsupported flow.json version` error. ### Transform 2: update `$schema`[​](#transform-2-update-schema "Direct link to transform-2-update-schema") ``` { "$schema": "https://walkeros.io/schema/flow/v4.json" } ``` Was `https://walkeros.io/schema/flow/v3.json`. The v3 schema file has been removed from the docs site. ### Transform 3: lift `web` / `server` into `config`[​](#transform-3-lift-web--server-into-config "Direct link to transform-3-lift-web--server-into-config") Every flow had either a `"web": { ... }` or a `"server": { ... }` block. Both are gone in v4. Replace them with a `config` block that carries `platform` plus any web settings: v3: ``` { "flows": { "default": { "web": { "windowCollector": "collector", "windowElb": "elb" } } } } ``` v4: ``` { "flows": { "default": { "config": { "platform": "web", "settings": { "windowCollector": "collector" } } } } } ``` The v3 `windowElb` name moves to the browser source: set `elb` in the browser source's `config.settings` (default `"elb"`, so it can usually be dropped). The `windowElb` key is deprecated in v4; when set, its value is forwarded to the browser source with a warning. Keeping `windowCollector` preserves your existing global; dropping it uses the v4 default, `window.walkerOS`. For server flows, the transform is the same: drop `"server": {}`, add `"config": { "platform": "server" }`. Server flows can also set `config.url` (used by `$flow.X.url` lookups, see below). ### Transform 4: lift `bundle` into `config`[​](#transform-4-lift-bundle-into-config "Direct link to transform-4-lift-bundle-into-config") In v3, `bundle` was a top-level sibling of `web` / `server`. In v4, it moves under `config`: v3: ``` { "flows": { "default": { "web": {}, "bundle": { "packages": { "@walkeros/collector": { "version": "latest" } } } } } } ``` v4: ``` { "flows": { "default": { "config": { "platform": "web", "bundle": { "packages": { "@walkeros/collector": { "version": "latest" } } } } } } } ``` `bundle.overrides` (transitive dependency pins) lives in the same place, under `config.bundle.overrides`. ## Worked example[​](#worked-example "Direct link to Worked example") A small v3 web flow: ``` { "version": 3, "$schema": "https://walkeros.io/schema/flow/v3.json", "flows": { "default": { "web": { "windowCollector": "collector", "windowElb": "elb" }, "bundle": { "packages": { "@walkeros/collector": { "version": "latest", "imports": ["startFlow"] }, "@walkeros/web-destination-gtag": { "version": "latest", "imports": ["destinationGtag"] } } }, "destinations": { "ga4": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "ga4": { "measurementId": "G-XXXXXXXXXX" } } } } } } } } ``` The same flow in v4: ``` { "version": 4, "$schema": "https://walkeros.io/schema/flow/v4.json", "flows": { "default": { "config": { "platform": "web", "settings": { "windowCollector": "collector" }, "bundle": { "packages": { "@walkeros/collector": { "version": "latest", "imports": ["startFlow"] }, "@walkeros/web-destination-gtag": { "version": "latest", "imports": ["destinationGtag"] } } } }, "destinations": { "ga4": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "ga4": { "measurementId": "G-XXXXXXXXXX" } } } } } } } } ``` Notice that `sources`, `destinations`, `transformers`, `stores`, and `collector` keep their old shapes. Only the platform/bundle plumbing moves. ## The new `$flow.X.Y` reference[​](#the-new-flowxy-reference "Direct link to the-new-flowxy-reference") `$flow.(.)?` resolves to the value at `flows..config.` in the same `flow.json` file. The `config` segment is implicit, so `$flow.server.url` walks to `flows.server.config.url`. This is useful when a web flow needs to point its API destination at the URL of a sibling server flow: ``` { "version": 4, "flows": { "server": { "config": { "platform": "server", "url": "https://api.example.com/collect" } }, "web": { "config": { "platform": "web" }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "$flow.server.url" } } } } } } } ``` Strictness rules: * `walkeros bundle` and `walkeros deploy` error if `$flow.X.Y` resolves to an empty value. * `walkeros validate` warns by default, escalates to error with `--strict`. * The error message points you to the source: set `flows.server.config.url`, or run `walkeros deploy server` first. ## TypeScript type renames[​](#typescript-type-renames "Direct link to TypeScript type renames") If your code imports `Flow.*` types from `@walkeros/core`, apply this rename catalog. Sub-namespaces are gone; everything is flat under `Flow`. | v3 | v4 | | ----------------------------- | -------------------------------------------------- | | `Flow.Config` (root file) | `Flow.Json` | | `Flow.Settings` (single flow) | `Flow` (interface) | | *(none)* | `Flow.Settings` (NEW: kv-bag inside `Flow.Config`) | | `Flow.Web`, `Flow.Server` | *(removed)* `config.platform` is a string | | `Flow.InlineCode` | `Flow.Code` | | `Flow.Packages` | `Flow.Bundle.packages` | | `Flow.Overrides` | `Flow.Bundle.overrides` | | `Flow.SourceReference` | `Flow.Source` | | `Flow.DestinationReference` | `Flow.Destination` | | `Flow.TransformerReference` | `Flow.Transformer` | | `Flow.StoreReference` | `Flow.Store` | | `Flow.ContractEntry` | `Flow.ContractRule` | Step-related types (`Flow.StepExample`, `Flow.StepExamples`, `Flow.StepCommand`, `Flow.StepEffect`, `Flow.StepOut`) are unchanged. `Flow.ContractSchema`, `Flow.ContractActions`, and `Flow.ContractEvents` are also unchanged. A typical TypeScript change looks like this: ``` // v3 import type { Flow } from '@walkeros/core'; function buildFlow(): Flow.Settings { return { web: {}, destinations: {} }; } ``` ``` // v4 import type { Flow } from '@walkeros/core'; function buildFlow(): Flow { return { config: { platform: 'web' }, destinations: {} }; } ``` ## No automated codemod[​](#no-automated-codemod "Direct link to No automated codemod") walkerOS v4 does not ship a `walkeros migrate` command. The transforms are mechanical (four edits per file, all string-level), and even complex configs produce a small, readable diff. Apply the four transforms manually, run `walkeros validate` to confirm the result parses, and you're done. If validate reports an error, the message will name the exact path that still looks like v3, e.g., a stray top-level `bundle` or a `web` block left behind. ## Walker commands take an Init object[​](#walker-commands-take-an-init-object "Direct link to Walker commands take an Init object") Three runtime `elb('walker ...')` commands changed shape in v4. The positional and shorthand forms are removed, every one of them now takes a single Init object. The `walker hook` command also has a real runtime handler now (in v3 it accepted arguments but never wired anything up). ### `walker destination`[​](#walker-destination "Direct link to walker-destination") Was: `elb('walker destination', destination, config?)`, or the `{ push }` shorthand, or `{ code }` plus a separate config argument. Now: a single Init object with `code` (the destination) and an optional `config`. ``` // v3 elb('walker destination', destinationGtag, { settings: { ga4: { measurementId: 'G-XXX' } }, }); // v4 elb('walker destination', { code: destinationGtag, config: { settings: { ga4: { measurementId: 'G-XXX' }, }, }, }); ``` ### `walker hook`[​](#walker-hook "Direct link to walker-hook") Was: `elb('walker hook', name, fn)` (no-op at runtime, the dispatch silently dropped the call). Now: `elb('walker hook', { name, fn })`, and the hook actually runs. ``` // v3 (silently ignored) elb('walker hook', 'prePush', (params, ...args) => params.fn(...args)); // v4 elb('walker hook', { name: 'prePush', fn: (params, ...args) => params.fn(...args), }); ``` ### `walker on`[​](#walker-on "Direct link to walker-on") Was: `elb('walker on', type, rules)`. Now: `elb('walker on', { type, rules })`. ``` // v3 elb('walker on', 'consent', { marketing: (consent, context) => { // ... }, }); // v4 elb('walker on', { type: 'consent', rules: { marketing: (consent, context) => { // ... }, }, }); ``` ## Programmatic APIs drop the `options` argument[​](#programmatic-apis-drop-the-options-argument "Direct link to programmatic-apis-drop-the-options-argument") If your code calls into the collector directly (rather than through `elb`), three signatures lost their third `options` parameter: * `collector.command(name, data)` (was `command(name, data, options)`) * `addDestination(collector, destination)` (was `addDestination(collector, destination, config)`) * `commonHandleCommand(collector, name, data)` (was `commonHandleCommand(collector, name, data, options)`) For `walker destination`, pass the config through the Init object's `config` field instead of a third argument. For `walker run`, the `runState` partial is now optional (`elb('walker run')` with no second argument is valid). ## Step shape: `import` replaces `code: ""`[​](#step-shape-import-replaces-code-name "Direct link to step-shape-import-replaces-code-name") Every step (source, transformer, destination, store) now accepts a new `import?: string` field. With `package`, `import` selects a named export from that package. Using `package` alone still loads the default export. Inline code stays object-only: `code: { push, type?, init? }`. The legacy string form `code: ""` is no longer accepted. Rename it to `import: ""`: ``` "destinations": { "bigquery": { "package": "@walkeros/server-destination-gcp", - "code": "destinationBigQuery", + "import": "destinationBigQuery", "config": { "settings": { ... } } } } ``` Empty step entries (no `package`, `import`, `code`, or chain fields) are now valid no-ops in any of the four step kinds. Validation raises one of these error codes when a step shape is invalid: * `MISSING_PACKAGE`: `import` set without `package`. * `OBSOLETE_CODE_STRING`: legacy `code: ""` form. Rename hint points to `import`. * `INVALID_IMPORT`: `import` is not a non-empty string. * `INVALID_CODE_SHAPE`: `code` is not an object with a `push` function. * `UNKNOWN_KEY` and `CONFLICT` are retained from earlier validators. --- # FlowMap Diagrams Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. **Goal:** Replace ASCII art flow diagrams in docs with styled FlowMap components from @walkeros/explorer. **Architecture:** Add FlowMap to MDXComponents for global availability, then update each doc file to use `` instead of ASCII diagrams like `Sources → Collector → Destinations`. **Tech Stack:** React, @walkeros/explorer (FlowMap component), Docusaurus MDX *** ## Task 1: Add FlowMap to MDXComponents[​](#task-1-add-flowmap-to-mdxcomponents "Direct link to Task 1: Add FlowMap to MDXComponents") **Files:** * Modify: `website/src/theme/MDXComponents.js` **Step 1: Add FlowMap import** Add `FlowMap` to the explorer imports: ``` import { CodeBox, CodeSnippet, PropertyTable, DestinationInitDemo, DestinationDemo, LiveCode, FlowMap, // Add this } from '@walkeros/explorer'; ``` **Step 2: Export FlowMap in the default object** Add to the exports: ``` export default { // ... existing exports FlowMap, // Add this }; ``` **Step 3: Verify the website builds** Run: `cd /workspaces/developer/walkerOS/website && npm run build` Expected: Build succeeds without errors *** ## Task 2: Replace diagram in getting-started/index.mdx[​](#task-2-replace-diagram-in-getting-startedindexmdx "Direct link to Task 2: Replace diagram in getting-started/index.mdx") **Files:** * Modify: `website/docs/getting-started/index.mdx` **Step 1: Replace the ASCII flow diagram** Replace: ``` ``` Sources → Collector → Destinations └──────────── Flow ────────────┘ ``` ``` With: ``` ``` **Step 2: Replace the mapping diagram** Replace: ``` ``` Raw Input → \[SOURCE MAPPING] → walkerOS Event → \[DEST MAPPING] → Vendor Format ``` ``` With: ``` ``` **Step 3: Verify page renders** Run: `cd /workspaces/developer/walkerOS/website && npm run start` Expected: Navigate to docs homepage, FlowMap diagrams render correctly *** ## Task 3: Replace diagram in collector/index.mdx[​](#task-3-replace-diagram-in-collectorindexmdx "Direct link to Task 3: Replace diagram in collector/index.mdx") **Files:** * Modify: `website/docs/collector/index.mdx` **Step 1: Find and replace the flow diagram** Replace the ASCII `Sources → Collector → Destinations` with: ``` ``` **Step 2: Replace the destination mapping diagram** Replace: `walkerOS Event → [DESTINATION MAPPING] → Vendor Format (GA4, API, etc.)` With: ``` ``` *** ## Task 4: Replace diagram in destinations/index.mdx[​](#task-4-replace-diagram-in-destinationsindexmdx "Direct link to Task 4: Replace diagram in destinations/index.mdx") **Files:** * Modify: `website/docs/destinations/index.mdx` **Step 1: Replace the flow diagram** Replace: `Sources → Collector → Destinations → External Platforms` With: ``` ``` *** ## Task 5: Replace diagram in sources/web/browser/index.mdx[​](#task-5-replace-diagram-in-sourceswebbrowserindexmdx "Direct link to Task 5: Replace diagram in sources/web/browser/index.mdx") **Files:** * Modify: `website/docs/sources/web/browser/index.mdx` **Step 1: Replace the flow diagram** Replace: `Browser Source → Collector → Destinations` With: ``` ``` *** ## Task 6: Replace diagram in sources/web/dataLayer/index.mdx[​](#task-6-replace-diagram-in-sourceswebdatalayerindexmdx "Direct link to Task 6: Replace diagram in sources/web/dataLayer/index.mdx") **Files:** * Modify: `website/docs/sources/web/dataLayer/index.mdx` **Step 1: Replace the flow diagram** Replace: `DataLayer Source → Collector → Destinations` With: ``` ``` *** ## Task 7: Replace diagram in destinations/web/gtag/index.mdx[​](#task-7-replace-diagram-in-destinationswebgtagindexmdx "Direct link to Task 7: Replace diagram in destinations/web/gtag/index.mdx") **Files:** * Modify: `website/docs/destinations/web/gtag/index.mdx` **Step 1: Replace the flow diagram** Replace: `Sources → Collector → Gtag Destination` With: ``` ``` *** ## Task 8: Replace diagrams in server source docs[​](#task-8-replace-diagrams-in-server-source-docs "Direct link to Task 8: Replace diagrams in server source docs") **Files:** * Modify: `website/docs/sources/server/fetch.mdx` * Modify: `website/docs/sources/server/aws.mdx` * Modify: `website/docs/sources/server/express.mdx` * Modify: `website/docs/sources/server/gcp.mdx` **Step 1: Replace fetch.mdx diagram** Replace: `Fetch Source → Collector → Destinations` With: ``` ``` **Step 2: Replace aws.mdx diagram** Replace: `AWS Lambda Source → Collector → Destinations` With: ``` ``` **Step 3: Replace express.mdx diagram** Replace: `Express Source → Collector → Destinations` With: ``` ``` **Step 4: Replace gcp.mdx diagram** Replace: `GCP Source → Collector → Destinations` With: ``` ``` *** ## Task 9: Replace diagram in getting-started/event-model.mdx[​](#task-9-replace-diagram-in-getting-startedevent-modelmdx "Direct link to Task 9: Replace diagram in getting-started/event-model.mdx") **Files:** * Modify: `website/docs/getting-started/event-model.mdx` **Step 1: Replace the mapping diagram** Replace: `walkerOS Event → [MAPPING] → Destination Format` With: ``` ``` *** ## Task 10: Final verification[​](#task-10-final-verification "Direct link to Task 10: Final verification") **Step 1: Build the website** Run: `cd /workspaces/developer/walkerOS/website && npm run build` Expected: Build succeeds **Step 2: Visual verification** Run: `cd /workspaces/developer/walkerOS/website && npm run serve` Expected: All FlowMap diagrams render correctly with hand-drawn styling *** ## Notes[​](#notes "Direct link to Notes") * ASCII diagrams that are inline links like `[See Bundled Mode setup →]` should NOT be replaced (these are navigation links, not flow diagrams) * The `highlight: true` prop emphasizes the current page's topic in the flow * All FlowMap components will automatically support light/dark themes via explorer's CSS variables --- # Sources Sources are the **entry points** for event data in walkerOS. They capture events from different environments and formats, then transform them into standardized events that the collector can process. ## How sources work[​](#how-sources-work "Direct link to How sources work") Sources capture events in their native format and send them to the collector: ## Available sources[​](#available-sources "Direct link to Available sources") ### Web[​](#web "Direct link to Web") #### Browser[​](#browser "Direct link to Browser") Captures events from web pages using DOM attributes: * DOM interactions (clicks, form submissions, visibility) * Automatic session and pageview tracking * Custom data attributes (`data-elb-*`) [Learn more →](/docs/sources/web/browser/.md) #### dataLayer[​](#datalayer "Direct link to dataLayer") Integrates with existing analytics implementations: * Works with GA4 and GTM dataLayer * Event transformation and filtering * Gradual migration support [Learn more →](/docs/sources/web/dataLayer/.md) ### Server[​](#server "Direct link to Server") Server sources accept HTTP requests containing walkerOS events and forward them to the collector. #### Express[​](#express "Direct link to Express") Turn-key HTTP event collection server with Express.js: * Standalone or embedded: Start server automatically or integrate with existing Express app * CORS support and health checks * Pixel tracking with 1x1 transparent GIF [Learn more →](/docs/sources/server/express.md) #### Fetch[​](#fetch "Direct link to Fetch") Web Standard Fetch API source for edge and serverless platforms: * Platform agnostic: Cloudflare Workers, Vercel Edge, Deno, Bun, Node.js 18+ * Web Standard `(Request) => Response` signature * Batch processing and CORS support [Learn more →](/docs/sources/server/fetch.md) #### AWS Lambda[​](#aws-lambda "Direct link to AWS Lambda") AWS Lambda HTTP handler for event collection: * Multi-platform: API Gateway REST (v1), HTTP API (v2), Function URLs * Auto-detection of API Gateway version * Pixel tracking and health checks [Learn more →](/docs/sources/server/aws.md) #### GCP Cloud Functions[​](#gcp-cloud-functions "Direct link to GCP Cloud Functions") Google Cloud Functions HTTP handler for event collection: * Plug-and-play: Direct assignment to Cloud Functions handler * Batch processing support * Configurable CORS [Learn more →](/docs/sources/server/gcp.md) ## Basic setup[​](#basic-setup "Direct link to Basic setup") Sources are configured in `startFlow`: ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; const { collector, elb } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { pageview: true, session: true, prefix: 'data-elb', }, }, }, }, destinations: { /* your destinations */ }, }); // elb is now browser.push (enhanced with DOM commands) // Use elb for manual event tracking await elb('product view', { id: 'P123', name: 'Laptop' }); ``` ## Primary source[​](#primary-source "Direct link to Primary source") `startFlow` returns an `elb` function based on your sources: * **With sources**: Returns first source's `push` method (enhanced features) * **Multiple sources**: First source is primary by default * **No sources**: Returns `collector.push` (basic functionality) ### Override primary source[​](#override-primary-source "Direct link to Override primary source") ``` const { elb } = await startFlow({ sources: { browser: { code: sourceBrowser }, dataLayer: { code: sourceDataLayer, primary: true }, // Override }, }); // elb is now dataLayer.push instead of browser.push ``` ## Reactive event handling[​](#reactive-event-handling "Direct link to Reactive event handling") Sources can react to collector events via the optional `on` method: ``` const mySource: Source.Init = async (config, env) => { return { type: 'custom', config, on: async (event, context) => { if (event === 'consent') { // React to consent changes } }, }; }; ``` **Available Events:** * `consent` - Consent state changes * `session` - Session lifecycle events * `ready` - Collector ready state * `run` - Collector run events ## Creating custom sources[​](#creating-custom-sources "Direct link to Creating custom sources") See [Create Your Own Source](/docs/sources/create-your-own.md) for a complete guide to building custom sources. ## Configuration[​](#configuration "Direct link to Configuration") These fields are available on every source, regardless of package. They wrap the package-specific `settings` field, which is documented on each source's page. | Property | Type | Description | More | | ------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `consent` | `WalkerOS.Consent` | Required consent states to process any events | | | `data` | `any \| any` | Global data transformation applied to all events | | | `include` | `Array` | Event sections to flatten into context.data | | | `mapping` | `Mapping.Rules` | Entity-action specific mapping rules | | | `policy` | `Mapping.Policy` | Pre-processing policy rules applied before mapping | | | `settings` | `Source.Settings` | Implementation-specific configuration | | | `credentials` | `Source.Credentials` | Optional credentials (source-defined shape) | | | `env` | `Source.BaseEnv` | Environment dependencies (platform-specific) | | | `id` | `string` | Source identifier (defaults to source key) | | | `primary` | `boolean` | Mark as primary (only one can be primary) | | | `require` | `Array` | Defer source initialization until these collector events fire (e.g., \["consent"]) | | | `logger` | `Logger.Config` | | | | `async` | `boolean` | Respond-first acknowledgement for response-producing server sources (express today; future fetch/lambda). When true (the default for such sources) the source responds 2xx ("accepted") before the event is delivered; when false it waits for delivery to settle. Browser/dataLayer sources have no HTTP response to defer and ignore it. A 2xx means accepted, not delivered. Default is per source type. | | | `setup` | `boolean \| object` | One-time setup options applied during source registration (boolean enables defaults, object configures specifics) | | | `ingest` | `any \| any` | Ingest metadata extraction mapping. Extracts values from raw request objects (Express req, Lambda event) using mapping syntax. | | | `disabled` | `boolean` | Completely skip this source (no init, no event capture) | | | `state` | `State.Config \| State.Config[]` | Declarative store get/set operations applied around this source | | | `init` | `boolean` | Init lifecycle flag set by collector to true after Instance.init() runs | | ## Next steps[​](#next-steps "Direct link to Next steps") * **[Browser source](/docs/sources/web/browser/.md)** - DOM-based tracking * **[DataLayer source](/docs/sources/web/dataLayer/.md)** - Legacy integration * **[Create your own](/docs/sources/create-your-own.md)** - Custom source guide --- # Create Your Own Source A source captures events from an environment (browser, server, third-party API) and forwards them to the walkerOS collector. ## The source interface[​](#the-source-interface "Direct link to The source interface") Sources are async functions that receive a context object and return a source instance: ``` type Source.Init = ( context: Source.Context, ) => Source.Instance | Promise>; ``` The context contains: ``` interface Source.Context { config: Partial>; // Settings, mapping, options env: Types['env']; // Environment (push, logger) logger: Logger; // Logging functions id: string; // Source identifier collector: Collector.Instance; // Collector reference } ``` The returned instance must implement: ``` interface Source.Instance { type: string; // Unique identifier config: Source.Config; // Merged configuration push: Elb.Fn; // Function to send events (forwards to env.elb) destroy?(): void | Promise; // Optional cleanup on?(event, context): void | Promise; // Optional reactive hooks } ``` ## Types bundle[​](#types-bundle "Direct link to Types bundle") Sources use a `Types` interface to bundle all TypeScript types: ``` import type { Source, Elb } from '@walkeros/core'; // 1. Define your settings interface Settings { captureClicks?: boolean; prefix?: string; } // 2. Define environment dependencies interface Env extends Source.BaseEnv { customAPI?: YourAPIType; } // 3. Bundle them together interface Types extends Source.Types {} ``` **The 4 type parameters**: 1. **Settings**: Source configuration options 2. **Mapping**: Event mapping (usually `never` for sources) 3. **Push**: External push signature (what `source.push` exposes) 4. **Env**: Internal dependencies (what source calls via `env.elb`) ## Context destructuring[​](#context-destructuring "Direct link to Context destructuring") The `context` parameter contains everything your source needs: ``` export const mySource: Source.Init = async (context) => { // Destructure what you need from context const { config = {}, env, logger, id } = context; const { push: envPush, customAPI } = env; // customAPI validation only needed if required by your source if (!customAPI) { throw new Error('Source requires customAPI in environment'); } // Use env.push to send events to collector await envPush({ name: 'my event', data: { value: 'example' } }); }; ``` The collector provides `env.push` (formerly `env.elb`). You provide other dependencies (like `window`, `document`, custom APIs) when configuring the source. ## Per-scope context (server sources)[​](#per-scope-context-server-sources "Direct link to Per-scope context (server sources)") A single source factory instance handles many concurrent invocations: an Express server processes overlapping requests, a Lambda may be reused across invocations, a queue consumer processes batches. Each logical unit of work, an HTTP request, a queue message, a websocket frame, is a **scope**. To keep ingest metadata and response delegates isolated between concurrent scopes, server sources wrap each invocation with `context.withScope`: ``` const push = async (req: Request, res: Response): Promise => { const respond = createRespond((options) => { // wire options into res.send / res.json }); // Each call to withScope builds a fresh Ingest from `req` via // `config.ingest` mapping, captures `respond`, and runs the body // with a per-scope env. Concurrent scopes never crosstalk. await context.withScope(req, respond, async (env) => { const eventData = req.body && typeof req.body === 'object' ? req.body : {}; await env.push(eventData); respond({ body: { success: true } }); }); }; ``` Inside the body, `env.push` carries the scope's ingest and respond all the way to destinations. Transformers and destinations read `env.respond` to delegate the HTTP response; `context.ingest` for source-extracted metadata. The scope ends when the body resolves. **Browser sources (and other single-scope sources) skip `withScope`.** A browser tab is a single logical scope for its entire lifetime; calling `env.push` directly on the factory env is correct. Only server sources handling concurrent inbound work need `withScope`. ## Minimal example[​](#minimal-example "Direct link to Minimal example") ``` import type { Source, Elb } from '@walkeros/core'; interface Settings { prefix?: string; } interface Env extends Source.BaseEnv {} interface Types extends Source.Types {} export const sourceCustom: Source.Init = async (context) => { const { config = {}, env } = context; const { push: envPush } = env; const settings: Source.Settings = { prefix: 'custom', ...config?.settings, }; const fullConfig: Source.Config = { ...config, settings, }; return { type: 'custom', config: fullConfig, push: envPush, }; }; ``` ## Complete example: Event API source[​](#complete-example-event-api-source "Direct link to Complete example: Event API source") Capturing events from a third-party API with event listeners: ``` import type { Source, Elb } from '@walkeros/core'; // Your external API interface interface ExternalAPI { on(event: string, handler: (data: unknown) => void): void; off(event: string, handler: (data: unknown) => void): void; } interface Settings { captureInteractions?: boolean; captureErrors?: boolean; prefix?: string; } interface Env extends Source.BaseEnv { api?: ExternalAPI; } interface Types extends Source.Types {} export const sourceEventAPI: Source.Init = async (context) => { const { config = {}, env } = context; const { push: envPush, api } = env; if (!api) throw new Error('Source requires api instance'); const settings: Source.Settings = { captureInteractions: true, captureErrors: true, prefix: 'app', ...config?.settings, }; const handlers = new Map void>(); // Register event handler const register = ( event: string, transform: (data: unknown) => { name: string; data: unknown }, ) => { const handler = (data: unknown) => { const transformed = transform(data); envPush(transformed); }; api.on(event, handler); handlers.set(event, handler); }; // Set up event listeners based on settings if (settings.captureInteractions) { register('userAction', (data) => ({ name: `${settings.prefix} interaction`, data, })); } if (settings.captureErrors) { register('error', (data) => ({ name: `${settings.prefix} error`, data, })); } return { type: 'event-api', config: { ...config, settings }, push: envPush, destroy: async () => { handlers.forEach((handler, event) => api.off(event, handler)); }, }; }; ``` ## Using your source[​](#using-your-source "Direct link to Using your source") ``` import { startFlow } from '@walkeros/collector'; import { sourceEventAPI } from './sourceEventAPI'; const externalAPI = getYourAPI(); const { elb } = await startFlow({ sources: { eventAPI: { code: sourceEventAPI, config: { settings: { captureInteractions: true, prefix: 'myapp', }, }, env: { api: externalAPI, // Collector provides elb automatically }, }, }, }); ``` ## Testing your source[​](#testing-your-source "Direct link to Testing your source") ### Test Utilities[​](#test-utilities "Direct link to Test Utilities") ``` // __tests__/test-utils.ts export function createMockPush() { const mock = jest.fn(); mock.mockResolvedValue({ event: { id: 'test-id' }, ok: true, }); return mock; } export function createMockAPI() { return { on: jest.fn(), off: jest.fn(), }; } ``` ### Test Example[​](#test-example "Direct link to Test Example") ``` import { sourceEventAPI } from '../index'; import type { Source, Collector } from '@walkeros/core'; import { createMockLogger } from '@walkeros/core'; import { createMockPush, createMockAPI } from './test-utils'; // Helper to create source context for testing function createSourceContext( config: Partial> = {}, env: Partial = {}, ): Source.Context { return { config, env: env as Types['env'], logger: env.logger || createMockLogger(), id: 'test-event-api', collector: {} as Collector.Instance, }; } describe('Event API Source', () => { it('validates required dependencies', async () => { const push = createMockPush(); await expect( sourceEventAPI(createSourceContext({}, { push })) ).rejects.toThrow('requires api'); }); it('registers event listeners', async () => { const api = createMockAPI(); const push = createMockPush(); await sourceEventAPI( createSourceContext( { settings: { captureInteractions: true } }, { push, api } ) ); expect(api.on).toHaveBeenCalledWith('userAction', expect.any(Function)); }); it('transforms and forwards events', async () => { const api = createMockAPI(); const push = createMockPush(); await sourceEventAPI(createSourceContext({}, { push, api })); // Simulate event const handler = api.on.mock.calls[0][1]; handler({ action: 'click' }); expect(push).toHaveBeenCalledWith({ name: 'app interaction', data: { action: 'click' }, }); }); it('cleans up on destroy', async () => { const api = createMockAPI(); const push = createMockPush(); const source = await sourceEventAPI(createSourceContext({}, { push, api })); await source.destroy?.(); expect(api.off).toHaveBeenCalled(); }); }); ``` ### Reading a step back from the collector[​](#reading-a-step-back-from-the-collector "Direct link to Reading a step back from the collector") When an integration test needs to call a step's raw `push` directly through the collector (a source's `push(req, res)`, a destination's `push(event, ctx)`, etc.), the bag types (`collector.sources`, `collector.destinations`, `collector.transformers`, `collector.stores`) erase the per-step generic. The `push` signature collapses to `Elb.Fn` on read. Use the typed accessors from `@walkeros/core` to recover the narrow type without casts: ``` import { Source } from '@walkeros/core'; import type { Types as ExpressTypes } from '@walkeros/server-source-express'; const expressSource = Source.getSource(collector, 'express'); await expressSource.push(req, res); // typed! ``` Symmetric helpers exist for every step kind: `Source.getSource`, `Destination.getDestination`, `Transformer.getTransformer`, `Store.getStore`. Each throws ` not found: ` when the id is unknown. The helpers are opt-in and have no runtime cost. ## Setup lifecycle (optional)[​](#setup-lifecycle-optional "Direct link to Setup lifecycle (optional)") Sources may implement an optional `setup()` lifecycle for one-time, operator-time provisioning, for example registering a webhook callback or creating a Pub/Sub subscription. Setup runs only when an operator explicitly invokes `walkeros setup source.`; the runtime never auto-invokes it. Opt in via `config.setup` in the flow config (`true` or an object). See [Setup lifecycle](/docs/destinations/create-your-own.md#setup-lifecycle-optional) on the destinations page for the full concept, and [walkeros setup CLI command](/docs/apps/cli.md#setup-command) for the operator-facing command. ## Conditional activation with `require`[​](#conditional-activation-with-require "Direct link to conditional-activation-with-require") Sources can declare dependencies on collector events. A source with `require` won't receive lifecycle events (via `on()`) until all specified events have fired: ``` await startFlow({ sources: { session: { code: sessionSource, config: { require: ['consent'] }, }, dataLayer: { code: dataLayerSource, config: { require: ['user'] }, }, }, }); ``` **How it works:** * Sources are registered in `collector.sources` immediately, and the collector runs `Instance.init()` on each one eagerly after registration. `Source.Config.init` flips to `true` once init has run. * `require` does **not** gate code execution. It gates `on()` delivery. Lifecycle events targeted at a source whose `require` is unmet are buffered in the source's `Instance.queueOn` array. * After each collector event (`consent`, `user`, `session`, `run`, etc.), the collector decrements the matching entry from each source's `require` list. When a source's require is empty (and `config.init === true`), its queued lifecycle events are replayed by calling `source.on(type, data)` for each entry, then the queue is cleared. * Chains work naturally: `consent` clears the session source's `require`, which fires `user`, which clears the dataLayer source's `require`, which then receives any queued events. **Common patterns:** | Require | Use case | | -------------------- | --------------------------------------- | | `['consent']` | Wait for CMP consent before tracking | | `['user']` | Wait for identity resolution | | `['session']` | Wait for session detection | | `['consent', 'run']` | Wait for both consent and collector run | ## Common patterns[​](#common-patterns "Direct link to Common patterns") ### Polling for API Readiness[​](#polling-for-api-readiness "Direct link to Polling for API Readiness") When the external API isn't immediately available: ``` async function waitForAPI(getter: () => T | null): Promise { return new Promise((resolve) => { const check = () => { const instance = getter(); if (instance) resolve(instance); else setTimeout(check, 100); }; check(); }); } // Usage const api = await waitForAPI(() => window.myAPI || null); ``` ## Source as adapter pattern[​](#source-as-adapter-pattern "Direct link to Source as adapter pattern") Sources bridge external systems and the collector: ``` External System ←→ Source (Adapter) ←→ Collector ``` **Two interfaces:** 1. **External (`source.push`)**: Platform-specific signature * Browser: `push(elem, data, options)` → Returns `Promise` * Server: `push(req, res)` → Returns `Promise` (writes HTTP response) * Your choice: Match your environment's needs 2. **Internal (`env.elb`)**: Standard collector interface * Always `Elb.Fn` - same across all sources * Sources translate external inputs → standard events → `env.elb(event)` **Example signatures:** ``` // Browser source export type Push = (elem?: Element, data?: Properties) => Promise; // HTTP handler source export type Push = (req: Request, res: Response) => Promise; // Standard event source export type Push = Elb.Fn; ``` The `Push` type parameter defines what your source exposes externally. Internally, all sources use `env.elb` to forward to the collector. ## Key concepts[​](#key-concepts "Direct link to Key concepts") * **Context pattern**: Sources receive a single context object with config, env, logger, id * **Validate custom dependencies**: Only check optional env properties your source needs * **Types bundle**: Use 4-parameter pattern for full type safety * **Adapter pattern**: External push adapts to environment, internal env.push stays standard * **Cleanup**: Implement `destroy()` to remove listeners * **Stateless**: Let collector manage state ## Package convention[​](#package-convention "Direct link to Package convention") Every walkerOS package includes machine-readable metadata for tooling and discovery. ### `walkerOS` field in package.json[​](#walkeros-field-in-packagejson "Direct link to walkeros-field-in-packagejson") ``` { "walkerOS": { "type": "source", "platform": "web" }, "keywords": ["walkeros", "walkeros-source"] } ``` | Field | Required | Description | | ---------- | -------- | ------------------------------------------ | | `walkerOS` | Yes | Object with `type` and `platform` metadata | ### Build-time generation[​](#build-time-generation "Direct link to Build-time generation") Use `buildDev()` from the shared tsup config to auto-generate `walkerOS.json`: ``` import { buildDev } from '@walkeros/config/tsup'; ``` This file contains your package's JSON Schemas and examples, enabling MCP tools and the CLI to validate configurations without installing your package. ### Optional: Hints[​](#optional-hints "Direct link to Optional: Hints") Packages can export a `hints` record from `src/dev.ts` to provide lightweight, actionable context beyond schemas and examples, such as capture timing, event formats, or troubleshooting tips. Hints are serialized into `walkerOS.json` and surfaced via MCP tools. See the `walkeros-create-source` skill for details. ## Publishing checklist[​](#publishing-checklist "Direct link to Publishing checklist") * [ ] `walkerOS` field in package.json * Keywords include `walkeros` and `walkeros-source` * [ ] `buildDev()` in tsup.config.ts * [ ] `dist/walkerOS.json` generated on build * [ ] `npm run test` passes * [ ] `npm run lint` passes ## Next steps[​](#next-steps "Direct link to Next steps") * Review [Browser Source](/docs/sources/web/browser/.md) for DOM patterns * Review [DataLayer Source](/docs/sources/web/dataLayer/.md) for interception patterns * Learn about [creating destinations](/docs/destinations/create-your-own.md) --- # Server sources Server sources accept HTTP requests containing walkerOS events and forward them to the collector. Choose based on your deployment environment. ## Available sources[​](#available-sources "Direct link to Available sources") | Source | Platform | Use Case | | -------------------------------------------------- | --------------- | ------------------------------------------ | | [Express](/docs/sources/server/express.md) | Node.js | Standalone server or existing Express app | | [Fetch](/docs/sources/server/fetch.md) | Edge/Serverless | Cloudflare Workers, Vercel Edge, Deno, Bun | | [AWS Lambda](/docs/sources/server/aws.md) | AWS | API Gateway, Lambda Function URLs | | [GCP Cloud Functions](/docs/sources/server/gcp.md) | Google Cloud | Cloud Functions HTTP triggers | ## Choosing a source[​](#choosing-a-source "Direct link to Choosing a source") **Express** - Best for traditional Node.js deployments, Docker containers, or when you need full control over the HTTP server. **Fetch** - Best for edge computing and serverless platforms that support the Web Standard Fetch API. **AWS Lambda** - Best for AWS-native deployments with API Gateway or Lambda Function URLs. **GCP Cloud Functions** - Best for Google Cloud deployments using Cloud Functions. ## Common features[​](#common-features "Direct link to Common features") All server sources provide: * **JSON event ingestion** (POST) * **Pixel tracking** (GET with 1x1 GIF response) * **CORS support** (configurable) * **Health checks** (liveness/readiness endpoints) --- [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/aws) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/aws)[ ](https://www.npmjs.com/package/@walkeros/server-source-aws) [Package](https://www.npmjs.com/package/@walkeros/server-source-aws) # AWS Lambda AWS Lambda source for walkerOS. Works across API Gateway REST (v1), API Gateway HTTP (v2), Lambda Function URLs, and direct invocation. Auto-detects the API Gateway version, supports optional pixel tracking with a 1x1 GIF response, and exposes a built-in health check endpoint. The `@walkeros/server-source-aws` package also ships [`sourceSqs`](/docs/sources/server/sqs.md) for ingesting from SQS queues; this page covers the Lambda handler only. Where this fits The AWS Lambda source is a **server source** in the walkerOS flow: It receives events via HTTP and forwards them to your destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-aws ``` * Integrated * Bundled ``` import { sourceLambda } from '@walkeros/server-source-aws'; import { startFlow } from '@walkeros/collector'; let handler: any; async function init() { if (handler) return handler; const { sources } = await startFlow({ sources: { lambda: { code: sourceLambda, config: { settings: { cors: true, healthPath: '/health' }, }, }, }, destinations: { // Your destinations }, }); handler = sources.lambda.push; return handler; } export const main = async (event: any, context: any) => { const h = await init(); return h(event, context); }; export { main as handler }; ``` Add to your `flow.json` sources: ``` "sources": { "lambda": { "package": "@walkeros/server-source-aws", "config": { "settings": { "cors": true, "healthPath": "/health" } } } } ``` Server sources require platform-specific handlers. For containerized deployments, see [Docker](/docs/apps/docker.md). [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------------------- | ------------------- | --------------------------------------------------------------------------------------------- | ---- | | `cors` | `boolean \| object` | CORS configuration: false = disabled, true = allow all origins, object = custom configuration | | | `timeout` | `integer` | Request timeout in milliseconds (max: 900000 for Lambda) | | | `enablePixelTracking` | `boolean` | Enable GET requests with 1x1 transparent GIF response for pixel tracking | | | `healthPath` | `string` | Health check endpoint path (e.g., /health) | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### API Gateway v1 POST A REST API Gateway v1 POST request with a JSON body is converted into a walker elb event. Event ``` { "httpMethod": "POST", "path": "/collect", "requestContext": { "requestId": "req-789", "identity": { "sourceIp": "203.0.113.42" } }, "queryStringParameters": null, "body": "{\"event\":\"page view\",\"data\":{\"title\":\"Home\"}}", "isBase64Encoded": false } ``` Out ``` elb({ "name": "page view", "data": { "title": "Home" } }) ``` ### Lambda GET An API Gateway v2 HTTP GET with query parameters is parsed into an elb event payload. Event ``` { "version": "2.0", "requestContext": { "http": { "method": "GET", "path": "/collect" }, "requestId": "req-456" }, "rawQueryString": "e=page+view&d=%7B%22title%22%3A%22Home%22%7D", "isBase64Encoded": false } ``` Out ``` elb({ "e": "page view", "d": "{\"title\":\"Home\"}" }) ``` ### Lambda POST An API Gateway v2 HTTP POST with a JSON body is converted into a walker elb event. Event ``` { "version": "2.0", "requestContext": { "http": { "method": "POST", "path": "/collect" }, "requestId": "req-123" }, "body": "{\"event\":\"page view\",\"data\":{\"title\":\"Home\"}}", "isBase64Encoded": false } ``` Out ``` elb({ "name": "page view", "data": { "title": "Home" } }) ``` ## Ingest metadata[​](#ingest-metadata "Direct link to Ingest metadata") Extract request metadata from Lambda events and forward it through the pipeline. `config.ingest` must use the `map` operator. Keys are output field names; values are direct field paths on the Lambda event (no prefix). A bare object like `{ ip: 'requestContext.identity.sourceIp' }` is silently inert: without the `map` operator the source passes the whole event through and no field is extracted. ``` const { sources } = await startFlow({ sources: { lambda: { code: sourceLambda, config: { settings: { cors: true }, ingest: { map: { ip: { key: 'requestContext.identity.sourceIp' }, // v1 ua: { key: 'requestContext.identity.userAgent' }, // v1 // Or for API Gateway v2: // ip: { key: 'requestContext.http.sourceIp' }, // ua: { key: 'requestContext.http.userAgent' }, }, }, }, }, }, }); ``` ### Available ingest paths[​](#available-ingest-paths "Direct link to Available ingest paths") **API Gateway v1 (REST API):** | Path | Description | | ----------------------------------- | ----------------- | | `requestContext.identity.sourceIp` | Client IP address | | `requestContext.identity.userAgent` | User agent string | | `headers.*` | HTTP headers | | `httpMethod` | HTTP method | **API Gateway v2 (HTTP API) / Function URLs:** | Path | Description | | ------------------------------- | ----------------- | | `requestContext.http.sourceIp` | Client IP address | | `requestContext.http.userAgent` | User agent string | | `requestContext.http.method` | HTTP method | | `headers.*` | HTTP headers | ## Supported platforms[​](#supported-platforms "Direct link to Supported platforms") | Platform | Status | | ------------------------- | --------- | | API Gateway REST API (v1) | Supported | | API Gateway HTTP API (v2) | Supported | | Lambda Function URLs | Supported | | Direct Lambda invocation | Supported | --- [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/express) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/express)[ ](https://www.npmjs.com/package/@walkeros/server-source-express) [Package](https://www.npmjs.com/package/@walkeros/server-source-express) # Express Turn-key HTTP event collection server with Express.js. Runs standalone or embeds inside an existing Express app, handles JSON POST events, pixel tracking via GET with a 1x1 transparent GIF, and configurable CORS. Where this fits The Express source is a **server source** in the walkerOS flow: It receives events via HTTP (POST/GET) and forwards them to configured destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-express ``` path setting renamed to paths The `path` setting has been renamed to `paths` (array). The old `path` still works but is deprecated and will be removed in the next major version. ``` // Before (deprecated) settings: { path: '/events' } // After settings: { paths: ['/events'] } ``` * Integrated * Bundled ### Standalone server[​](#standalone-server "Direct link to Standalone server") ``` import { startFlow } from '@walkeros/collector'; import { sourceExpress } from '@walkeros/server-source-express'; const { collector } = await startFlow({ sources: { express: { code: sourceExpress, config: { settings: { port: 8080, cors: true, }, }, }, }, destinations: { // Your destinations }, }); // Server running at http://localhost:8080 // POST /collect - JSON event ingestion // GET /collect - Pixel tracking ``` ### App-only mode[​](#app-only-mode "Direct link to App-only mode") ``` const { collector } = await startFlow({ sources: { express: { code: sourceExpress, config: { settings: { // No port = app only mode paths: ['/events'], }, }, }, }, }); // Access Express app for custom integration const app = collector.sources.express.app; app.use(yourAuthMiddleware); app.listen(3000); ``` Add to your `flow.json` sources: ``` "sources": { "express": { "package": "@walkeros/server-source-express", "config": { "settings": { "port": 8080, "cors": true } } } } ``` Server sources require platform-specific handlers. For containerized deployments, see [Docker](/docs/apps/docker.md). [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------- | ---- | | `port` | `integer` | HTTP server port to listen on. Use 0 for random available port. If not provided, server will not start (app only mode) | | | `path` | `string` | Deprecated: use paths instead | | | `paths` | `Array` | Route paths to register. String shorthand registers GET+POST. RouteConfig allows per-route method control. | | | `cors` | `boolean \| object` | CORS configuration: false = disabled, true = allow all origins (default), object = custom configuration | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Pixel GET An Express GET to /collect with query parameters is parsed into an elb event payload for pixel tracking. Event ``` { "method": "GET", "path": "/collect", "query": { "e": "page view", "d": "{\"title\":\"Home\"}" } } ``` Out ``` elb({ "e": "page view", "d": "{\"title\":\"Home\"}" }) ``` ### POST event An Express POST to /collect with a JSON body becomes a single walker elb event. Event ``` { "method": "POST", "path": "/collect", "body": { "name": "page view", "data": { "title": "Home", "url": "https://example.com/" } } } ``` Out ``` elb({ "name": "page view", "data": { "title": "Home", "url": "https://example.com/" } }) ``` ## HTTP methods[​](#http-methods "Direct link to HTTP methods") | Method | Endpoint | Description | | ------- | ---------- | -------------------------------- | | POST | `/collect` | JSON event ingestion | | GET | `/collect` | Pixel tracking (returns 1x1 GIF) | | OPTIONS | `/collect` | CORS preflight | Health checks Health check endpoints (`/health` and `/ready`) are provided by the [runner](/docs/apps/runner.md), not by individual sources. This means health checks work regardless of which source type you use. ## Ingest metadata[​](#ingest-metadata "Direct link to Ingest metadata") Extract request metadata (IP, user agent, headers) and forward it through the pipeline to transformers and destinations. `config.ingest` must use the `map` operator. Keys are output field names; values are direct field paths on the request scope (no `req.` prefix). A bare object like `{ ip: 'ip' }` is silently inert: without the `map` operator the source passes the whole request through and no field is extracted. ``` import { startFlow } from '@walkeros/collector'; import { sourceExpress } from '@walkeros/server-source-express'; const { collector } = await startFlow({ sources: { express: { code: sourceExpress, config: { settings: { port: 8080 }, ingest: { map: { ip: { key: 'ip' }, ua: { key: 'headers.user-agent' }, origin: { key: 'headers.origin' }, referer: { key: 'headers.referer' }, }, }, }, }, }, }); ``` ## Available ingest paths[​](#available-ingest-paths "Direct link to Available ingest paths") | Path | Description | | ----------- | ------------------------------------------------ | | `ip` | Client IP address | | `headers.*` | HTTP headers (user-agent, origin, referer, etc.) | | `protocol` | Request protocol (http/https) | | `method` | HTTP method (GET, POST, etc.) | | `hostname` | Request hostname | | `url` | Full request URL | ## Advanced mapping[​](#advanced-mapping "Direct link to Advanced mapping") Use walkerOS mapping features for complex extraction: ``` ingest: { map: { // Simple path ip: { key: 'ip' }, // Custom function country: { fn: (req) => geoip.lookup(req.ip)?.country }, // Conditional extraction devMode: { key: 'headers.x-debug', condition: (req) => req.hostname === 'localhost', }, // Nested structure request: { map: { ua: { key: 'headers.user-agent' }, origin: { key: 'headers.origin' }, }, }, }, } ``` ## Example request[​](#example-request "Direct link to Example request") ``` curl -X POST http://localhost:8080/collect \ -H "Content-Type: application/json" \ -d '{"event":"page view","data":{"title":"Home"}}' ``` --- [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/fetch) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/fetch)[ ](https://www.npmjs.com/package/@walkeros/server-source-fetch) [Package](https://www.npmjs.com/package/@walkeros/server-source-fetch) # Fetch Web Standard Fetch API source for walkerOS. Platform-agnostic `(Request) => Response` signature that runs on Cloudflare Workers, Vercel Edge, Deno, Bun, and Node.js 18+. Supports batch processing, configurable CORS, and pixel tracking via a 1x1 transparent GIF for GET requests. Where this fits The Fetch source is a **server source** in the walkerOS flow: It receives events via the Fetch API and forwards them to your destinations. Works on any platform supporting Web Standards. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-fetch ``` path setting renamed to paths The `path` setting has been renamed to `paths` (array). The old `path` still works but is deprecated and will be removed in the next major version. ``` // Before (deprecated) settings: { path: '/events' } // After settings: { paths: ['/events'] } ``` * Integrated * Bundled ### Cloudflare Workers[​](#cloudflare-workers "Direct link to Cloudflare Workers") ``` import { sourceFetch } from '@walkeros/server-source-fetch'; import { startFlow } from '@walkeros/collector'; const { sources } = await startFlow({ sources: { fetch: { code: sourceFetch, config: { settings: { paths: ['/collect'], cors: true }, }, }, }, }); export default { fetch: sources.fetch.push }; ``` Add to your `flow.json` sources: ``` "sources": { "fetch": { "package": "@walkeros/server-source-fetch", "config": { "settings": { "paths": ["/collect"], "cors": true } } } } ``` Server sources require platform-specific handlers. For containerized deployments, see [Docker](/docs/apps/docker.md). [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------- | ------------------- | ------------------------------------------------------------------------------------------------------ | ---- | | `path` | `string` | Deprecated: use paths instead | | | `paths` | `Array` | Route paths to handle. String shorthand accepts GET+POST. RouteConfig allows per-route method control. | | | `cors` | `boolean \| object` | CORS configuration: false = disabled, true = allow all (default), object = custom | | | `maxRequestSize` | `integer` | Maximum request body size in bytes | | | `maxBatchSize` | `integer` | Maximum events per batch request | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Batch POST A fetch POST with a batch array produces one walker elb event per batched item preserving order. Event ``` { "method": "POST", "url": "http://localhost/collect", "body": { "batch": [ { "name": "page view", "data": { "title": "Home" } }, { "name": "button click", "data": { "id": "cta" } } ] } } ``` Out ``` elb({ "name": "page view", "data": { "title": "Home" } }); elb({ "name": "button click", "data": { "id": "cta" } }) ``` ### Pixel GET A fetch GET with query parameters in the URL is parsed into an elb event payload for pixel-style tracking. Event ``` { "method": "GET", "url": "http://localhost/collect?e=page+view&d=%7B%22title%22%3A%22Home%22%7D" } ``` Out ``` elb({ "e": "page view", "d": "{\"title\":\"Home\"}" }) ``` ### POST event A fetch POST request with a JSON body becomes a single walker elb event in a fetch-based server. Event ``` { "method": "POST", "url": "http://localhost/collect", "body": { "name": "page view", "data": { "title": "Docs", "url": "https://example.com/docs" } } } ``` Out ``` elb({ "name": "page view", "data": { "title": "Docs", "url": "https://example.com/docs" } }) ``` ## Ingest metadata[​](#ingest-metadata "Direct link to Ingest metadata") Extract request metadata and forward it through the pipeline. `config.ingest` must use the `map` operator. Keys are output field names; values are direct field paths on the `Request` (no `req.` prefix), or `{ fn }` for header access via `.get()`. A bare object like `{ url: 'url' }` is silently inert: without the `map` operator the source passes the whole request through and no field is extracted. ``` const { sources } = await startFlow({ sources: { fetch: { code: sourceFetch, config: { settings: { cors: true }, ingest: { map: { ua: { fn: (req) => req.headers.get('user-agent') }, origin: { fn: (req) => req.headers.get('origin') }, url: { key: 'url' }, }, }, }, }, }, }); ``` ### Available ingest paths[​](#available-ingest-paths "Direct link to Available ingest paths") | Path | Description | | --------------------- | -------------------------------------------------------- | | `url` | Full request URL | | `headers.get('name')` | Via function: `{ fn: (req) => req.headers.get('name') }` | note The Fetch API uses `Request` objects where headers are accessed via `.get()` method. Use mapping functions for header extraction. ## Usage[​](#usage "Direct link to Usage") ### Single event[​](#single-event "Direct link to Single event") ``` fetch('https://your-endpoint.com/collect', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ name: 'page view', data: { title: 'Home', path: '/' }, }), }); ``` ### Batch events[​](#batch-events "Direct link to Batch events") ``` fetch('https://your-endpoint.com/collect', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ batch: [ { name: 'page view', data: { title: 'Home' } }, { name: 'button click', data: { id: 'cta' } }, ], }), }); ``` --- [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/gcp) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/gcp)[ ](https://www.npmjs.com/package/@walkeros/server-source-gcp) [Package](https://www.npmjs.com/package/@walkeros/server-source-gcp) # GCP Cloud Functions Google Cloud Functions source for walkerOS. Lightweight runtime adapter with plug-and-play assignment to a Cloud Functions handler, batch processing for multiple events per request, and configurable CORS. The `@walkeros/server-source-gcp` package also ships [`sourcePubSubPull` and `sourcePubSubPush`](/docs/sources/server/pubsub.md) for ingesting from Pub/Sub topics; this page covers the Cloud Functions handler only. Where this fits The GCP Cloud Functions source is a **server source** in the walkerOS flow: It receives events via HTTP and forwards them to your destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-gcp @google-cloud/functions-framework ``` * Integrated * Bundled ``` import { sourceCloudFunction } from '@walkeros/server-source-gcp'; import { startFlow } from '@walkeros/collector'; import { http } from '@google-cloud/functions-framework'; const { sources } = await startFlow({ sources: { gcp: { code: sourceCloudFunction, config: { settings: { cors: true, batch: true }, }, }, }, destinations: { // Your destinations }, }); // Plug-and-play: source.push IS the Cloud Function handler http('walkerHandler', sources.gcp.push); ``` Add to your `flow.json` sources: ``` "sources": { "gcp": { "package": "@walkeros/server-source-gcp", "config": { "settings": { "cors": true, "batch": true } } } } ``` Server sources require platform-specific handlers. For containerized deployments, see [Docker](/docs/apps/docker.md). [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------- | ------------------- | --------------------------------------------------------------------------------------------- | ---- | | `cors` | `boolean \| object` | CORS configuration: false = disabled, true = allow all origins, object = custom configuration | | | `timeout` | `integer` | Request timeout in milliseconds (max: 540000 for GCP) | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Order POST A Cloud Function HTTP POST carrying an order payload becomes a walker order complete event. Event ``` { "method": "POST", "body": { "event": "order complete", "data": { "id": "ORD-700", "total": 99.99, "currency": "EUR" } }, "headers": { "content-type": "application/json" } } ``` Out ``` elb({ "name": "order complete", "data": { "id": "ORD-700", "total": 99.99, "currency": "EUR" } }) ``` ### POST event A GCP Cloud Function HTTP POST with a JSON body becomes a single walker elb event. Event ``` { "method": "POST", "body": { "event": "page view", "data": { "title": "Home", "url": "https://example.com/" } }, "headers": { "content-type": "application/json" } } ``` Out ``` elb({ "name": "page view", "data": { "title": "Home", "url": "https://example.com/" } }) ``` ## Request format[​](#request-format "Direct link to Request format") ### Single event[​](#single-event "Direct link to Single event") ``` { "event": "page view", "data": { "title": "Home Page", "path": "/" } } ``` ### Batch events[​](#batch-events "Direct link to Batch events") ``` { "events": [ { "event": "page view", "data": { "title": "Page 1" } }, { "event": "button click", "data": { "id": "btn1" } } ] } ``` ## Ingest metadata[​](#ingest-metadata "Direct link to Ingest metadata") Extract request metadata and forward it through the pipeline. `config.ingest` must use the `map` operator. Keys are output field names; values are direct field paths on the request scope (no `req.` prefix). A bare object like `{ ip: 'ip' }` is silently inert: without the `map` operator the source passes the whole request through and no field is extracted. ``` const { sources } = await startFlow({ sources: { gcp: { code: sourceCloudFunction, config: { settings: { cors: true }, ingest: { map: { ip: { key: 'ip' }, ua: { key: 'headers.user-agent' }, origin: { key: 'headers.origin' }, }, }, }, }, }, }); ``` ### Available ingest paths[​](#available-ingest-paths "Direct link to Available ingest paths") | Path | Description | | ----------- | --------------------------------------- | | `ip` | Client IP address | | `headers.*` | HTTP headers (user-agent, origin, etc.) | | `method` | HTTP method | | `hostname` | Request hostname | ## Pub/Sub[​](#pubsub "Direct link to Pub/Sub") The same `@walkeros/server-source-gcp` package also exports `sourcePubSubPull` (streaming pull subscriber) and `sourcePubSubPush` (HTTP webhook handler) for ingesting events from a Pub/Sub topic. See the [Pub/Sub source page](/docs/sources/server/pubsub.md) for full settings, lifecycle, decoders, OIDC verification, and setup reference. --- # Google Pub/Sub [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/gcp) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/gcp)[ ](https://www.npmjs.com/package/@walkeros/server-source-gcp) [Package](https://www.npmjs.com/package/@walkeros/server-source-gcp) Subscribes to a [Google Cloud Pub/Sub](https://cloud.google.com/pubsub) topic and forwards each delivered message to the walkerOS collector. Ships two delivery models from the same package: a long-running streaming pull subscriber for container deployments, and an HTTP push webhook handler for serverless. Decoders for JSON, text, and raw payloads. Idempotent subscription provisioning with optional dead-letter wiring. Optional OIDC verification on the push handler. The source ships inside `@walkeros/server-source-gcp` alongside the Cloud Functions handler; install the package once and import `sourcePubSubPull` or `sourcePubSubPush`. Where this fits Pub/Sub is a **server source** in the walkerOS flow: Forwards each Pub/Sub message to the collector. Pull mode runs a streaming subscriber in long-lived containers; push mode receives HTTP envelopes in serverless deployments. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-gcp ``` ## Choosing pull vs push[​](#choosing-pull-vs-push "Direct link to Choosing pull vs push") | Aspect | Pull (`sourcePubSubPull`) | Push (`sourcePubSubPush`) | | ----------------- | ------------------------------------------------------ | ----------------------------------------------------------- | | Process model | Long-running streaming subscriber | Stateless HTTP request handler | | Hosting | Cloud Run with `min-instances >= 1`, GKE, GCE, on-prem | Cloud Run scale-to-zero, Cloud Functions, Lambda | | Idle cost | Pay for always-on compute | Zero when idle | | Throughput | Highest (gRPC streaming) | Limited by HTTP overhead | | Latency | Lowest (warm stream, single-digit ms) | Higher (HTTP per message, cold-start adds 100-500ms) | | Auth direction | Subscriber holds credentials, connects outbound | Pub/Sub holds credentials, posts inbound (OIDC verify) | | Subscription type | `subscription.type = "pull"` | `subscription.type = "push"` with `pushConfig.pushEndpoint` | | Best fit | Long-running collector services, high-volume streams | Serverless walkerOS endpoints, hybrid setups | Both modes share the same `@walkeros/server-source-gcp` package, the same auth resolution, the same decoders, and the same setup command. Pick the one that matches your hosting model. ## Pull subscriber[​](#pull-subscriber "Direct link to Pull subscriber") ``` import { sourcePubSubPull } from '@walkeros/server-source-gcp'; import { startFlow } from '@walkeros/collector'; await startFlow({ sources: { pubsub: { code: sourcePubSubPull, config: { settings: { projectId: 'YOUR_PROJECT_ID', subscription: 'YOUR_SUBSCRIPTION_NAME', }, }, }, }, destinations: { // your destinations }, }); ``` The pull source is event-driven: `init()` opens the streaming subscription and forwards each delivered message to the collector via `env.push`. The source's `push` method is a deliberate no-op stub. There is no external invocation; the subscriber pushes events autonomously. `destroy()` closes the subscriber gracefully, honoring `shutdownTimeoutMs` (default 30000). ### Pull settings[​](#pull-settings "Direct link to Pull settings") | Field | Type | Default | Description | | ------------------- | ------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------- | | `projectId` | `string` | (required) | GCP project id. | | `subscription` | `string` | (required) | Subscription short name. | | `topic` | `string` | (optional) | Topic short name. Used by setup auto-create when `setup.createTopic` is on. | | `credentials` | `string \| ServiceAccountCredentials` | (ADC fallback) | Service account JSON (parsed object or JSON string). Set on `config.credentials` (sibling of `settings`). | | `apiEndpoint` | `string` | (SDK default) | Override for the emulator (e.g. `localhost:8085`). | | `decoder` | `'json' \| 'text' \| 'raw'` | `'json'` | How the subscriber decodes message bodies before forwarding. | | `flowControl` | `{ maxMessages?, maxBytes? }` | `{ maxMessages: 100, maxBytes: 10 MB }` | Subscriber pull rate (in-flight cap). | | `ackDeadline` | `number` | `60` | Per-message ack window in seconds. | | `shutdownTimeoutMs` | `number` | `30000` | Graceful drain budget on `destroy()`. | | `onPushError` | `'nack' \| 'ack'` | `'nack'` | Behavior when forwarding to the collector throws. | ## Push webhook[​](#push-webhook "Direct link to Push webhook") ``` import express from 'express'; import { sourcePubSubPush } from '@walkeros/server-source-gcp'; import { startFlow } from '@walkeros/collector'; const { sources } = await startFlow({ sources: { pubsub: { code: sourcePubSubPush, config: { settings: { decoder: 'json' } }, }, }, destinations: { // your destinations }, }); const app = express(); app.use(express.json()); app.post('/pubsub-push', sources.pubsub.push); app.listen(8080); ``` Pub/Sub posts each message envelope to the configured endpoint as a JSON body matching the [Pub/Sub push spec](https://cloud.google.com/pubsub/docs/push). The handler decodes the envelope, base64-decodes the `data` field, runs the configured decoder, and forwards to the collector. Status codes: * `200`: forwarded successfully. * `400`: malformed envelope (missing `message.data`, invalid base64). * `401`: OIDC verification failed (when enabled). * `500`: collector push failed; Pub/Sub will retry per the subscription's retry policy. ### Push settings[​](#push-settings "Direct link to Push settings") In addition to the shared base (`projectId`, `apiEndpoint`, `decoder` in `settings`, plus `config.credentials`): | Field | Type | Default | Description | | ------------ | --------- | ------- | ------------------------------------------------------------------------- | | `verifyOidc` | `boolean` | `false` | When `true`, verifies the bearer token Pub/Sub attaches to push requests. | | `audience` | `string` | - | Required when `verifyOidc` is true. Must match your endpoint URL exactly. | ## Authentication[​](#authentication "Direct link to Authentication") Three modes for the pull subscriber, evaluated in order: 1. **Application Default Credentials (ADC).** Default. Works on GCP-native runtimes (Cloud Run, Cloud Functions, GKE) and locally with `gcloud auth application-default login`. 2. **Service account JSON.** Pass `config.credentials`, either parsed or as JSON string (the source JSON-parses strings). Combine with `$env.NAME` to inject from an environment variable. (The package-specific `settings.credentials` still works but is deprecated.) 3. **Pre-configured client.** Pass an existing `PubSub` SDK instance as `settings.client`. For the push handler, auth is reversed: Pub/Sub holds the identity and signs requests with an OIDC token. The handler verifies the token against GCP public keys when `verifyOidc: true` and `audience` is set. Off by default because misconfigured OIDC silently rejects all messages, so opt in once your endpoint is publicly reachable. ## Setup[​](#setup "Direct link to Setup") Provision the subscription once per environment: ``` walkeros setup source.pubsub ``` Idempotent. Defaults: 60-second ack deadline, project-default retention, no filter, no dead-letter policy. Drift on `ackDeadlineSeconds`, `messageRetentionDuration`, `deadLetterPolicy`, or `filter` emits `WARN setup.drift {...}` and never auto-mutates. `config.setup`: * `false` (default): no provisioning. * `true`: provisions the subscription against an existing topic with safe defaults. * `{ ackDeadlineSeconds, messageRetentionDuration, filter, deadLetterPolicy, retryPolicy, ... }`: object form for explicit overrides. Optional one-shot helpers: * `setup.createTopic: true`: auto-create the topic if it does not exist (requires `settings.topic`). Useful when source and topic owners share a deployment. * `setup.deadLetterPolicy.createDeadLetterTopic: true`: auto-create the dead-letter topic referenced in `deadLetterPolicy.deadLetterTopic`. The provisioning identity needs `pubsub.subscriptions.create` (and `pubsub.topics.create` if auto-creating). The runtime pull identity needs `roles/pubsub.subscriber` on the subscription. Topic provisioning is owned by the [Pub/Sub destination](/docs/destinations/server/pubsub.md) when the topic is the producer's responsibility; the source can also auto-create as a convenience for self-contained setups. ## Decoders[​](#decoders "Direct link to Decoders") | Decoder | Behavior | | ---------------- | ----------------------------------------- | | `json` (default) | `JSON.parse(data.toString('utf8'))` | | `text` | `data.toString('utf8')` (forwarded as-is) | | `raw` | The raw `Buffer` is forwarded unchanged | A decoder that throws on malformed payloads triggers `onPushError` (pull mode): `nack` redelivers, `ack` drops with a debug log. ## Emulator[​](#emulator "Direct link to Emulator") Both pull and push honor `PUBSUB_EMULATOR_HOST`. For explicit configuration, set `settings.apiEndpoint` (e.g. `localhost:8085`). ``` gcloud beta emulators pubsub start --host-port=localhost:8085 export PUBSUB_EMULATOR_HOST=localhost:8085 ``` ## Lifecycle[​](#lifecycle "Direct link to Lifecycle") The pull source is the canonical long-running listener pattern in walkerOS. Existing server sources (express, lambda, cloudfunction, fetch) are request-handlers; pull is fundamentally different: 1. `init()` starts the streaming subscriber. It returns the `Source.Instance` and the subscriber runs in the background. 2. The instance's `push` method is a no-op stub. The framework never calls it; the subscriber forwards events autonomously. 3. `destroy()` closes the subscriber. On shutdown the source stops accepting new messages, drains in-flight ack/nacks, and force-closes after `shutdownTimeoutMs`. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") * **`NOT_FOUND` on the subscription**: the subscription does not exist. Run `walkeros setup source.` once or create it via `gcloud pubsub subscriptions create`. * **`PERMISSION_DENIED` / `UNAUTHENTICATED`** on the pull subscriber: the runtime service account lacks `roles/pubsub.subscriber` on the subscription. * **Push handler returns 401 for every message**: OIDC misconfiguration. Verify `settings.audience` matches your endpoint URL exactly, including scheme and path. * **Messages decoded incorrectly**: switch the decoder. JSON is the default; use `text` for plain-text payloads, `raw` for binary. ## Next steps[​](#next-steps "Direct link to Next steps") * [Pub/Sub destination](/docs/destinations/server/pubsub.md) for publishing events to a Pub/Sub topic * [GCP source overview](/docs/sources/server/gcp.md) for the Cloud Functions HTTP handler --- # AWS SQS [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/aws) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/sources/aws)[ ](https://www.npmjs.com/package/@walkeros/server-source-aws) [Package](https://www.npmjs.com/package/@walkeros/server-source-aws) Subscribes to an [Amazon SQS](https://aws.amazon.com/sqs/) queue and forwards each delivered message to the walkerOS collector. Long-running streaming subscriber backed by SQS long-poll. Decoders for JSON, text, and raw payloads. Idempotent queue provisioning with optional dead-letter wiring and SNS-topic subscription. EU-central-1 default. The source ships inside `@walkeros/server-source-aws` alongside the Lambda handler; install the package once and import `sourceSqs` for SQS. Where this fits SQS is a **server source** in the walkerOS flow: Pulls messages from an SQS queue at consumer pace via long-poll. Standard pattern: SNS topic fans out to one or more SQS queues; walkerOS pulls each queue independently. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-source-aws ``` ``` import { sourceSqs } from '@walkeros/server-source-aws'; import { startFlow } from '@walkeros/collector'; await startFlow({ sources: { sqs: { code: sourceSqs, config: { settings: { queueName: 'YOUR_QUEUE_NAME', region: 'eu-central-1', }, }, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---- | | `queueName*` | `string` | SQS queue short name (like walkeros-events). Required for both setup and runtime poll. | | | `region` | `string` | AWS region (like eu-central-1). Default: eu-central-1. | | | `queueUrl` | `string` | Optional pre-resolved queue URL. When set, init skips the GetQueueUrl lookup. | | | `client` | `any` | Pre-configured AWS SQSClient instance. Bypasses construction when supplied. | | | `config` | `any` | AWS SDK SQSClientConfig (credentials, endpoint overrides, retries). | | | `decoder` | `'json' \| 'text' \| 'raw'` | Decoder for the message body. json (default) parses JSON, text forwards UTF-8, raw forwards a Buffer. | | | `maxMessages` | `integer` | SQS receive batch size. Cap 10. Default: 10. | | | `waitTimeSeconds` | `integer` | Long-poll duration in seconds. Cap 20. Default: 20. | | | `visibilityTimeout` | `integer` | Per-receive visibility timeout override. Default: queue-configured value. | | | `shutdownTimeoutMs` | `integer` | Graceful shutdown timeout in milliseconds. Default: 30000. After this window, destroy() force-closes. | | | `onPushError` | `'nack' \| 'ack'` | Behavior when forwarding to the collector throws. nack (default) skips DeleteMessage so the message redelivers; ack drops it. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Lifecycle[​](#lifecycle "Direct link to Lifecycle") The SQS source follows the long-running listener pattern, same shape as the [Pub/Sub pull source](/docs/sources/server/pubsub.md): 1. `init()` resolves the queue URL via `GetQueueUrl`, captures the queue ARN, and starts the long-poll loop as a background task. 2. The instance's `push()` method is a deliberate no-op stub for production. The framework never calls it; the loop forwards events autonomously. 3. `destroy()` stops the loop, drains in-flight ack/nacks, and force-closes after `shutdownTimeoutMs` (default 30000). ## Authentication[​](#authentication "Direct link to Authentication") The AWS SDK v3 picks up credentials in this order: 1. **Environment variables**: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`. Easiest for local development. 2. **Shared credentials file** (`~/.aws/credentials`) and config file (`~/.aws/config`). Standard CLI / SDK convention. 3. **IAM role attached to the runtime** (EC2 instance profile, ECS task role, EKS service account, Lambda execution role). Recommended for production. 4. **Pre-configured `SQSClient`**: pass an instance via `settings.client` for full control. The runtime identity needs `sqs:ReceiveMessage`, `sqs:DeleteMessage`, and `sqs:GetQueueUrl` on the target queue (and `sqs:GetQueueAttributes` for the ARN capture at init). ## Setup[​](#setup "Direct link to Setup") Provision the queue once per environment: ``` walkeros setup source.sqs ``` Idempotent and authoritative for declared fields. Re-running on an existing queue is a safe no-op when declared state matches; if it differs, declared state wins. Operator-managed extras (additional tags, additional policy SIDs) are left untouched. `config.setup`: * `false` (default): no provisioning. Operator must run setup explicitly. * `true`: provisions with safe defaults (`eu-central-1`, standard queue, default visibility timeout). * Object form: explicit overrides. See the schema in the Configuration block above. Optional one-shot helpers: * `setup.deadLetter`: wire a dead-letter queue. Auto-creates the DLQ if it does not exist. * `setup.snsTopicArn`: subscribe the queue to an SNS topic and apply the queue policy granting `sns:SendMessage` from that topic. The provisioning identity needs `sqs:CreateQueue`, `sqs:GetQueueAttributes`, `sqs:GetQueueUrl`, `sqs:TagQueue`, `sqs:SetQueueAttributes`, and `sns:Subscribe` (only when subscribing to an SNS topic). ## Decoders[​](#decoders "Direct link to Decoders") | Decoder | Behavior | | ---------------- | ----------------------------------------------------- | | `json` (default) | `JSON.parse(body)`. Throws on malformed JSON. | | `text` | The raw `body` string is forwarded as-is. | | `raw` | The body is wrapped in `{ data: { payload: body } }`. | A decoder that throws on malformed payloads triggers `onPushError`: `nack` lets visibility expire (SQS redelivers; eventually the DLQ if wired), `ack` deletes the message with a debug log. ## Backpressure[​](#backpressure "Direct link to Backpressure") SQS long-poll is naturally backpressured: each `ReceiveMessage` returns up to `maxMessages` (cap 10) and waits up to `waitTimeSeconds` (cap 20) for messages to arrive. The loop processes the batch sequentially and only issues the next `ReceiveMessage` after the previous batch is forwarded and acked. If the collector slows down, polling slows down with it. No tuning required for typical loads. For higher throughput run multiple worker instances against the same queue; SQS distributes messages across consumers automatically. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") * **`QueueDoesNotExist` at init**: the queue name is correct but the queue is not provisioned. Run `walkeros setup source.` once. * **`AccessDenied` on `ReceiveMessage`**: the runtime IAM identity lacks `sqs:ReceiveMessage` on the queue. Grant the queue ARN explicitly. * **Messages stuck in flight**: a handler threw before deleting and visibility timeout has not yet expired. With `onPushError: 'nack'` (default) this is intentional: SQS redelivers after the visibility window. Tune `visibilityTimeout` to your handler's expected processing time. * **Decoder throws on every message**: the queue carries a different payload shape than declared. Switch the decoder (`text`, `raw`) or wire a transformer that pre-processes the payload. ## Next steps[​](#next-steps "Direct link to Next steps") * [AWS SNS destination](/docs/destinations/server/sns.md) for the producer side. Standard pattern: SNS topic fans out to this SQS queue. * [GCP Pub/Sub source](/docs/sources/server/pubsub.md) for the GCP analog (streaming pull subscriber). * [AWS Lambda source](/docs/sources/server/aws.md) for invocation-triggered ingestion. --- [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/browser)[ ](https://www.npmjs.com/package/@walkeros/web-source-browser) [Package](https://www.npmjs.com/package/@walkeros/web-source-browser) # Browser source Captures user interactions directly from the DOM using `data-elb-*` attributes. Extracts tracking data from markup, tracks pageviews automatically or manually, exposes an enhanced `elb` function with `walker init` support, and provides a trigger system covering click, submit, load, hover, scroll, and visible. Where this fits The browser source is a **source** in the walkerOS flow: It reads \`data-elb-\*\` attributes from your HTML and generates events. The Collector routes these to your configured destinations. ## Installation[​](#installation "Direct link to Installation") * Integrated * Bundled Install the packages: ``` npm install @walkeros/collector @walkeros/web-source-browser @walkeros/web-source-session ``` Configure in your code: ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { sourceSession } from '@walkeros/web-source-session'; const { collector, elb } = await startFlow({ sources: { browser: sourceBrowser, session: sourceSession, // Add for session tracking }, }); ``` Add to your `flow.json` sources: ``` "sources": { "browser": { "package": "@walkeros/web-source-browser" }, "session": { "package": "@walkeros/web-source-session", "config": { "settings": { "storage": true } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---- | | `prefix` | `string` | Prefix for data attributes (default: data-elb) | | | `scope` | `string` | DOM scope for event tracking (default: document) | | | `pageview` | `boolean` | Enable automatic pageview tracking | | | `capture` | `boolean` | Read click/submit triggers in the capture phase so tagged elements are resolved at click time before app handlers run (set false for the previous bubble-phase behavior) | | | `elb` | `string` | Name for global elb function | | | `name` | `string` | Custom name for source instance | | | `elbLayer` | `boolean \| string \| any` | Enable elbLayer for async command queuing (boolean, string, or Elb.Layer) | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Click event A button click with walker data attributes is captured as an entity action event with the mapped label data. Event ``` "" ``` Out ``` elb({ "name": "cta click", "entity": "cta", "action": "click", "data": { "label": "Sign Up" }, "context": {}, "globals": {}, "nested": [], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "click" }) ``` ### Context and globals Walker context and globals attributes attach ambient metadata to events emitted by the browser source. Event ``` "
Try Now
" ``` Out ``` elb({ "name": "cta signup", "entity": "cta", "action": "signup", "data": { "label": "Try Now" }, "context": { "test": [ "engagement_flow", 0 ] }, "globals": { "language": "en", "plan": "premium" }, "nested": [], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "click" }) ``` ### Data attribute types Walker data attributes parse scalar, boolean, and array values into typed fields on the emitted event. Event ``` "
" ``` Out ``` elb({ "name": "product select", "entity": "product", "action": "select", "data": { "price": 99.99, "available": true, "colors": [ "red", "blue" ] }, "context": {}, "globals": {}, "nested": [], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "click" }) ``` ### Impression event An intersection observer impression on a div with walker attributes emits a banner view event. Event ``` "
" ``` Out ``` elb({ "name": "banner view", "entity": "banner", "action": "view", "data": { "type": "promotional", "position": "sidebar" }, "context": {}, "globals": {}, "nested": [], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "impression" }) ``` ### Nested entities A page load produces a page view plus a product view whose nested size entity is captured on the product event. Event ``` "
" ``` Out ``` elb({ "name": "page view", "data": { "domain": "example.com", "title": "", "referrer": "", "id": "/" }, "context": {}, "globals": {}, "trigger": "load", "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" } }); elb({ "name": "product view", "entity": "product", "action": "view", "data": { "id": "SKU-42", "name": "Sneakers" }, "context": {}, "globals": {}, "nested": [ { "entity": "size", "data": { "selected": "large", "inStock": true }, "context": {}, "nested": [] } ], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "load" }) ``` ### Page view A page load trigger captures the current URL, title, and referrer as a walker page view event. Event ``` "" ``` Out ``` elb({ "name": "page view", "data": { "domain": "example.com", "title": "Documentation", "referrer": "https://example.com/", "id": "/docs" }, "context": {}, "globals": {}, "trigger": "load", "source": { "type": "browser", "platform": "web", "url": "https://example.com/docs", "referrer": "https://example.com/" } }) ``` ### Submit event A form submit with walker data attributes produces a signup complete event carrying the selected plan. Event ``` "
" ``` Out ``` elb({ "name": "signup complete", "entity": "signup", "action": "complete", "data": { "plan": "premium" }, "context": {}, "globals": {}, "nested": [], "source": { "type": "browser", "platform": "web", "url": "https://example.com/", "referrer": "" }, "trigger": "submit" }) ``` ## Enhanced elb[​](#enhanced-elb "Direct link to Enhanced elb") The browser source returns an enhanced `elb` function with additional features: ``` // Collector elb: Basic event tracking collector.push('product view', { id: 'P123' }); // Browser source elb: Enhanced with DOM commands elb('walker init', document.querySelector('#app')); elb('product view', { id: 'P123' }); // Also tracks events ``` **Browser source elb adds:** * **DOM commands**: `walker init` for asynchronous DOM element loading * **Flexible arguments**: support for multiple argument patterns * **elbLayer integration**: automatic processing of queued commands * **Element parameters**: support for element parameters in commands See [Commands](/docs/sources/web/browser/commands.md) for complete API documentation. ## Next steps[​](#next-steps "Direct link to Next steps") * [Session source](/docs/sources/web/session/.md) - session and device tracking * [Tagging guide](/docs/sources/web/browser/tagging/html-attributes.md) - learn about data attributes * [Commands reference](/docs/sources/web/browser/commands.md) - full API documentation * [Tagger tool](/docs/sources/web/browser/tagger.md) - visual tagging tool --- # Browser source commands The browser source provides an enhanced `elb` function that supports browser-specific features like DOM interaction, elbLayer communication, and automatic initialization. These commands are processed by the browser source translation layer before being passed to the collector. ## elb[​](#elb "Direct link to elb") The browser source provides an enhanced `elb` function that supports flexible argument patterns and browser-specific features. ``` // Import from browser source import { elb } from '@walkeros/web-source-browser'; window.elb = elb; // Or define the elb function manually in the browser function elb() { (window.elbLayer = window.elbLayer || []).push(arguments); } ``` Usage options: ``` elb("entity action", data, ...); elb({event: "entity action", data: { foo: "bar"}}); ``` The browser source owns the `window.elb` name: during initialization it assigns the function to `window[settings.elb]` (default `elb`). Calls return a promise that resolves with the processing result. ### elbLayer processing[​](#elblayer-processing "Direct link to elbLayer processing") `window.elbLayer` is an append-only queue: entries are recorded and never removed, so inspecting the array always shows the full input history. Both `window.elb(...)` and `window.elbLayer.push(...)` are processed through the same translation layer. Processing order: * `walker` commands apply as soon as the source is initialized, so a queued `walker consent` takes effect before any queued events. * Events are processed once the source has started (after the first `walker run`), in the order they were pushed. * From then on, all entries are processed strictly in push order. ## config[​](#config "Direct link to config") Configure the browser source during initialization through `startFlow`. These settings control browser-specific behavior: ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; const { collector } = await startFlow({ sources: { browser: { code: sourceBrowser, config: { settings: { elb: 'elb', // Name to assign the elb function to the window elbLayer: true, // Enable elbLayer for async command queuing pageview: true, // Trigger a page view event by default prefix: 'data-elb', // Attributes prefix used by the walker for DOM scanning }, }, }, }, }); ``` note Browser source configuration must be done during initialization. Settings like `prefix` and `elbLayer` cannot be changed after the source is created. Session tracking is provided by the separate `@walkeros/web-source-session` source. ## run[​](#run "Direct link to run") A `run` initializes the browser source and triggers automatic DOM scanning and event setup. It will: * Initialize DOM event listeners * Scan for `data-elb` attributes * Trigger a `page view` event by default * Process the `elbLayer` stack ``` elb('walker run'); ``` A run accepts a partial state parameter: ``` elb('walker run', { group: 'group1d' }); ``` ### Virtual pageviews in SPAs[​](#virtual-pageviews-in-spas "Direct link to Virtual pageviews in SPAs") Use `walker run` to trigger virtual pageviews when navigating between routes in Single Page Applications. Call it on each route change to fire a new `page view` event and re-scan the DOM: ``` // React Router example useEffect(() => { window.elb('walker run'); }, [location]); // Next.js App Router example useEffect(() => { window.elb('walker run'); }, [pathname]); ``` tip For SPAs, use `walker run` for route changes (triggers pageview), and `walker init` for dynamically loaded content within the same page (no pageview). ## init[​](#init "Direct link to init") Re-initializes event listeners on one or multiple target elements without triggering a page view. Useful for dynamically loaded content like product lists, infinite scroll, or wizard steps. ``` elb('walker init', element); // Single element elb('walker init', [element1, element2]); // Multiple elements ``` This command is essential for Single Page Applications (SPAs) where content is added dynamically after the initial page load. Re-initializing a scope clears and rebuilds that scope's `pulse`, `wait`, `hover`, `scroll`, `visible`, and `impression` triggers. The `load` trigger fires (again) on each call. ``` // After loading new products via AJAX const productList = document.getElementById('product-list'); fetchProducts().then(() => { elb('walker init', productList); }); ``` tip Use `walker init` after adding new DOM elements with `data-elb` attributes. Unlike `walker run`, this does not trigger a page view event - it only enables tracking on the specified elements. For content that a SPA injects repeatedly into the same container, the [`data-elbobserve`](/docs/sources/web/browser/tagging/html-attributes.md#auto-init-with-data-elbobserve) attribute is a declarative alternative: mark the container once and injected tagged content is registered automatically, without a `walker init` call after each injection. ## run vs init comparison[​](#run-vs-init-comparison "Direct link to run vs init comparison") | Command | Triggers pageview | Re-scans DOM | Use case | | ------------- | ----------------- | -------------------- | ----------------------------------------------------------- | | `walker run` | Yes | Full page | SPA route changes, virtual pageviews | | `walker init` | No | Target elements only | Dynamically loaded content (product lists, infinite scroll) | ## Integration with collector[​](#integration-with-collector "Direct link to Integration with collector") Browser source commands work in conjunction with [collector commands](/docs/collector/commands.md). The browser source handles DOM-specific functionality while the collector manages destinations, consent, and user data. Common workflow: 1. Configure browser source in `startFlow` 2. Browser source automatically initializes (or use `walker run` for manual control) 3. Use `walker init` for dynamic content --- # Tagger [Web](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/browser) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/browser)[ ](https://www.npmjs.com/package/@walkeros/web-source-browser) [Package](https://www.npmjs.com/package/@walkeros/web-source-browser) The tagger is a utility for generating HTML data attributes that the walkerOS browser source uses for event tracking. It provides a fluent interface to create properly formatted and escaped data attributes for your HTML elements. ## Why use the tagger?[​](#why-use-the-tagger "Direct link to Why use the tagger?") The tagger solves several challenges when working with walkerOS data attributes: * **Consistent formatting** - Ensures data attributes follow walkerOS conventions * **Automatic escaping** - Handles special characters in values (semicolons, colons, quotes, backslashes) * **Type safety** - Provides TypeScript support for better development experience * **Fluent API** - Chainable methods for building complex attribute sets * **Maintainability** - Centralized logic for attribute generation ## When to use the tagger[​](#when-to-use-the-tagger "Direct link to When to use the tagger") Use the tagger when you need to: * Generate data attributes programmatically in JavaScript/TypeScript * Handle dynamic values that may contain special characters * Build complex attribute sets with multiple properties * Ensure consistent tagging across your application * Integrate walkerOS tracking into component-based frameworks (React, Vue, etc.) ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-source-browser ``` ## Initialization[​](#initialization "Direct link to Initialization") The tagger is initialized using the `createTagger` factory function: ``` import { createTagger } from '@walkeros/web-source-browser'; // Create with default configuration const tagger = createTagger(); // Create with custom configuration const customTagger = createTagger({ prefix: 'data-elb', }); ``` ## Configuration[​](#configuration "Direct link to Configuration") | Property | Type | Description | More | | -------- | -------- | ------------------------------------------- | ---- | | `prefix` | `string` | Custom prefix for generated data attributes | | ## Usage examples[​](#usage-examples "Direct link to Usage examples") ### Basic data tagging (without entity)[​](#basic-data-tagging-without-entity "Direct link to Basic data tagging (without entity)") ``` const tagger = createTagger(); // Using tagger with a scope parameter sets naming for data attributes only const attributes = tagger('product') .data('id', '123') .data('name', 'Widget') .get(); // Result: // { // 'data-elb-product': 'id:123;name:Widget' // } // Note: No 'data-elb' entity attribute is created ``` ### Entity tagging[​](#entity-tagging "Direct link to Entity tagging") ``` // To create an entity attribute, use the .entity() method const attributes = tagger() .entity('product') .data('id', '123') .data('name', 'Widget') .get(); // Result: // { // 'data-elb': 'product', // 'data-elb-product': 'id:123;name:Widget' // } ``` ### Action mapping[​](#action-mapping "Direct link to Action mapping") ``` const attributes = tagger() .action('load', 'view') .action('click', 'select') .get(); // Result: // { // 'data-elbaction': 'load:view;click:select' // } ``` ### Context and global properties[​](#context-and-global-properties "Direct link to Context and global properties") ``` const attributes = tagger('product') .data('id', 123) .context('test', 'engagement') .globals('lang', 'en') .get(); // Result: // { // 'data-elb': 'product', // 'data-elb-product': 'id:123', // 'data-elbcontext': 'test:engagement', // 'data-elbglobals': 'lang:en' // } ``` ### Scoped generic properties[​](#scoped-generic-properties "Direct link to Scoped generic properties") `scoped()` emits the `data-elb_` attribute: a generic property that only reaches triggers nested below the element, unlike `data-elb-` which every trigger in the entity receives. It uses the same value syntax and escaping as the other methods. ``` const attributes = tagger() .entity('product') .data('name', 'A') .scoped('size', 'L') .get(); // Result: // { // 'data-elb': 'product', // 'data-elb-product': 'name:A', // 'data-elb_': 'size:L' // } ``` ### Multiple entity scopes[​](#multiple-entity-scopes "Direct link to Multiple entity scopes") ``` // Starting with a naming scope const attributes = tagger('product') .data('id', 123) .entity('user') // Changes both entity attribute and naming scope .data('name', 'John') .get(); // Result: // { // 'data-elb': 'user', // 'data-elb-product': 'id:123', // 'data-elb-user': 'name:John' // } ``` ### Order matters[​](#order-matters "Direct link to Order matters") ``` // Data before entity uses original scope const attributes = tagger('product') .data('id', 123) .data('price', 99.99) .entity('cart') // Changes scope for future data calls .data('quantity', 2) .get(); // Result: // { // 'data-elb': 'cart', // 'data-elb-product': 'id:123;price:99.99', // 'data-elb-cart': 'quantity:2' // } ``` ### Value escaping[​](#value-escaping "Direct link to Value escaping") ``` const attributes = tagger() .data('description', 'Product with: special; chars & "quotes"') .get(); // Result: // { // 'data-elb-': 'description:Product with\\: special\\; chars & \\"quotes\\"' // } ``` ## Available methods (API reference)[​](#available-methods-api-reference "Direct link to Available methods (API reference)") ##### `tagger(scope?: string)`[​](#taggerscope-string "Direct link to taggerscope-string") Creates a new tagger instance. The optional scope parameter sets the naming scope for data attributes without creating an entity attribute. ``` // Without scope - generic data attributes tagger().data('key', 'value'); // Creates: data-elb-="key:value" // With scope - scoped data attributes (no entity attribute) tagger('product').data('id', '123'); // Creates: data-elb-product="id:123" ``` ##### `entity(name: string)`[​](#entityname-string "Direct link to entityname-string") Sets the entity attribute and updates the naming scope for subsequent data calls. ``` tagger().entity('product').data('id', '123'); // Creates: data-elb="product" data-elb-product="id:123" // Entity changes the naming scope tagger('foo').entity('bar').data('a', 1); // Creates: data-elb="bar" data-elb-bar="a:1" ``` ##### `data(key: string, value: Property)` | `data(object: Properties)`[​](#datakey-string-value-property--dataobject-properties "Direct link to datakey-string-value-property--dataobject-properties") Adds data properties using the current naming scope. ``` // Single property tagger('product').data('id', 123); // Creates: data-elb-product="id:123" // Multiple properties tagger('product').data({ id: 123, name: 'Widget', price: 99.99 }); // Creates: data-elb-product="id:123;name:Widget;price:99.99" ``` ##### `action(trigger: string, action?: string)` | `action(object: Record)`[​](#actiontrigger-string-action-string--actionobject-recordstring-string "Direct link to actiontrigger-string-action-string--actionobject-recordstring-string") Adds action mappings for event triggers. Creates a `data-elbaction` attribute. ``` // Single action tagger().action('load', 'view'); // Combined trigger:action tagger().action('load:view'); // Multiple actions tagger().action({ load: 'view', click: 'select', impression: 'view' }); ``` ##### `actions(trigger: string, action?: string)` | `actions(object: Record)`[​](#actionstrigger-string-action-string--actionsobject-recordstring-string "Direct link to actionstrigger-string-action-string--actionsobject-recordstring-string") Adds action mappings for event triggers. Creates a `data-elbactions` attribute. ``` // Single action tagger().actions('load', 'view'); // Combined trigger:action tagger().actions('load:view'); // Multiple actions tagger().actions({ load: 'view', click: 'select', visible: 'visible' }); // Can be combined with action() method tagger().action('click', 'select').actions('load', 'view'); ``` ##### `context(key: string, value: Property)` | `context(object: Properties)`[​](#contextkey-string-value-property--contextobject-properties "Direct link to contextkey-string-value-property--contextobject-properties") Adds context properties that apply to all events. ``` // Single context tagger().context('test', 'engagement'); // Multiple contexts tagger().context({ test: 'engagement', position: 'header', type: 'promo' }); ``` ##### `globals(key: string, value: Property)` | `globals(object: Properties)`[​](#globalskey-string-value-property--globalsobject-properties "Direct link to globalskey-string-value-property--globalsobject-properties") Adds global properties that persist across page views. ``` // Single global tagger().globals('lang', 'en'); // Multiple globals tagger().globals({ lang: 'en', plan: 'paid', version: '1.0' }); ``` ##### `scoped(key: string, value: Property)` | `scoped(object: Properties)`[​](#scopedkey-string-value-property--scopedobject-properties "Direct link to scopedkey-string-value-property--scopedobject-properties") Adds path-scoped generic properties. Creates a `data-elb_` attribute that only applies to triggers nested below the element. ``` // Single scoped property tagger().scoped('size', 'L'); // Creates: data-elb_="size:L" // Multiple scoped properties tagger().scoped({ size: 'L', color: 'red' }); // Creates: data-elb_="size:L;color:red" ``` ##### `link(id: string, type: string)` | `link(object: Record)`[​](#linkid-string-type-string--linkobject-recordstring-string "Direct link to linkid-string-type-string--linkobject-recordstring-string") Adds link relationships between elements. ``` // Single link tagger().link('details', 'parent'); // Multiple links tagger().link({ details: 'parent', modal: 'child', sidebar: 'child' }); ``` ##### `get()`[​](#get "Direct link to get") Generates the final HTML attributes object. ``` // With naming scope only const attributes = tagger('product').data('id', '123').get(); // Returns: { 'data-elb-product': 'id:123' } // With entity attribute const attributes = tagger().entity('product').data('id', '123').get(); // Returns: { 'data-elb': 'product', 'data-elb-product': 'id:123' } ``` All methods return the tagger instance for method chaining, except `get()` which returns the final attributes object. ## Common use cases[​](#common-use-cases "Direct link to Common use cases") ### Product listing page[​](#product-listing-page "Direct link to Product listing page") ``` // For product cards with nearest entity tracking only function ProductCard({ product }) { return (
{product.name}
); } // For product cards that also need page context tracking function ProductCardWithContext({ product }) { return (
{product.name}
); } ``` ### Shopping cart[​](#shopping-cart "Direct link to Shopping cart") ``` // For cart items that need both entity and data attributes function CartItem({ item }) { return (
{item.name}
); } ``` ### Dynamic component tracking[​](#dynamic-component-tracking "Direct link to Dynamic component tracking") ``` // Reusable tracking function function trackComponent(type, data, actions = {}) { return tagger(type) .data(data) .action(actions) .get(); } // Usage ``` --- # HTML Attributes Tag your web components using `data-elb` attributes to enable structured event tracking without custom JavaScript. By adding a few simple attributes to your markup, you can track user behavior such as clicks, views, form submissions, and more. This technique is especially helpful for **component libraries** and **design systems**, where it is important to **scale** event tagging in a maintainable, reusable way. You can bake tracking into components once, then use them everywhere, without extra code or tracking configuration. You'll learn how to: * Define entities, actions, and triggers directly in your HTML * Add contextual and global properties * Handle dynamic values, arrays, and type casting * Link elements across DOM boundaries * Structure nested data automatically If you're looking for a **lightweight, declarative** way to implement analytics and event tracking, `data-elb` tagging is your foundation. ## Concept[​](#concept "Direct link to Concept") Tag a page... ```
data-elbactions="TRIGGER:ACTION" data-elbcontext="KEY:VALUE" data-elbglobals="KEY:VALUE" />

Setting up tracking easily

Analytics

``` ... to get a structured event as a result: ``` { name: 'promotion view', // Name as a combination of entity and action data: { // Arbitrary properties related to the entity name: 'Setting up tracking easily', category: 'analytics', }, context: { // Provides additional information about the state during the event test: ['engagement', 0] // Key, [value, order] }, globals: { // General properties that apply to every event language: 'en' }, custom: {}, // Additional space for individual setups user: { // Contains user identifiers for different identification levels // Require consent and set manually for sessions building and cross-device id: 'us3r1d', device: 'c00k131d', session: 's3ss10n1d', }, nested: [], // List of nested entities consent: { functional: true }, // Status of the granted consent state(s) id: '0123456789abcdef', // W3C Trace Context span_id (16 hex chars) trigger: 'visible', // Name of the trigger that fired entity: 'promotion', // The entity name involved in the event action: 'view', // The specific action performed on the entity timestamp: 1647261462000, // Time when the event fired timing: 3.14, // Duration how long it took to trigger this event source: { // Details about the origin of the event type: 'browser', // Source kind (e.g. browser, dataLayer) platform: 'web', // Runtime platform (web or server) schema: '4', // walkerOS event schema version url: 'https://github.com/elbwalker/walkerOS', // Page URL referrer: 'https://www.walkeros.io/' // Referrer URL } } ``` Updating from v3 The top-level `group`, `count`, and `version` fields are gone. The `id` is now a [W3C Trace Context](https://www.w3.org/TR/trace-context/) `span_id` (16 lowercase hex chars). Inside `source`, `type` is now the source kind (`browser`, `dataLayer`, ...), the runtime is captured by the new `platform` field, the previous `id` is now `url`, and `previous_id` is now `referrer`. note You are entirely **free** to define naming conventions. ## Entity and action[​](#entity-and-action "Direct link to Entity and action") You define the entity **scope** by setting the `data-elb` attribute with the name of an entity to an element, e.g. `data-elb="promotion"`. The default entity is `page` when no `data-elb` is set. An **action** can be added by setting one of the following attributes on the **same level** or **child elements** in combination with a **matching trigger**: * **`data-elbaction`** - applies action to the **nearest entity only** * **`data-elbactions`** - applies action to **all entities** in the DOM hierarchy Both attributes use the same syntax, e.g., `data-elbaction="visible:view"` or `data-elbactions="click:select"` to fire events when triggered. Migration Note The behavior of `data-elbaction` changed in @walkeros to apply to nearest entity only. For the previous @elbwalker "all entities" behavior, use `data-elbactions`. See the [Migration Guide](/docs/guides/migration.md#data-elbaction-vs-data-elbactions) for details. To define the entities' **properties**, set the **composited attribute** `data-elb-ENTITY` with the key and value, e.g. `data-elb-promotion="name:tagging is fun;position:overlay"`. ## Triggers[​](#triggers "Direct link to Triggers") There is a bunch of pre-built triggers. You don't have to deal with event listener or mutation observer initialization. | **Trigger** | **Definition** | | ----------- | ------------------------------------------------------------------------------------ | | load | after loading a page when DOM is ready | | click | when an element or a child is clicked | | impression | once, after the element has been on screen for one continuous second | | visible | like impression, but re-arms and fires again each time the element re-qualifies | | hover | each time the mouse enters the corresponding element | | submit | on valid form submission | | wait(ms) | waits ms seconds (15 seconds by default) until triggering | | pulse(ms) | recurring trigger every ms seconds (15 seconds by default) if the page is not hidden | note Trigger names are predefined and to be selected from the list, while the `action` can be an arbitrarily defined name. ### Impression and visible triggers[​](#impression-and-visible-triggers "Direct link to Impression and visible triggers") `impression` fires once, and `visible` fires every time, after an element has been on screen for one continuous second in a foreground tab. "On screen" means at least half the element, or half the viewport, whichever is smaller, along each axis. A 716px card on a 450px-tall viewport therefore qualifies once about 225px of it is showing, rather than the 358px the previous rule required: an element bigger than the viewport no longer needs to cover almost the entire screen to count. The exact point is sampled rather than measured continuously, so it can land within about 1% of the element's height either side of that mark. `visible` re-arms only once the element drops back below that bar and then requalifies, so it can fire repeatedly as a user scrolls past it and back. note Extremely tall elements, beyond roughly 100x the viewport height, are not detected. Such an element never occupies enough of its own area to register, however far it is scrolled. ### Abbreviation[​](#abbreviation "Direct link to Abbreviation") If the trigger and action values are equal, e.g. for click events, you can shorten the implementation: ``` Use the short version, instead of long ``` ### Parameters[​](#parameters "Direct link to Parameters") Some triggers require more information during initialization, while others accept optional parameters. The scroll trigger needs to know the percentage a user scrolls down, a wait trigger the number of milliseconds until the action gets triggered. Use brackets behind the trigger to pass that information. ```

``` ### Action filter[​](#action-filter "Direct link to Action filter") At some point, you might want to nest one entity inside another. To prevent an action to trigger unwanted entities, you can restrict the action to a specific entity by adding the name, e.g. `data-elbaction="load:view(product)` or `data-elbactions="load:view(product)"`. If the trigger event gets called, the result will only include the property values from the specific entities. ```
only the bar hello event fires.
``` ### Click trigger[​](#click-trigger "Direct link to Click trigger") When you click an element, the browser source reads the click in the capture phase, so it resolves the `data-elbaction` and the surrounding entity from the DOM as it exists at click time, before your app's own click handlers run. If it finds a `data-elbaction` with the `click` trigger on the clicked element or any of its parents, it fires the action. Often the image or a whole div inside a button gets clicked, not the button itself; the parent walk still resolves the action and entity for you. ``` ``` info Reading in the capture phase means `stopPropagation` in your app or a third-party widget no longer prevents a tagged click from being captured, and the entity is read before a click-driven re-render can unmount it (a common cause of events falling back to `page` in single-page apps). To restore the previous bubbling behavior, set `capture: false` on the browser source. ## Linking elements[​](#linking-elements "Direct link to Linking elements") Use the `data-elblink` tag to extend the scope of an entity by elements placed somewhere else (like modals). Specific IDs connect linked elements. They are hierarchically and can either be a parent or a child. ```
...
...
...

...

``` The second element is the parent, triggering the visible action for the `info visible` event. There can be multiple children, but there is only one parent element per ID. note `data-elb`, `data-elbaction`, `data-elbactions`, `data-elbcontext`, `data-elbglobals`, and `data-elblink` are reserved attributes, whereas `data-elb-*` attributes may be arbitrary combinations based on the related entity name. `data-elb_` is a reserved path-scoped generic that uses the same value syntax as `data-elb-` but only reaches triggers nested below it. Actions and properties can be set anywhere inside an elb scope. warning Spaces in entities, e.g., "shopping cart" or actions, e.g., "add to cart" will be replaced by underscores to "shopping\_cart" and "add\_to\_cart". tip Spaces in property values are no problem, e.g. `"category: 'summer sale'"` works fine. But it is better to set them in quotes when doing so or when using symbols, especially `:` or `;` ## Data[​](#data "Direct link to Data") ### Basic attributes[​](#basic-attributes "Direct link to Basic attributes") To specify `data`, use the name of the entity. The `data` attributes have to be inside of the entity scope or a parent. ```

...

...

``` ``` { data: { source: "parent", key: "value", foo: "bar", } } ``` ### Hierarchy[​](#hierarchy "Direct link to Hierarchy") There is a hierarchy for the `data` properties, where the order defines which values to use for similar keys. Based on the triggering action element, the closest ones or parent values will be preferred over the others. ```

``` Based on which element gets clicked, the event will contain the following data * family: `{ key: 'baz' }`, the last found data-property * parent: `{ key: 'foo' }`, a direct data-value * child: `{ key: 'bar' }`, direct value closer than the parent * sibling: `{ key: 'foo' }`, no value specified, so it takes the parent's value ### Type casting[​](#type-casting "Direct link to Type casting") Property values will be cast to their type, supporting string, number & boolean. ```

{ string: "text" }

{ int: 42, float: 3.14 }

{ bool: true }

``` ### Multiple attributes[​](#multiple-attributes "Direct link to Multiple attributes") Browsers override duplicate attributes. Hence an element can only have one `data-elb`, `data-elb-ENTITY`, and/or `data-elbaction` attribute at a time. Nevertheless, it’s possible to define multiple entities, properties, and/or actions within one attribute using quotes and semicolons. A semicolon splits key-value pairs. Therefore, it’s necessary to escape values that contain a semicolon. Quotes are here to meet your needs. To prevent a mistaken value-split, use single quotes. ```

{ "b": "a", "r": true }

{ "b": "a;r" }

``` If a single quote is part of the value, escape it with a backslash: ```

{ "bar": "it's escaped" }

``` The semicolon can be used as a separator to list multiple values inside of a `data-elb` or `data-elbaction` attribute. ```

{ "a": 1, "b": 2 }

``` ### Dynamic field values[​](#dynamic-field-values "Direct link to Dynamic field values") You might want to measure dynamic field values, e.g. the quantity of a product or the value of the selected element. Use a `#` at the beginning, followed by the attribute name to access the value of the element attribute. ```
Everyday Ruck Snack
``` To capture a selected option from a list, use `elb-ENTITY="KEY:#selected"` to get size:20L ``` ``` ### Arrays[​](#arrays "Direct link to Arrays") To use array types, add the `[]` suffix to a property's name, such as `size[]:m`. It will generate de-duplicated data properties. ```

``` ``` { data: { size: ["s", "l"], }, // ... } ``` ### Generic properties[​](#generic-properties "Direct link to Generic properties") Leave the entity name empty (only `data-elb-`) to add the property to any related entity. Explicitly named properties are preferred over generic ones. ```

``` note Explicit properties are preferred over generic ones. ``` { data: { p: 'v', // parent k: 'v', // explicit g: 'v', // generic o: 'v' // overridden by explicit }, // ... } ``` ### Scoped generic properties[​](#scoped-generic-properties "Direct link to Scoped generic properties") The blanket `data-elb-` generic is added to every trigger inside its entity, because the entity collects it through a descendant search. Use `data-elb_` (trailing underscore, no dash) when a property should reach only the triggers nested below it. It carries the same `key:value` payload as `data-elb-` (including `[]` arrays and `#dynamic` values), but it is collected only while bubbling up from the triggered element, so sibling triggers outside its branch never receive it. ```
``` Clicking the first button bubbles up through the `data-elb_` element, so it receives `size:L` along with the blanket `color:red`. Clicking the second button never passes through that element, so it has no `size`. ``` // first button { data: { name: 'A', color: 'red', size: 'L' } } // second button { data: { name: 'A', color: 'red' } } ``` A scoped value on an element closer to the trigger wins over a value set farther up the tree, including an explicit entity property on a higher element. ## Globals[​](#globals "Direct link to Globals") There might be properties that don't belong to just one event but to **all events on a page**. Those properties are called **globals**, and are **read from the DOM for every event**. Globals are arbitrary, like the data property. What is unique about them is that you can define them anywhere on a page using the `data-elbglobals` attribute. ```
``` This example will lead to the following event: ``` { "event": "entity action", "data": { "foo": "bar" }, "globals": { "outof": "scope" } // other properties omitted } ``` info Globals are collected fresh from the DOM on every event, so a `data-elbglobals` value changed between events is reflected in the next one. ## User[​](#user "Direct link to User") `data-elbuser` sets **user identity** that is applied to the **page view and every event after it**. Unlike globals, which are re-read from the DOM for every event, the user is collected **once per run**, right before the page view, and stored as persistent collector state, the same state the [walker user](/docs/collector/commands.md#user) command writes to. Tag any element with the identifiers you have available: ```
``` The page view, and every event pushed after it, carries the user: ``` { "event": "page view", "user": { "id": "u123", "loggedin": true } // other properties omitted } ``` Because the user is collected once, right before the page view, only the page view and later events carry it. Events that already fired earlier in the run, such as `window.elbLayer` pushes replayed before the page view, are not retroactively stamped. If multiple elements carry `data-elbuser`, their values are merged (last one in the DOM wins per key). An absent `data-elbuser` attribute leaves any existing user untouched; it never wipes a user set another way, such as by an earlier `walker user` call. warning Use fully anonymized & arbitrary IDs by default and check your options with persistent user IDs with your data protection officer. ## Context[​](#context "Direct link to Context") Context doesn't work like globals for every event, but is helpful information for every framing context an event is embedded in. A context could be a position, a test, or specific components for example. ```
click me
``` The context properties are tuples with the value and an index, starting at the closest parent (`[value, index]`). Access them via `event.context.key[0]`. ``` { event: "promotion click", data: { title: "click me" }, globals: { plan: "paid" }, context: { test: ["engagement", 1], recommendation: ["smart_ai", 0], }, // other properties omitted } ``` tip At elbwalker we often use context for predefined journeys and stages to measure events along a specific user journey in structured way. ## Nested entities[​](#nested-entities "Direct link to Nested entities") A `data-elb` entity within another `data-elb` entity is called a **nested entity**. The walker algorithm runs through nested entities and treats them like regular entities by gathering all related information. Nested entities are accessible in the nested array of each event. Each element is a regular entity. ```
``` This example will lead to the following event on load: ``` { "event": "mother view", "data": { "label": "caring" }, "nested": [ { "entity": "son", "data": { "age": 23 } }, { "entity": "daughter", "data": { "age": 32 }, "nested": [{ "entity": "baby", "data": { "status": "infant" } }], }, { "entity": "baby", "data": { "status": "infant" } }, ], // other properties omitted } ``` Nested entities that are nested inside another entity will be captured on both levels. note Nested entities are not available for auto-captured page view events. tip Learn more about the walkerOS event model in our documentation. ## Shadow DOM[​](#shadow-dom "Direct link to Shadow DOM") walkerOS tags elements inside shadow DOM. Entity and context resolution reaches upward through the shadow boundary: an element inside a shadow root resolves its `data-elb` entity and `data-elbcontext` from light-DOM ancestors above the host, for both open and closed roots. One limit applies to every shadow root: `data-elbglobals` is not collected inside shadow DOM, so define globals in the light DOM. ### Open shadow roots[​](#open-shadow-roots "Direct link to Open shadow roots") Elements inside `mode: 'open'` shadow roots are discovered and tracked automatically. Properties and context are collected, clicks and form submissions resolve to the inner element, and the `visible` and `impression` triggers fire when the element enters the viewport. Scroll depth is measured against the viewport, so it stays correct for elements nested inside a shadow root. ### Closed shadow roots[​](#closed-shadow-roots "Direct link to Closed shadow roots") A closed shadow root is invisible from its host (`host.shadowRoot` is `null`), so the page scan cannot reach into it. To track a closed subtree, pass the root reference returned by `attachShadow` to `walker init`: ``` const root = host.attachShadow({ mode: 'closed' }); // render your tagged markup into root elb('walker init', root); ``` Its elements then behave like any other tagged markup: `load` triggers fire on the scan, and `visible` and `impression` fire on viewport entry. Two triggers still cannot reach inside a closed root: * **click and submit** resolve through the event's composed path, which stops at the host, so an interaction inside a closed root is attributed to the host rather than the inner element. * **automatic discovery** never reaches a closed root, only the explicit `walker init` reference does. ## Auto-init with data-elbobserve[​](#auto-init-with-data-elbobserve "Direct link to Auto-init with data-elbobserve") In a single-page app, tagged content often appears after the initial page scan: a route renders a new view, a list loads lazily, a chat streams in messages. The normal way to track that content is to call `walker init` on its container after each injection. Mark the container with `data-elbobserve` instead, and the browser source watches it: any tagged content injected into it is registered for tracking automatically, and its triggers are cleaned up when the content is removed. You no longer call `walker init` per injection. Statically present tagged content inside the container is still registered by the normal page scan. `data-elbobserve` only changes how **future** injections are handled. ```
``` When your app renders tagged markup into that container, no follow-up call is needed: ``` // The SPA renders tagged markup into the observed container. // No walker init call is required afterwards. container.innerHTML = `

Everyday Ruck Snack

`; ``` `data-elbobserve` is a presence-only attribute: its presence marks the container and it takes no value. ### Open shadow roots[​](#open-shadow-roots-1 "Direct link to Open shadow roots") A `data-elbobserve` container inside an open shadow root is watched by its own observer, so tagged content injected into that shadow root is auto-registered the same way. A closed shadow root cannot be discovered, so keep using an explicit `walker init` with the root reference for closed roots (see [Shadow DOM](#shadow-dom) above). ### Scope it tightly[​](#scope-it-tightly "Direct link to Scope it tightly") Put `data-elbobserve` on the smallest container that wraps the injected content, never on the app root. The observer watches the whole subtree below the marked container, so a tight scope keeps it cheap and avoids re-processing unrelated DOM. ### Limitations[​](#limitations "Direct link to Limitations") * **Node recycling and virtualization**: a framework that reuses the *same* element instance for new content is skipped, because that element is already registered and this version does not watch attribute changes. A changed action on a recycled node is missed. For virtualized lists that recycle nodes, keep using explicit `walker init`, or ensure the framework creates fresh nodes. * **Overlapping scopes**: do not place a `walker init ` sub-scope observe container over the same subtree as a document-scope observe container. Injected content stays safe (an already-registered element is not registered twice), but the adds are re-processed redundantly. * **SVG content**: only HTML elements are auto-registered. An SVG element carrying `data-elbaction` inside an observed container is not picked up, the same as with the static page scan. * **Closed shadow roots**: not reachable by discovery, use explicit `walker init` with the root reference. ### Not in this version[​](#not-in-this-version "Direct link to Not in this version") These are deferred and may arrive later: * **Attribute-watching**: reacting to a `data-elbaction` added to an already-present element. * **A throttle option** for high-frequency mutations. * **A function-callback parameter** on the attribute. --- # JavaScript Capture events programmatically using the `elb()` function instead of HTML attributes. ## When to use[​](#when-to-use "Direct link to When to use") * Dynamic events not tied to DOM elements * Custom application logic * Events triggered by API responses * Complex conditional tracking ## Basic usage[​](#basic-usage "Direct link to Basic usage") ``` // Push an event elb('product view', { id: 'P123', name: 'Widget' }); // With context elb('order complete', { total: 99.99 }, { test: 'checkout_v2' }); ``` ## Event structure[​](#event-structure "Direct link to Event structure") The elb function accepts: 1. **Event name**: `"entity action"` format (required) 2. **Data object**: Properties for the event (optional) 3. **Context**: Contextual information (optional) 4. **Trigger**: The trigger name (optional) 5. **Nested**: Nested entities (optional) ``` elb('entity action', data, context, trigger, nested); ``` ## Examples[​](#examples "Direct link to Examples") ### Page events[​](#page-events "Direct link to Page events") ``` elb('page view', { title: document.title, path: location.pathname }); ``` ### User actions[​](#user-actions "Direct link to User actions") ``` function handleAddToCart(product) { elb('product add', { id: product.id, name: product.name, price: product.price, quantity: 1 }); } ``` ### Form submissions[​](#form-submissions "Direct link to Form submissions") ``` form.addEventListener('submit', (e) => { elb('form submit', { id: form.id, type: form.dataset.formType }); }); ``` ### API response tracking[​](#api-response-tracking "Direct link to API response tracking") ``` async function fetchProducts() { const response = await fetch('/api/products'); const products = await response.json(); elb('products load', { count: products.length, category: currentCategory }); return products; } ``` ### Error tracking[​](#error-tracking "Direct link to Error tracking") ``` window.addEventListener('error', (event) => { elb('error occur', { message: event.message, source: event.filename, line: event.lineno }); }); ``` ## Object syntax[​](#object-syntax "Direct link to Object syntax") You can also pass events as objects: ``` elb({ event: 'product view', data: { id: 'P123', name: 'Widget' }, context: { test: 'engagement' } }); ``` ## Comparison with HTML Attributes[​](#comparison-with-html-attributes "Direct link to Comparison with HTML Attributes") | Use Case | HTML Attributes | JavaScript | | -------------------- | --------------- | ---------- | | Click tracking | Best | Works | | Visibility tracking | Best | Manual | | Dynamic values | Possible | Best | | Non-DOM events | Not possible | Required | | API response events | Not possible | Required | | Conditional tracking | Limited | Best | tip Use HTML attributes for DOM-based interactions (clicks, visibility, hovers) and JavaScript for dynamic or non-DOM events. ## See also[​](#see-also "Direct link to See also") * [HTML Attributes](/docs/sources/web/browser/tagging/html-attributes.md) - Declarative DOM-based tracking * [Tagger](/docs/sources/web/browser/tagger.md) - Programmatic attribute generation * [Commands](/docs/sources/web/browser/commands.md) - Full elb API reference --- # CMP sources walkerOS provides ready-made source packages for popular Consent Management Platforms. Each source listens for CMP events and automatically calls `elb('walker consent', state)`, with no manual event listener code needed. | CMP | Package | | ------------------------------------------------------------ | --------------------------------------- | | [CookieFirst](/docs/sources/web/cmps/cookiefirst/.md) | `@walkeros/web-source-cmp-cookiefirst` | | [CookiePro / OneTrust](/docs/sources/web/cmps/cookiepro/.md) | `@walkeros/web-source-cmp-cookiepro` | | [Usercentrics](/docs/sources/web/cmps/usercentrics/.md) | `@walkeros/web-source-cmp-usercentrics` | ## Conditional activation with `require`[​](#conditional-activation-with-require "Direct link to conditional-activation-with-require") Some sources and destinations shouldn't act on lifecycle events right away. They may depend on consent status being resolved first, or need information provided by other sources (like session IDs). The `require` option solves this by gating `on()` delivery (for sources) or initialization (for destinations) until specific collector events have fired. Add `require` to any source or destination `config` to gate it on the listed events: ``` { "sources": { "consent": { "package": "@walkeros/web-source-cmp-cookiefirst" }, "session": { "package": "@walkeros/web-source-session", "config": { "require": ["consent"] } }, "dataLayer": { "package": "@walkeros/web-source-dataLayer", "config": { "require": ["consent"] } } }, "destinations": { "ga4": { "package": "@walkeros/web-destination-gtag", "config": { "require": ["consent"], "consent": { "marketing": true } } } } } ``` In this example the CMP source initializes immediately and fires `elb('walker consent', state)` once the user makes a choice. That `consent` event unblocks the session source, the dataLayer source, and the GA4 destination, all at once. ### How `require` works[​](#how-require-works "Direct link to how-require-works") 1. Sources are registered immediately and their `Instance.init()` runs eagerly after registration. `require` does not block init — it gates `on()` delivery. Lifecycle events targeted at a source with unmet `require` are buffered in `Instance.queueOn`. 2. Destinations with `require` are held as **pending** during initialization until their listed events fire. 3. A `require` entry is satisfied by the collector's **current recorded state**, not only by a future event. So order does not matter: a step activates whether the required state (such as a CMP applying consent) was recorded before or after the step registered. The collector reconciles every pending step against current state at each state change, at the run barrier, and once after all sources have registered. 4. Sources whose `require` just emptied have their queued lifecycle events replayed via `source.on(type, data)`. Destinations whose `require` just emptied are registered and become eligible to initialize/send; any previously queued events are preserved and delivered once the collector is allowed to push, so no data is lost. ### Key points[​](#key-points "Direct link to Key points") * **CMP sources must not have `require`**: they need to fire consent immediately, so the collector should be able to call `on('consent', …)` on them right away * **Accepted values**: any collector event type, such as `"consent"`, `"session"`, `"user"`, `"run"`, `"config"`, `"globals"`, etc. * **AND logic**: `require: ["consent", "session"]` waits for **both** events * **Cascading**: a CMP source fires `consent` → session source's require clears, its queued `on('consent', …)` replays, it fires `session` → a dataLayer source waiting for `["session"]` clears next ## How CMP sources work[​](#how-cmp-sources-work "Direct link to How CMP sources work") 1. Detecting if the CMP is already loaded and reading existing consent 2. Listening for CMP-specific events on consent changes 3. Mapping CMP categories to walkerOS consent groups via `categoryMap` 4. Calling `elb('walker consent', state)` with the mapped state CMP and session sources perform their initial consent read during `init()`, not in the source factory. The factory stays side-effect free; the consent emit happens once the collector runs the source's `init()`. Combined with the current-state `require` matching above, this means a `require:["consent"]` step activates reliably regardless of which source provided the consent or when. All CMP sources support `explicitOnly` (default: `true`) to filter out implicit/default consent and only process active user choices. For more details on how consent works in walkerOS, see the [consent management guide](/docs/guides/consent/.md). --- # CookieFirst Integrates [CookieFirst](https://cookiefirst.com/) consent management with walkerOS by listening for CookieFirst events and translating consent categories to walkerOS consent groups. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-source-cmp-cookiefirst ``` ``` import { startFlow } from '@walkeros/collector'; import { sourceCookieFirst } from '@walkeros/web-source-cmp-cookiefirst'; await startFlow({ sources: { consent: { code: sourceCookieFirst, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------ | ---- | | `categoryMap` | `Record` | Map the CMP's consent categories (keys) to walkerOS consent groups (values). | | | `explicitOnly` | `boolean` | Only process consent after the user made an explicit choice. Ignores default/implicit states. Default: true. | | | `globalName` | `string` | Custom name for the CookieFirst global on window. Default: 'CookieFirst'. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Category map override Custom categoryMap remaps performance to statistics instead of analytics Event ``` { "necessary": true, "functional": false, "performance": true, "advertising": false } ``` Mapping ``` { "settings": { "categoryMap": { "performance": "statistics" } } } ``` Out ``` elb("walker consent", { "functional": true, "statistics": true, "marketing": false }) ``` ### cf\_init detection CMP detected via cf\_init CustomEvent (primary detection path) Event ``` { "necessary": true, "functional": false, "performance": false, "advertising": true } ``` Out ``` elb("walker consent", { "functional": true, "analytics": false, "marketing": true }) ``` ### Full consent CookieFirst reports all categories granted and the source forwards a walker consent command with all groups true. Event ``` { "necessary": true, "functional": true, "performance": true, "advertising": true } ``` Out ``` elb("walker consent", { "functional": true, "analytics": true, "marketing": true }) ``` ### Partial consent A partial CookieFirst grant maps analytics and marketing to false in the emitted walker consent command. Event ``` { "necessary": true, "functional": true, "performance": false, "advertising": false } ``` Out ``` elb("walker consent", { "functional": true, "analytics": false, "marketing": false }) ``` ## How it works[​](#how-it-works "Direct link to How it works") 1. **Initialization**: Checks if CookieFirst is already loaded and processes existing consent state from `window.CookieFirst.consent`. 2. **cf\_init event**: Listens for the `cf_init` event fired when CookieFirst banner initializes. 3. **cf\_consent event**: Listens for the `cf_consent` event fired when user changes consent preferences. 4. **Consent mapping**: Translates CookieFirst categories to walkerOS consent groups and calls `elb('walker consent', state)`. ### Default category mapping[​](#default-category-mapping "Direct link to Default category mapping") ``` { necessary: 'functional', functional: 'functional', performance: 'analytics', advertising: 'marketing', } ``` When multiple CookieFirst categories map to the same walkerOS group (e.g., both `necessary` and `functional` to `functional`), the source uses OR logic: if ANY source category is `true`, the target group is `true`. ### Custom mapping example[​](#custom-mapping-example "Direct link to Custom mapping example") ``` await startFlow({ sources: { consent: { code: sourceCookieFirst, config: { settings: { categoryMap: { performance: 'statistics', // Use 'statistics' instead of 'analytics' }, explicitOnly: true, }, }, }, }, }); ``` ## Reference[​](#reference "Direct link to Reference") * [CookieFirst public API documentation](https://support.cookiefirst.com/hc/en-us/articles/360011568738-Cookie-Banner-Public-API-documentation) * [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/cmps/cookiefirst) --- # CookiePro / OneTrust Integrates [CookiePro / OneTrust](https://www.onetrust.com/) consent management with walkerOS by mapping CookiePro category IDs (`C0001`, `C0002`, ...) to walkerOS consent groups. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-source-cmp-cookiepro ``` ``` import { startFlow } from '@walkeros/collector'; import { sourceCookiePro } from '@walkeros/web-source-cmp-cookiepro'; await startFlow({ sources: { consent: { code: sourceCookiePro, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------- | ------------------------ | ------------------------------------------------------------------------------------------------- | ---- | | `categoryMap` | `Record` | Map the CMP's consent categories (keys) to walkerOS consent groups (values). | | | `explicitOnly` | `boolean` | Only process consent after the user closed the OneTrust banner (IsAlertBoxClosed). Default: true. | | | `globalName` | `string` | Custom name for the OneTrust global on window. Default: 'OneTrust'. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Category map override Custom categoryMap remaps C0002 from analytics to statistics Event ``` ",C0001,C0002," ``` Mapping ``` { "categoryMap": { "C0002": "statistics" } } ``` Out ``` elb("walker consent", { "functional": true, "statistics": true, "marketing": false }) ``` ### Full consent A CookiePro OptanonConsent cookie listing all groups is translated into a walker consent command with all true. Event ``` ",C0001,C0002,C0003,C0004,C0005," ``` Out ``` elb("walker consent", { "functional": true, "analytics": true, "marketing": true }) ``` ### Minimal consent Only the necessary CookiePro group is granted so analytics and marketing map to false in the walker consent command. Event ``` ",C0001," ``` Out ``` elb("walker consent", { "functional": true, "analytics": false, "marketing": false }) ``` ### SDK loaded detection Immediate detection when OneTrust SDK is already loaded with IsAlertBoxClosed() = true Event ``` ",C0001,C0003,C0004," ``` Out ``` elb("walker consent", { "functional": true, "analytics": false, "marketing": true }) ``` ## How it works[​](#how-it-works "Direct link to How it works") 1. **Already loaded**: Checks if `window.OneTrust` and `window.OptanonActiveGroups` already exist. If so, processes consent immediately. 2. **OptanonWrapper**: If the SDK isn't loaded yet, wraps the global `OptanonWrapper` callback (preserving any existing wrapper). The wrapper self-unwraps after the first call. 3. **OneTrustGroupsUpdated event**: Listens for the `OneTrustGroupsUpdated` window event, which fires on every consent change. 4. **Parsing**: Splits the `OptanonActiveGroups` comma-separated string, maps category IDs through `categoryMap`, and calls `elb('walker consent', state)`. Sets explicit `false` for all mapped groups not in the active list. ### CookiePro categories[​](#cookiepro-categories "Direct link to CookiePro categories") CookiePro registers cookies and assigns them to categories. Those categories are mapped to walkerOS consent groups: ![CookiePro categorizations](/assets/images/cookiepro-categorizations-f51a6f89c20bf03ee21163b913637225.png) ### Default category mapping[​](#default-category-mapping "Direct link to Default category mapping") ``` { C0001: 'functional', // Strictly Necessary C0002: 'analytics', // Performance C0003: 'functional', // Functional C0004: 'marketing', // Targeting C0005: 'marketing', // Social Media } ``` Category ID comparison is case-insensitive. Unmapped category IDs are ignored since CookiePro's opaque IDs are meaningless without a mapping. All mapped walkerOS groups receive explicit `true`/`false` values. Absent groups are set to `false` so destinations know which consent is denied. Custom entries are merged with the default mapping. Specify only the categories you want to override. All other defaults remain active. ### Custom mapping example[​](#custom-mapping-example "Direct link to Custom mapping example") ``` await startFlow({ sources: { consent: { code: sourceCookiePro, config: { settings: { categoryMap: { C0002: 'statistics', // Use 'statistics' instead of 'analytics' }, explicitOnly: true, }, }, }, }, }); ``` ### Timing considerations[​](#timing-considerations "Direct link to Timing considerations") The source handles all timing scenarios: * **SDK loads before source**: The "already loaded" check reads existing consent from `OptanonActiveGroups` immediately. * **Source loads before SDK**: The `OptanonWrapper` wrapping intercepts the SDK's init callback. * **`explicitOnly` (default)**: Uses `OneTrust.IsAlertBoxClosed()` to determine if the user has actively interacted with the consent banner. ## Reference[​](#reference "Direct link to Reference") * [OneTrust SDK documentation](https://my.onetrust.com/s/article/UUID-66bcaaf1-c7ca-5f32-6760-c75a1337c226) * [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/cmps/cookiepro) --- # Usercentrics Integrates [Usercentrics](https://usercentrics.com/) consent management with walkerOS using the official Usercentrics events and consent getters, mapping category or service consent state to walkerOS consent groups. We recommend the source package below over a hand-written event listener: it handles version detection, returning-visitor restore, and explicit-consent gating for you. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-source-cmp-usercentrics ``` ``` import { startFlow } from '@walkeros/collector'; import { sourceUsercentrics } from '@walkeros/web-source-cmp-usercentrics'; await startFlow({ sources: { consent: { code: sourceUsercentrics, config: { settings: { categoryMap: { essential: 'functional', functional: 'functional', marketing: 'marketing', }, }, }, }, }, }); ``` ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `categoryMap` | `Record` | Map the CMP's consent categories (keys) to walkerOS consent groups (values). | | | `explicitOnly` | `boolean` | Only publish when the user has actively decided (V3: consent.type EXPLICIT; V2: an EXPLICIT entry in service consent history). Implicit/default page-load states are suppressed. Set false to publish any snapshot including implicit. Default: true. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Category map override A custom categoryMap remaps essential to functional and functional to analytics before emitting the walker consent command. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "marketing", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } } ] ``` Mapping ``` { "settings": { "categoryMap": { "essential": "functional", "functional": "analytics" } } } ``` Out ``` elb("walker consent", { "functional": true, "analytics": true, "marketing": true }) ``` ### Consent change via CMP event An ACCEPT\_ALL decision fires UC\_UI\_CMP\_EVENT; the source re-reads the services and emits the updated consent. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "marketing", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } } ] ``` Out ``` elb("walker consent", { "essential": true, "functional": true, "marketing": true }) ``` ### First visit implicit (suppressed) A first-visit snapshot carrying only implicit history is suppressed by the default explicitOnly gate, so no consent command is emitted. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "implicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": false, "history": [ { "type": "implicit", "status": false } ] } }, { "categorySlug": "marketing", "consent": { "status": false, "history": [ { "type": "implicit", "status": false } ] } } ] ``` Out ``` // no output ``` ### Full consent Usercentrics reports every category accepted via an explicit decision; the source emits a walker consent command granting essential, functional, and marketing. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "marketing", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } } ] ``` Out ``` elb("walker consent", { "essential": true, "functional": true, "marketing": true }) ``` ### Minimal consent A "Deny all" explicit decision leaves only essential granted; functional and marketing are emitted as false. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": false, "history": [ { "type": "explicit", "status": false } ] } }, { "categorySlug": "marketing", "consent": { "status": false, "history": [ { "type": "explicit", "status": false } ] } } ] ``` Out ``` elb("walker consent", { "essential": true, "functional": false, "marketing": false }) ``` ### Returning visitor static read When the CMP is already initialized with a stored explicit decision, the static read at init re-publishes that choice without any further event. Event ``` [ { "categorySlug": "essential", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "functional", "consent": { "status": true, "history": [ { "type": "explicit", "status": true } ] } }, { "categorySlug": "marketing", "consent": { "status": false, "history": [ { "type": "explicit", "status": false } ] } } ] ``` Out ``` elb("walker consent", { "essential": true, "functional": true, "marketing": false }) ``` | Setting | Type | Default | Description | | -------------- | ------------------------ | ------------------- | ---------------------------------------------------------- | | `apiVersion` | `'auto' \| 'v2' \| 'v3'` | `'auto'` | Which Usercentrics API to target (auto-detects by default) | | `categoryMap` | `Record` | `{}` | Maps Usercentrics categories to walkerOS consent groups | | `explicitOnly` | `boolean` | `true` | Only publish when the user has actively decided | | `v3EventName` | `string` | `'UC_UI_CMP_EVENT'` | V3 event name, override only for a custom admin event | There is no configurable data-layer event setting: the source listens to the always-emitted official Usercentrics events, so no `eventName` configuration or Usercentrics admin window-event setup is required. ### V2 vs V3 support[​](#v2-vs-v3-support "Direct link to V2 vs V3 support") The source supports both Usercentrics V2 (`window.UC_UI`) and V3 (`window.__ucCmp`) APIs. With the default `apiVersion: 'auto'`, detection runs at init: * If the CMP is already initialized, consent is read **statically** through the official getters (V2 `UC_UI.getServicesBaseInfo()`, V3 `__ucCmp.getConsentDetails()`). * If no CMP is present yet, the source listens for `UC_UI_INITIALIZED` and reads the current state once the CMP signals it is ready, so late-loading CMPs are still caught. * When both APIs are available, V3 is preferred. Set `apiVersion: 'v2'` or `'v3'` to force a specific integration. ### Custom mapping example[​](#custom-mapping-example "Direct link to Custom mapping example") ``` await startFlow({ sources: { consent: { code: sourceUsercentrics, config: { settings: { categoryMap: { essential: 'functional', functional: 'functional', marketing: 'marketing', }, explicitOnly: true, }, }, }, }, }); ``` ## How it works[​](#how-it-works "Direct link to How it works") The source uses Usercentrics' official integration surface across both API versions: 1. **Already initialized**: if the CMP loaded before the source, consent is read statically through the official getters (V2 `UC_UI.getServicesBaseInfo()`, V3 `__ucCmp.getConsentDetails()`). 2. **CMP loads after the source**: the source listens for `UC_UI_INITIALIZED` and reads the current consent state once the CMP is ready. 3. **User decisions**: the source listens for `UC_UI_CMP_EVENT` (consent actions `ACCEPT_ALL`, `DENY_ALL`, and `SAVE`) and republishes the updated state. 4. **Category mapping**: maps categories via `categoryMap` and calls `elb('walker consent', state)` with the mapped consent state. ### Explicit consent[​](#explicit-consent "Direct link to Explicit consent") By default (`explicitOnly: true`), the source publishes only states the user has actively decided. It reads this from the official consent metadata: V3 `consent.type === EXPLICIT`, and V2 an `EXPLICIT` entry in the service consent history. First-visit defaults stay suppressed. Set `explicitOnly: false` to also publish implicit/default consent. ### Timing considerations[​](#timing-considerations "Direct link to Timing considerations") A returning visitor's prior choice is applied on page load, either from the static getter read (CMP already initialized) or on `UC_UI_INITIALIZED` (CMP loads later). First-visit defaults stay suppressed under the default `explicitOnly: true`. Set `explicitOnly: false` to publish any snapshot, including implicit defaults. Ensure the consent source has no `require` constraints so it initializes immediately. ## Reference[​](#reference "Direct link to Reference") * [Usercentrics developer documentation](https://usercentrics.com/docs/) * [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/cmps/usercentrics) --- [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/datalayer)[ ](https://www.npmjs.com/package/@walkeros/web-source-datalayer) [Package](https://www.npmjs.com/package/@walkeros/web-source-datalayer) # DataLayer source Integrates with existing Google Analytics 4 and GTM dataLayer implementations by intercepting `dataLayer.push()` calls. Where this fits The DataLayer source is a **web source** in the walkerOS flow: It bridges existing GA4/GTM implementations to walkerOS destinations. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/web-source-datalayer ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { sourceDataLayer } from '@walkeros/web-source-datalayer'; const { collector, elb } = await startFlow({ sources: { dataLayer: { code: sourceDataLayer, config: { settings: { name: 'dataLayer', // Name of global dataLayer array prefix: 'dataLayer', // Event prefix for filtering }, }, }, }, destinations: { // Your destinations }, }); ``` Add to your `flow.json` sources: ``` "sources": { "dataLayer": { "package": "@walkeros/web-source-datalayer", "config": { "settings": { "name": "dataLayer", "prefix": "dataLayer" } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | -------- | ---------- | ------------------------------------------------------------------------ | ---- | | `name` | `string` | DataLayer variable name (default: dataLayer) | | | `prefix` | `string` | Event prefix for filtering which events to process | | | `filter` | `function` | Custom filter function: (event: unknown) => boolean \| Promise\ | | ## Mapping[​](#mapping "Direct link to Mapping") Per-event rules under `config.mapping`. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). | Property | Type | Description | More | | --------- | ----- | ---------------------------------------------------------------------------------------------------------------- | ---- | | `command` | `any` | Mapping.Value evaluated against the raw dataLayer arguments to build walker command data (e.g. consent updates). | | ## Examples ### Consent update A gtag consent update is captured from the dataLayer as a walker dataLayer consent update event. Event ``` [ "consent", "update", { "ad_storage": "granted", "analytics_storage": "granted" } ] ``` Out ``` elb({ "name": "dataLayer consent update", "data": { "ad_storage": "granted", "analytics_storage": "granted" } }) ``` ### Direct dataLayer event A plain object pushed directly onto the dataLayer is captured as a walker dataLayer custom event. Event ``` { "event": "custom_event", "category": "engagement", "label": "video_play" } ``` Out ``` elb({ "name": "dataLayer custom_event", "data": { "category": "engagement", "label": "video_play" } }) ``` ### gtag add\_to\_cart A gtag add\_to\_cart call pushed to the dataLayer is captured as a walker dataLayer add\_to\_cart event. Event ``` [ "event", "add_to_cart", { "currency": "EUR", "value": 15.25, "items": [ { "item_id": "SKU_12345", "item_name": "T-Shirt", "item_variant": "red", "quantity": 1, "price": 15.25 } ] } ] ``` Out ``` elb({ "name": "dataLayer add_to_cart", "data": { "currency": "EUR", "value": 15.25, "items": [ { "item_id": "SKU_12345", "item_name": "T-Shirt", "item_variant": "red", "quantity": 1, "price": 15.25 } ] } }) ``` ### gtag purchase A gtag purchase call pushed to the dataLayer is captured as a walker dataLayer purchase event with item details. Event ``` [ "event", "purchase", { "transaction_id": "T-12345", "value": 25.42, "currency": "EUR", "items": [ { "item_id": "SKU-1", "item_name": "T-Shirt", "quantity": 1 } ] } ] ``` Out ``` elb({ "name": "dataLayer purchase", "data": { "transaction_id": "T-12345", "value": 25.42, "currency": "EUR", "items": [ { "item_id": "SKU-1", "item_name": "T-Shirt", "quantity": 1 } ] } }) ``` ### gtag view\_item A gtag view\_item call pushed to the dataLayer is captured as a walker dataLayer view\_item event with item data. Event ``` [ "event", "view_item", { "currency": "EUR", "value": 29.99, "items": [ { "item_id": "SKU_67890", "item_name": "Sneakers", "item_category": "Footwear", "price": 29.99 } ] } ] ``` Out ``` elb({ "name": "dataLayer view_item", "data": { "currency": "EUR", "value": 29.99, "items": [ { "item_id": "SKU_67890", "item_name": "Sneakers", "item_category": "Footwear", "price": 29.99 } ] } }) ``` ## How it works[​](#how-it-works "Direct link to How it works") The dataLayer source intercepts `dataLayer.push()` calls and transforms them into walkerOS events: 1. **Intercepts** existing `dataLayer.push()` calls 2. **Filters** events based on prefix or custom filter function 3. **Transforms** dataLayer format to walkerOS event format 4. **Forwards** to collector for processing ``` // Existing dataLayer code (unchanged) window.dataLayer = window.dataLayer || []; dataLayer.push({ event: 'purchase', value: 99.99 }); // Automatically captured and transformed by sourceDataLayer // → Sent to walkerOS collector as standard event ``` ## Custom filtering[​](#custom-filtering "Direct link to Custom filtering") Filter which dataLayer events get processed: ``` const { elb } = await startFlow({ sources: { dataLayer: { code: sourceDataLayer, config: { settings: { filter: (event) => { // Only process purchase and add_to_cart events if (typeof event === 'object' && event !== null) { const e = event as { event?: string }; return e.event === 'purchase' || e.event === 'add_to_cart'; } return false; }, }, }, }, }, }); ``` ## Source-level mapping[​](#source-level-mapping "Direct link to Source-level mapping") The source emits walkerOS events whose `name` is `" "`. The collector splits `name` on the first space into `entity` and `action`, so the prefix becomes the entity and the gtag action becomes the action. Source-level mapping rules key on that pair: ``` "sources": { "dataLayer": { "package": "@walkeros/web-source-datalayer", "config": { "settings": { "prefix": "dataLayer" }, "mapping": { "dataLayer": { "add_to_cart": { "name": "product add", "data": { "map": { "id": "items.0.item_id", "name": "items.0.item_name", "price": "value", "currency": "currency", "quantity": "items.0.quantity" } } }, "purchase": { "name": "order complete", "data": { "map": { "id": "transaction_id", "total": "value", "currency": "currency" } } } } } } } } ``` For the gtag commands `consent`, `config`, and `set`, the action equals the command name (any trailing token such as `update` or a measurement ID is dropped by the entity/action split). Branch on the dropped value via rule-level `condition` or `event.data` if you need to distinguish, for example, `consent default` from `consent update`. If you set a custom `prefix` (for example `"gtag"`), use that string as the entity key: `mapping.gtag.add_to_cart`. ## Migration strategy[​](#migration-strategy "Direct link to Migration strategy") Use the dataLayer source for gradual migration from GA4/GTM: ### Phase 1: Add walkerOS alongside existing dataLayer[​](#phase-1-add-walkeros-alongside-existing-datalayer "Direct link to Phase 1: Add walkerOS alongside existing dataLayer") ``` // Existing code continues to work dataLayer.push({ event: 'purchase', value: 99.99 }); // Now also captured by walkerOS ``` ### Phase 2: Map dataLayer events to walkerOS destinations[​](#phase-2-map-datalayer-events-to-walkeros-destinations "Direct link to Phase 2: Map dataLayer events to walkerOS destinations") ``` destinations: { ga4: { code: destinationGtag, config: { mapping: { // Map dataLayer purchase to GA4 format } } } } ``` ### Phase 3: Gradually replace dataLayer.push with elb()[​](#phase-3-gradually-replace-datalayerpush-with-elb "Direct link to Phase 3: Gradually replace dataLayer.push with elb()") ``` // Old dataLayer.push({ event: 'purchase', value: 99.99 }); // New elb('order complete', { total: 99.99 }); ``` ## Next steps[​](#next-steps "Direct link to Next steps") * Review [event mapping](/docs/mapping/.md) for transforming dataLayer events * Check [GA4 destination](/docs/destinations/web/gtag/.md) for Google Analytics integration --- [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/web/sources/session)[ ](https://www.npmjs.com/package/@walkeros/web-source-session) [Package](https://www.npmjs.com/package/@walkeros/web-source-session) # Session source Standalone session detection and management that can be composed with any walkerOS source. ## Installation[​](#installation "Direct link to Installation") * Integrated * Bundled Install the packages: ``` npm install @walkeros/collector @walkeros/web-source-session ``` Configure in your code: ``` import { startFlow } from '@walkeros/collector'; import { sourceBrowser } from '@walkeros/web-source-browser'; import { sourceSession } from '@walkeros/web-source-session'; const { collector, elb } = await startFlow({ sources: { browser: sourceBrowser, session: { code: sourceSession, config: { settings: { storage: true, }, }, }, }, }); ``` Add to your `flow.json` sources: ``` "sources": { "browser": { "package": "@walkeros/web-source-browser" }, "session": { "package": "@walkeros/web-source-session", "config": { "settings": { "storage": true } } } } ``` [See Bundled Mode setup →](/docs/getting-started/modes/bundled.md) | [CLI reference →](/docs/apps/cli.md) ## Detection methods[​](#detection-methods "Direct link to Detection methods") The session source uses two complementary methods: | Method | Storage | Use Case | | ----------- | ------------ | ------------------------------- | | **Window** | None | Privacy-first, per-page session | | **Storage** | localStorage | Cross-page session tracking | ### Window-based detection[​](#window-based-detection "Direct link to Window-based detection") Without storage, session detection relies on browser signals: **Navigation type:** ``` // Session starts when navigation type indicates new visit const isNewSession = performance.navigation.type === 0 || // Navigate (new page load) document.referrer === '' || // No referrer (direct visit) !isSameHost(document.referrer); // External referrer ``` **Marketing parameters:** UTM and other marketing parameters trigger session start. The following are recognized by default: ``` // UTM parameters → mapped to short names utm_source → source utm_medium → medium utm_campaign → campaign utm_term → term utm_content → content // Click IDs → stored as clickId + original key gclid, dclid, fbclid, msclkid, ttclid, twclid, igshid, sclid ``` To capture additional URL parameters, pass a `parameters` map. Each entry maps a URL parameter name to the property name it should be stored as in the session data: ``` session: { code: sourceSession, config: { settings: { storage: true }, parameters: { ref: 'referral', // ?ref=newsletter → { referral: 'newsletter' } affiliate_id: 'affiliate', // ?affiliate_id=abc → { affiliate: 'abc' } }, }, } ``` ### Storage-based detection[​](#storage-based-detection "Direct link to Storage-based detection") With `storage: true`, sessions persist across page loads: ``` session: { code: sourceSession, config: { settings: { storage: true, length: 30, // Session timeout in minutes }, }, } ``` **Session lifecycle:** ``` ┌─────────────────────────────────────────────────────────────┐ │ Session Timeline │ ├─────────────────────────────────────────────────────────────┤ │ │ │ User visits Page 2 Page 3 30min idle │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ │ │ [Session 1] ─────────────────────────────► [Expires] │ │ id: abc123 │ │ │ │ User returns │ │ │ │ │ ▼ │ │ [Session 2] │ │ id: def456 │ └─────────────────────────────────────────────────────────────┘ ``` **Storage keys:** | Key | Default | Description | | ------------ | -------------- | ------------------------------- | | `sessionKey` | `elbSessionId` | localStorage key for session ID | | `deviceKey` | `elbDeviceId` | localStorage key for device ID | ## Session start event[​](#session-start-event "Direct link to Session start event") When a new session is detected, the source pushes a `session start` event and sets the user's session and device IDs on the collector: ``` { name: 'session start', data: { isStart: true, // Always true when this event fires isNew: true, // True only on the device's very first session id: 'abc123', // Session ID device: 'xyz789', // Device ID (storage mode only) storage: true, // Whether storage is active count: 5, // Total session count for this device runs: 1, // Page views in this session start: 1704067200, // Session start timestamp referrer: 'google.com', // Hostname only (not full URL) marketing: { source: 'google', medium: 'cpc', campaign: 'winter-sale', clickId: 'gclid', // Which click ID platform (if present) gclid: 'AW-123...', // The actual click ID value } } } ``` **`isStart` vs `isNew`:** | Field | Meaning | | --------------- | ---------------------------------------------------------------- | | `isStart: true` | A new session began on this page load | | `isNew: true` | This is the device's very first session ever (storage mode only) | A returning visitor starting their second session has `isStart: true` but `isNew: false`. `isNew` only appears in storage mode since window-only mode has no device memory. **Click ID parameters** are stored twice: `clickId` identifies which platform originated the click (e.g. `'gclid'`), and a separate field stores the actual value (e.g. `gclid: 'AW-123...'`). This makes it easy to filter by platform without inspecting the value. ## Configuration[​](#configuration "Direct link to Configuration") This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see [source configuration](/docs/sources/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `storage` | `boolean` | Enable persistent storage for session/device IDs | | | `consent` | `string \| array` | Consent key(s) required to enable storage mode | | | `length` | `number` | Session timeout in minutes | | | `pulse` | `boolean` | Keep session alive on each event | | | `sessionKey` | `string` | Storage key for session ID | | | `sessionStorage` | `'local' \| 'session'` | Storage type for session | | | `deviceKey` | `string` | Storage key for device ID | | | `deviceStorage` | `'local' \| 'session'` | Storage type for device | | | `deviceAge` | `number` | Device ID age in days | | | `cb` | `function` | Custom session callback function or false to disable | | | `clickIds` | `Array` | Custom click-ID registry. Entries with a \`param\` matching a default override the platform name in place; new params append to the end of the priority list. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### New marketing session A visit with UTM parameters starts a new session and emits walker user, session, and session start calls. Event ``` { "storage": true } ``` Out ``` elb("user", { "session": "s3ss10n-id", "device": "d3v1c3-id" }); elb("session", { "id": "s3ss10n-id", "start": 1700000000000, "isNew": true, "count": 1, "runs": 1, "marketing": true, "source": "google", "medium": "cpc", "campaign": "winter-sale", "referrer": "", "device": "d3v1c3-id", "isStart": true, "storage": true, "updated": 1700000000000 }); elb({ "name": "session start", "data": { "id": "s3ss10n-id", "start": 1700000000000, "isNew": true, "count": 1, "runs": 1, "marketing": true, "source": "google", "medium": "cpc", "campaign": "winter-sale", "referrer": "", "device": "d3v1c3-id", "isStart": true, "storage": true, "updated": 1700000000000 } }) ``` ### Returning visitor A returning visit with a google referrer reuses the stored device id and increments the session count. Event ``` { "storage": true } ``` Out ``` elb("user", { "session": "n3w-s3ss10n", "device": "d3v1c3-id" }); elb("session", { "id": "n3w-s3ss10n", "start": 1700001000000, "isNew": false, "count": 3, "runs": 1, "referrer": "google.com", "device": "d3v1c3-id", "isStart": true, "storage": true, "updated": 1700001000000 }); elb({ "name": "session start", "data": { "id": "n3w-s3ss10n", "start": 1700001000000, "isNew": false, "count": 3, "runs": 1, "referrer": "google.com", "device": "d3v1c3-id", "isStart": true, "storage": true, "updated": 1700001000000 } }) ``` ## Consent integration[​](#consent-integration "Direct link to Consent integration") Session detection adapts to consent state: | Consent State | Behavior | | --------------- | ---------------------------------- | | No consent | Window-only detection (no storage) | | Consent granted | Storage-based with device ID | | Consent revoked | Falls back to window-only | ``` // Configure consent requirement session: { code: sourceSession, config: { settings: { consent: 'analytics', // Wait for this consent key storage: true, }, }, } // Grant consent to enable storage elb('walker consent', { analytics: true }); ``` ## Pulse mode[​](#pulse-mode "Direct link to Pulse mode") With `pulse: true`, the source updates the session's last-seen timestamp on each page load without counting it as a new page view and without triggering a new `session start` event. Use this for heartbeat-style presence tracking where you want to extend session lifetime without inflating `runs`: ``` session: { code: sourceSession, config: { settings: { storage: true, pulse: true, // Extends session, does not increment runs }, }, } ``` ## Custom session callback[​](#custom-session-callback "Direct link to Custom session callback") The `cb` setting lets you hook into session detection. It receives the detected session data, the collector instance, and the default callback: ``` session: { code: sourceSession, config: { settings: { cb: (session, collector, defaultCb) => { // Run custom logic first console.log('New session:', session.id); // Then run the default behavior (sets user IDs + pushes event) defaultCb(session, collector); }, }, }, } ``` Set `cb: false` to completely disable the default behavior. No `user` commands and no `session start` event will be emitted. This is useful when you want full control over what happens after detection. ## Next steps[​](#next-steps "Direct link to Next steps") * [Browser Source](/docs/sources/web/browser/.md) - DOM-based event tracking * [DataLayer Source](/docs/sources/web/dataLayer/.md) - GTM/GA4 integration * [Consent Guide](/docs/guides/consent/.md) - Consent management patterns --- # Stores Without stores, each event in a flow is processed in isolation: no shared state, no caching, no persisted data between requests. Stores add a **shared memory layer** that any component in your flow (transformers, destinations) can read from and write to. A common example: serving the walker.js script from your own infrastructure. The file lives in a store (S3 or filesystem) with `file: true`, and a transformer reads it on each request: no hardcoded paths, no rebuilds needed to swap the file. Stores hold one canonical value type: structured data, with binary as a first-class leaf. By default a store is a structured key-value store (sessions, lookups, cached responses). Set `file: true` on a byte-native backend (filesystem, S3, GCS) to make it persist raw bytes byte-exact instead, the mode for serving assets such as walker.js. The Sheets store is structured-only and rejects `file: true`. The `file` flag is documented in the [Configuration](#configuration) table below. ## When stores are active[​](#when-stores-are-active "Direct link to When stores are active") Stores are initialized **before** any other component in the flow. Once running, they're available to transformers and destinations via `$store.storeId` wiring. A transformer might cache a processed result; a destination might check a store before making an external API call. ``` Flow startup order: 1. Stores ← initialized first 2. Transformers ← can reference stores via $store.id 3. Destinations ← can reference stores via $store.id 4. Sources ← start pushing events ``` ## Choosing a store[​](#choosing-a-store "Direct link to Choosing a store") | Store | Platform | Persistence | Speed | Best for | | ------------------------------------------- | ----------- | ------------------------ | ------------------------- | -------------------------------------------- | | [**Filesystem**](/docs/stores/server/fs.md) | Server only | Disk (survives restarts) | Fast (local I/O) | Local dev, Docker with baked-in assets | | [**S3**](/docs/stores/server/s3.md) | Server only | Cloud (always available) | Network I/O | Managed deployments, shared assets, hot-swap | | [**GCS**](/docs/stores/server/gcs.md) | Server only | Cloud (always available) | Network I/O | Cloud Run / GKE deployments, GCP-native | | [**Sheets**](/docs/stores/server/sheets.md) | Server only | Cloud (Google Sheets) | Slow (HTTP, rate-limited) | Demos, prototypes with a spreadsheet UI | For ephemeral, in-process caching, use the built-in [cache tier](/docs/stores/cache.md) on any of the stores above, no separate package needed. To read from or write to a store from a step without wiring `$store` and writing `$code:`, use the declarative [`state`](/docs/collector/state.md) block on a source, transformer, or destination. ## Wiring stores to components[​](#wiring-stores-to-components "Direct link to Wiring stores to components") Components reference stores via `$store.storeId` in their `env` configuration. The collector resolves this at startup and passes the live store instance to the component. * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { storeS3Init } from '@walkeros/server-store-s3'; import { transformerFile } from '@walkeros/server-transformer-file'; await startFlow({ stores: { assets: { code: storeS3Init, config: { file: true, settings: { bucket: 'my-assets', endpoint: 'https://s3.eu-west-1.amazonaws.com', accessKeyId: process.env.S3_ACCESS_KEY, secretAccessKey: process.env.S3_SECRET_KEY, }, }, }, }, transformers: { file: { code: transformerFile, config: { settings: { prefix: '/static' } }, env: { store: '$store.assets' }, // wired to the assets store }, }, }); ``` ``` { "stores": { "assets": { "package": "@walkeros/server-store-s3", "config": { "file": true, "settings": { "bucket": "my-assets", "endpoint": "https://s3.eu-west-1.amazonaws.com", "accessKeyId": "$env.S3_ACCESS_KEY", "secretAccessKey": "$env.S3_SECRET_KEY" } } } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.assets" } } } } ``` ## Common patterns[​](#common-patterns "Direct link to Common patterns") Stores are rarely used alone. They work together with transformers to enable analytics use cases that aren't possible with stateless event processing. | Pattern | Store | Transformer | What it enables | | ----------------------------------- | -------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------- | | **Self-hosted tag delivery** | S3 or Filesystem (`file: true`) | [File](/docs/transformers/file.md) | Serve walker.js and tracking pixels byte-exact from your own infrastructure | | **Cookie-free user identification** | Built-in cache tier | [Fingerprint](/docs/transformers/fingerprint.md) | Store session hashes server-side, identify users without cookies or PII | | **Response caching** | Built-in cache tier | [Cache](/docs/collector/cache.md) | Deduplicate identical requests, reduce load on downstream analytics APIs | | **Quota-friendly Sheets lookups** | Sheets + [cache tier](/docs/stores/cache.md) | Any transformer | Memoize slow API reads, absorb rate limits via TTL | | **Local development** | Filesystem | [File](/docs/transformers/file.md) | Serve assets from disk during development, no S3 credentials needed | ## Available stores[​](#available-stores "Direct link to Available stores") ### Filesystem[​](#filesystem "Direct link to Filesystem") Local filesystem store for server flows. Reads and writes files relative to a base path with path traversal protection. Good for local development and Docker deployments with baked-in assets. [Learn more →](/docs/stores/server/fs.md) ### S3[​](#s3 "Direct link to S3") S3-compatible object storage. Works with AWS S3, Cloudflare R2, Scaleway, DigitalOcean Spaces, Backblaze B2, MinIO, and any S3-compatible provider. The recommended store for managed cloud deployments. [Learn more →](/docs/stores/server/s3.md) ### GCS[​](#gcs "Direct link to GCS") Google Cloud Storage with zero runtime dependencies. Built-in auth supports Application Default Credentials (ADC) on Cloud Run / GKE and explicit service account JSON for non-GCP environments. The recommended store for GCP-native deployments. [Learn more →](/docs/stores/server/gcs.md) ### Sheets[​](#sheets "Direct link to Sheets") Google Sheets store with zero runtime dependencies. Row-per-key storage using the Sheets v4 REST API. Designed for demos and small prototypes where the spreadsheet is the operator-facing UI. Rate-limited; pair with the [cache tier](/docs/stores/cache.md) for any non-trivial throughput. [Learn more →](/docs/stores/server/sheets.md) ## Caching on stores[​](#caching-on-stores "Direct link to Caching on stores") Wrap any of the stores above with a read-through, write-through cache tier by setting `cache` on the store declaration. The default tier uses the collector's built-in in-memory LRU. Compose multi-tier chains via `cache.store`. [Learn more →](/docs/stores/cache.md) ## Configuration[​](#configuration "Direct link to Configuration") These fields are available on every store, regardless of package. They wrap the package-specific `settings` field, which is documented on each store's page. | Property | Type | Description | More | | ------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `settings` | `Store.Settings` | Implementation-specific configuration | | | `credentials` | `Store.Credentials` | Optional credentials (store-defined shape) | | | `env` | `Store.Env` | Environment dependencies (platform-specific) | | | `id` | `string` | Store instance identifier (defaults to store key) | | | `logger` | `Logger.Config` | | | | `setup` | `boolean \| object` | One-time setup options applied during store registration (boolean enables defaults, object configures specifics) | | | `file` | `boolean` | Persist values as raw bytes (byte-exact), bypassing the structured codec. Default false (structured). Set true only on byte-native stores for asset serving; sheets rejects it. | | --- # Store-level cache Store-level cache wraps any `Flow.Store` with a read-through, write-through cache tier. It absorbs round-trips to slow backings (Google Sheets, HTTP APIs, S3) without changing how components call the store. The wrapping is transparent: a transformer wired to `$store.crm` does not know whether reads hit a memory cache, a Redis tier, or the underlying API. The wrapper is a `Store.Instance` itself, so all the rules from [Stores](/docs/stores/.md) still apply. tip Store-level cache is different from the [event-level Cache](/docs/collector/cache.md) configured on sources, transformers, and destinations. Event cache short-circuits the pipeline on hit. Store cache memoizes `get`/`set`/`delete` calls on a single store. Both can be used in the same flow. ## Values and TTL[​](#values-and-ttl "Direct link to Values and TTL") Stores hold one canonical value type: structured data (`StoreValue`), with binary (`Uint8Array`) as a first-class leaf. A shared core serialization codec round-trips that value to and from each backing, so any store can hold structured HTTP responses, events, push results, and binary bodies. JSON-shaped data (strings, numbers, booleans, `null`, arrays, objects) and `Uint8Array` survive a round-trip exactly; a `Date` becomes a string and an `Error` loses its fields. The cache layer owns TTL, not the store. The store persists values; the cache wraps a store and manages expiry. An expired entry is treated as a miss and re-fetched from the backing. The supported cache backends are the filesystem, S3, GCS, and the in-memory tier. The Sheets store is not a cache backend: it stores a single JSON cell per key and cannot persist the cache envelope, so wiring it as a cache backing throws. A store's byte-native [`file` mode](/docs/stores/.md) (`config.file: true`) is for serving raw assets byte-exact, not for caching. The cache wraps structured stores; a store in `file` mode and a `cache` on the same store is flagged by `flow_validate`. ## Minimal example[​](#minimal-example "Direct link to Minimal example") Enable the built-in in-memory tier (`__cache`) on any store by setting `cache` on the store declaration. No extra store needed: ``` { "stores": { "sessions": { "package": "@walkeros/server-store-sheets", "config": { "settings": { "id": "1AbC...", "sheet": "Sessions" } }, "cache": { "rules": [{ "ttl": 300 }] } } } } ``` Every `sessions.get(K)` now checks the in-memory tier first. On miss, it reads from Sheets and populates the tier with a 300-second TTL. The next 300 seconds of identical reads hit memory and skip the Sheets API entirely. The same rule applies to writes: `sessions.set(K, V)` writes to Sheets first, then to the tier on a best-effort basis (see [Write-through error policy](#write-through-error-policy) below). ## Cache rule shape[​](#cache-rule-shape "Direct link to Cache rule shape") Store cache rules use a stricter subset of the rule shape used by event caches: | Field | Type | Notes | | ------- | ----------------- | ------------------------------------------------------------ | | `ttl` | `number` | **Required.** Time-to-live in seconds. | | `match` | `MatchExpression` | Optional. Matches against `{ key, value? }`, not event data. | Store rules **never** accept `key` or `update`. The cache key comes from the caller (`store.get(K)`); there is no event to transform. Rules can match on the store key or value via `match`: ``` "cache": { "rules": [ { "match": { "key": "key", "operator": "prefix", "value": "session:" }, "ttl": 300 }, { "ttl": 60 } ] } ``` First rule caches only keys starting with `session:` for five minutes. The second catches everything else for one minute. Rules evaluate top-down, first match wins. ## Multi-tier composition[​](#multi-tier-composition "Direct link to Multi-tier composition") `cache.store` references another store in the same flow. That store can have its own `cache`, and so on. The wrapper hierarchy is transparent: consumers wire to the top-level store and the wrapping resolves automatically. ``` { "stores": { "redis": { "package": "@walkeros/server-store-redis", "config": { "settings": { "url": "$env.REDIS_URL" } }, "cache": { "rules": [{ "ttl": 300 }] } }, "api": { "package": "@walkeros/server-store-api", "config": { "settings": { "endpoint": "$env.API_URL" } }, "cache": { "store": "redis", "rules": [{ "ttl": 86400 }] } } }, "transformers": { "enrich": { "package": "@walkeros/transformer-enrich", "env": { "store": "$store.api" } } } } ``` The lookup chain on `api.get(K)`: 1. Check the `api` wrapper's tier (Redis). HIT, return. 2. On Redis MISS, fall through to the next layer. The Redis wrapper's own tier (the built-in memory `__cache`) is checked first. HIT, return and **populate Redis**. 3. On both MISS, call the underlying API. 4. The value flows back up. Each traversed tier is populated on the unwind: memory `__cache`, then Redis, then the `api` tier. Subsequent reads hit the topmost tier that has a fresh value. Omitting `cache.store` falls back to the collector's built-in `__cache` tier. There is no separate memory-store package to install. ## Built-in `__cache` tier[​](#built-in-__cache-tier "Direct link to built-in-__cache-tier") The collector ships a single shared `__cache` instance used as the default cache tier when `cache.store` is omitted. It is an in-memory LRU map: * **Entry cap:** `maxEntries: 10000` (fixed in v1). * **LRU access ordering:** reads reorder entries; least-recently-used drop first when full. * **Batched eviction:** on overflow, evict down to 80% (`8000`) in one pass. * **Active TTL sweep:** a periodic sweeper drops expired entries every 60 seconds. Each wrapped store gets an automatic namespace prefix (the store id by default) so multiple stores sharing `__cache` do not collide. Override with `cache.namespace: "myns"` for an explicit prefix. The collector logs the resolved namespace at startup, one line per wrapped store. ## Coherence model[​](#coherence-model "Direct link to Coherence model") Read this section before relying on the cache for anything correctness-sensitive. * **Read-your-writes (in-process):** yes. `wrapped.set(K, V)` populates the local cache on success, so a subsequent `wrapped.get(K)` in the same process sees `V`. * **Cross-process consistency:** eventual, bounded by the longest TTL in the chain. If memory TTL is 60s and Redis TTL is 300s, a value changed in the backing by another writer is served stale from memory for up to 60s, from Redis for up to 300s. There is no invalidation channel. * **Tier-skipping repopulation:** a MISS in tier N that HITs in tier N+1 repopulates tier N on the unwind. Subsequent reads hit tier N. Set TTLs accordingly. Use short TTLs (1-60s) for mostly-static lookups behind a fast backing; use long TTLs (minutes-hours) for cold, expensive lookups where staleness is tolerable. ## Write-through error policy[​](#write-through-error-policy "Direct link to Write-through error policy") `wrapped.set(K, V)` runs two steps with explicit failure handling: 1. **Backing first.** Await `backing.set(K, V)`. If this throws, the wrapper throws. The cache is not touched. 2. **Cache best-effort.** If the backing succeeded and a rule matches, attempt `cache.store.set(...)`. If this throws, the wrapper logs a warning and returns success. Backing is the source of truth. A failed cache write degrades performance (next read misses) but does not corrupt correctness. A failed backing write is real failure and surfaces to the caller. `wrapped.delete(K)` follows the same shape: backing first (throws on failure), then best-effort cache delete (logs on failure). A failed cache delete leaves a poisoned entry that serves stale data until TTL; the warning lets operators react. ## Single-flight deduplication[​](#single-flight-deduplication "Direct link to Single-flight deduplication") Multiple concurrent `wrapped.get(K)` calls on a cold cache produce **exactly one** backing call. The wrapper holds an in-flight promise registry keyed by the namespaced key: subsequent callers receive the same promise until the first resolves. This eliminates the thundering-herd failure mode that motivates store-level cache in the first place: 50 simultaneous events looking up the same `session:abc` key against a Sheets backend trigger one Sheets read, not 50. ## Observability[​](#observability "Direct link to Observability") Each wrapped store exposes counters via the collector telemetry channel. Per-store keys are `walkeros.store_cache..`: | Counter | Meaning | | ------------------- | -------------------------------------------------- | | `hits` | Reads served by the cache tier | | `misses` | Reads that fell through to the backing | | `populates` | Cache writes triggered by successful backing reads | | `writes` | Successful `set` calls | | `deletes` | Successful `delete` calls | | `evictions_entries` | Entries evicted because the tier hit `maxEntries` | | `evictions_ttl` | Entries evicted by the active TTL sweep | | `inflight_dedups` | Concurrent calls dedup-merged into one inflight | Counters surface through the same telemetry hook the collector uses for flow events. Wire your own logger or telemetry destination to consume them. For interactive debugging, the wrapped store instance exposes a `counters` accessor: ``` const { collector } = await startFlow({ /* ... */ }); const snapshot = collector.stores.sessions.counters; console.log(snapshot); // { hits: 412, misses: 18, populates: 18, writes: 0, deletes: 0, // evictions_entries: 0, evictions_ttl: 0, inflight_dedups: 7 } ``` ## Known limitations[​](#known-limitations "Direct link to Known limitations") * **No negative caching.** A `get(K)` that returns `undefined` from the backing is not populated. Every subsequent call for that key re-hits the backing until the value exists. Workaround: write a sentinel value on the first miss, treat it as "not present" in your transformer logic. * **No cross-process invalidation.** Writes from one process do not invalidate caches in other processes. TTL is the only mechanism. * **No `stop` field on store cache.** `stop` is an event-cache concept (halt the pipeline on hit). Store reads always fall through on miss; the field is rejected by the schema. * **No `update` field on store cache.** `update` mutates events on hit. Stores have no event to mutate. * **Renaming a store is a breaking change** to anything caching through it (`cache.store: "X"` references break). Document and migrate explicitly. ## Migration from `@walkeros/store-memory`[​](#migration-from-walkerosstore-memory "Direct link to migration-from-walkerosstore-memory") The dedicated `@walkeros/store-memory` package was removed once the built-in `__cache` reached feature parity. Migration is one-line per occurrence: * If a memory store was used **only** as a cache target (`cache.store: "memory"`): omit `cache.store`. The wrapper falls back to the built-in `__cache` automatically. * If the memory store was wired into a component's `env` for non-cache use: replace with a small inline `Map` inside the component, or use one of the persistent stores in [Stores](/docs/stores/.md). `flow_validate` rejects `package: "@walkeros/store-memory"` with the replacement instruction. ## Related[​](#related "Direct link to Related") * [Stores](/docs/stores/.md) — store interface, `$store.` wiring, lifecycle. * [Collector: Cache](/docs/collector/cache.md) — event-level cache on sources, transformers, destinations. * [Reference Syntax](/docs/guides/reference-syntax.md) — `$store.`, `$env.`, `$var.` references. --- # Filesystem [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/fs) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/fs)[ ](https://www.npmjs.com/package/@walkeros/server-store-fs) [Package](https://www.npmjs.com/package/@walkeros/server-store-fs) Local filesystem store for walkerOS server flows. Reads and writes files relative to a base directory with path traversal protection. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-store-fs ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { storeFsInit } from '@walkeros/server-store-fs'; await startFlow({ stores: { assets: { code: storeFsInit, config: { settings: { basePath: './public', }, }, }, }, }); ``` Add to your `flow.json`: ``` "stores": { "assets": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./public" } } } } ``` ## Configuration[​](#configuration "Direct link to Configuration") This store uses the standard store config wrapper (consent, data, env, id, ...). For the shared fields see [store configuration](/docs/stores/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | -------- | -------------------------------------------------------------------------------- | ---- | | `basePath*` | `string` | Root directory for file operations. All keys are resolved relative to this path. | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Read file Read an existing file and receive its raw bytes byte-exact Event ``` { "operation": "get", "key": "walker.js" } ``` Out ``` get("walker.js", "Bytes") ``` ### Write file Write creates intermediate directories automatically Event ``` { "operation": "set", "key": "js/custom/tracker.js", "value": "Buffer<(function(){...})()>" } ``` Out ``` set("js/custom/tracker.js", "Buffer<(function(){...})()>") ``` ## Structured vs file mode[​](#structured-vs-file-mode "Direct link to Structured vs file mode") The fs store has two modes, decided once at init by `config.file`: * **Structured (default):** values are structured `StoreValue` data, serialized to and from disk by the shared core codec. Use for session state, lookups, and cached responses. * **File (`file: true`):** values persist as raw bytes byte-exact. `set()` accepts a `Uint8Array` or `string` and writes it untouched; `get()` hands the exact bytes back. Use for serving assets such as walker.js, where byte fidelity matters. One store instance is exactly one mode. Set `file: true` on the store declaration to opt into byte-exact serving. ## API[​](#api "Direct link to API") ``` const file = await store.get('walker.js'); // StoreValue | undefined await store.set('data.json', { ok: true }); // structured value await store.set('walker.js', new Uint8Array([/*…*/])); // file mode: raw bytes await store.delete('old-file.txt'); // void ``` In file mode, `get()` returns the bytes as a `Uint8Array` leaf and `set()` rejects non-`Uint8Array`, non-`string` values with a clear error, so served assets stay intact byte-for-byte. ## File serving pattern[​](#file-serving-pattern "Direct link to File serving pattern") Use with the file transformer to serve static assets from the local filesystem. Set `file: true` so the store persists and returns bytes byte-exact: ``` { "stores": { "assets": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./public" }, "file": true } } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.assets" } } } } ``` A request to `/static/walker.js` reads `./public/walker.js` from disk. Omit `file` (or set it false) for a structured key-value store instead. ## Security[​](#security "Direct link to Security") * **Path traversal protection**: Requests with `..`, absolute paths, or backslash traversal are rejected and logged as warnings * **Base path scoping**: All operations are restricted to the configured `basePath` directory * **Intermediate directories**: `set()` creates parent directories automatically via `mkdir -p` ## When to use[​](#when-to-use "Direct link to When to use") | Scenario | Recommended store | | --------------------------- | ----------------------------------------------------------------- | | Local development | **Filesystem**: files on disk, no credentials needed | | Docker with baked-in assets | **Filesystem**: mount or copy files into the image | | Cloud / managed deployments | [S3](/docs/stores/server/s3.md): files in a bucket, hot-swappable | | Caching / ephemeral data | Built-in [cache tier](/docs/stores/cache.md): in-process, no I/O | --- # GCS [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/gcs) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/gcs)[ ](https://www.npmjs.com/package/@walkeros/server-store-gcs) [Package](https://www.npmjs.com/package/@walkeros/server-store-gcs) Google Cloud Storage store with zero runtime dependencies. Uses raw `fetch` against the GCS JSON API with built-in auth: Application Default Credentials (ADC) on Cloud Run / GKE, or explicit service account JWT for non-GCP environments. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-store-gcs ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { storeGcsInit } from '@walkeros/server-store-gcs'; await startFlow({ stores: { assets: { code: storeGcsInit, config: { // Omit config.credentials for ADC on Cloud Run/GKE settings: { bucket: 'my-assets', prefix: 'public', }, }, }, }, }); ``` Add to your `flow.json`: ``` "stores": { "assets": { "package": "@walkeros/server-store-gcs", "config": { "settings": { "bucket": "my-assets", "prefix": "public" } } } } ``` ## Configuration[​](#configuration "Direct link to Configuration") This store uses the standard store config wrapper (consent, data, env, id, ...). For the shared fields see [store configuration](/docs/stores/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | ------------------ | ----------------------------------------------------------------------------------------------------------- | ---- | | `bucket*` | `string` | GCS bucket name | | | `prefix` | `string` | Key prefix prepended to all store keys for scoping | | | `credentials` | `string \| object` | Service account JSON (string or object). Omit for ADC on Cloud Run/GKE (deprecated: use config.credentials) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Prefix scoping Key "walker.js" with prefix "public/" resolves to GCS path "public/walker.js" Event ``` { "operation": "get", "key": "walker.js", "settings": { "bucket": "my-assets", "prefix": "public" } } ``` Out ``` get("public/walker.js", "Bytes<...>") ``` ### Read with ADC Read object from GCS bucket using ADC (no credentials needed), bytes byte-exact Event ``` { "operation": "get", "key": "walker.js" } ``` Out ``` get("walker.js", "Bytes<(function(){...})()>") ``` ## Provisioning[​](#provisioning "Direct link to Provisioning") The package ships an idempotent `setup()` lifecycle that creates the GCS bucket described in flow config. It is invoked only by the explicit operator command: ``` walkeros setup store. ``` It never runs automatically and never alters an existing bucket. ### Setup options[​](#setup-options "Direct link to Setup options") | Option | Type | Default | Notes | | -------------- | ----------------------------------------------------- | ------------ | -------------------------------------------------------------- | | `projectId` | `string` | (resolved) | GCP project that owns the bucket. Resolution order below. | | `location` | `string` | `'EU'` | Multi-region or regional location. | | `storageClass` | `'STANDARD' \| 'NEARLINE' \| 'COLDLINE' \| 'ARCHIVE'` | `'STANDARD'` | Default object storage class. | | `versioning` | `boolean` | `false` | Object versioning. Off by default; opt in. | | `lifecycle` | `{ rule: unknown[] }` | (none) | Applied at create. Drift detection NOT included for lifecycle. | | `kmsKeyName` | `string` | (none) | Customer-managed encryption key (CMEK) at create time. | | `labels` | `Record` | (none) | Cost-allocation labels. | `bucket` is taken from `settings.bucket` and is NOT duplicated under `setup`. ### projectId resolution[​](#projectid-resolution "Direct link to projectId resolution") The GCS create call requires a project. Resolution order: 1. Explicit `setup.projectId`. 2. `project_id` field inside the `config.credentials` service-account JSON. 3. `process.env.GOOGLE_CLOUD_PROJECT` (Cloud Run / GKE convention). 4. Throws with an actionable error if none of the above is available. ### Behavior[​](#behavior "Direct link to Behavior") * **Idempotent**: HTTP 409 (bucket exists) is treated as success. The setup never patches or mutates an existing bucket. * **Drift detection**: when the bucket already exists, setup performs a `GET /b/` and logs `WARN setup.drift { field, declared, actual }` for any of `location`, `storageClass`, `versioning`, `iamConfiguration` (uniform bucket-level access, public access prevention), and `labels` that do not match. Drift is logged, never auto-fixed. * **Defaults enforced at create**: uniform bucket-level access on, public access prevention enforced. These are baked in by the package. ### Runtime hard-fail[​](#runtime-hard-fail "Direct link to Runtime hard-fail") At runtime, the first `get` / `set` / `delete` call issues a single `HEAD /b/` per process per bucket. On 404, it throws with an actionable message: ``` GCS bucket not found: in project . Run "walkeros setup store." to create it. ``` Operators see the error pointing at the exact command to fix it. Subsequent operations in the same process skip the check via an in-memory cache. ## Authentication[​](#authentication "Direct link to Authentication") ### Cloud Run / GKE (ADC)[​](#cloud-run--gke-adc "Direct link to Cloud Run / GKE (ADC)") When running on GCP infrastructure, omit `credentials`. The store fetches access tokens from the metadata server automatically, with no configuration needed. ### Non-GCP (service account)[​](#non-gcp-service-account "Direct link to Non-GCP (service account)") Pass a service account JSON via `config.credentials` with a `$env.` reference. The store signs JWTs locally using `node:crypto` and exchanges them for access tokens via Google's OAuth2 endpoint. ``` "config": { "credentials": "$env.GCS_SA_KEY", "settings": { "bucket": "my-assets", "prefix": "public" } } ``` The `GCS_SA_KEY` environment variable should contain the full service account JSON (with `client_email` and `private_key` fields). ## File serving pattern[​](#file-serving-pattern "Direct link to File serving pattern") The primary use case is serving static files via the file transformer. Set `file: true` so the store persists and returns bytes byte-exact: ``` { "stores": { "assets": { "package": "@walkeros/server-store-gcs", "config": { "settings": { "bucket": "my-assets", "prefix": "public" }, "file": true } } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.assets" } } } } ``` A request to `/static/walker.js` looks up `public/walker.js` in the `my-assets` bucket. Omit `file` for a structured key-value store instead. ## Security[​](#security "Direct link to Security") * **Key validation**: Path traversal attempts (`..`, absolute paths) are rejected * **Prefix scoping**: The `prefix` setting restricts all operations to a subdirectory * **No ambient credentials**: ADC only works on GCP infrastructure; off-GCP requires explicit SA JSON ## Structured vs file mode[​](#structured-vs-file-mode "Direct link to Structured vs file mode") The GCS store has two modes, decided once at init by `config.file`: * **Structured (default):** values are structured `StoreValue` data, serialized by the shared core codec and stored with `Content-Type: application/json`. Use for session state, lookups, and cached responses. * **File (`file: true`):** values persist as raw bytes byte-exact, stored with the real mime derived from the key (or `application/octet-stream` when unknown). `set()` accepts a `Uint8Array` or `string`; `get()` hands the exact bytes back. Use for serving assets such as walker.js. One store instance is exactly one mode. ## API[​](#api "Direct link to API") ``` const file = await store.get('walker.js'); // StoreValue | undefined await store.set('data.json', { ok: true }); // structured value await store.set('walker.js', new Uint8Array([/*…*/])); // file mode: raw bytes await store.delete('old-file.txt'); // void ``` In file mode, `get()` returns the bytes as a `Uint8Array` leaf and `set()` rejects non-`Uint8Array`, non-`string` values with a clear error. --- # S3 [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/s3) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/s3)[ ](https://www.npmjs.com/package/@walkeros/server-store-s3) [Package](https://www.npmjs.com/package/@walkeros/server-store-s3) S3-compatible object storage store using [s3mini](https://github.com/good-lly/s3mini) (\~20 KB, zero dependencies). Works with AWS S3, Cloudflare R2, Scaleway, DigitalOcean Spaces, Backblaze B2, MinIO, and any S3-compatible provider. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-store-s3 ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { storeS3Init } from '@walkeros/server-store-s3'; await startFlow({ stores: { assets: { code: storeS3Init, config: { settings: { bucket: 'my-assets', endpoint: 'https://s3.eu-west-1.amazonaws.com', accessKeyId: process.env.S3_ACCESS_KEY, secretAccessKey: process.env.S3_SECRET_KEY, region: 'eu-west-1', prefix: 'public', }, }, }, }, }); ``` Add to your `flow.json`: ``` "stores": { "assets": { "package": "@walkeros/server-store-s3", "config": { "settings": { "bucket": "my-assets", "endpoint": "https://s3.eu-west-1.amazonaws.com", "accessKeyId": "$env.S3_ACCESS_KEY", "secretAccessKey": "$env.S3_SECRET_KEY", "region": "eu-west-1", "prefix": "public" } } } } ``` ## Configuration[​](#configuration "Direct link to Configuration") This store uses the standard store config wrapper (consent, data, env, id, ...). For the shared fields see [store configuration](/docs/stores/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------------ | -------- | -------------------------------------------------- | ---- | | `bucket*` | `string` | S3 bucket name | | | `endpoint*` | `string` | S3-compatible endpoint URL | | | `accessKeyId*` | `string` | S3 access key ID | | | `secretAccessKey*` | `string` | S3 secret access key | | | `region` | `string` | AWS region for SigV4 signing | | | `prefix` | `string` | Key prefix prepended to all store keys for scoping | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Prefix scoping Key "walker.js" with prefix "public/" resolves to S3 path "public/walker.js" Event ``` { "operation": "get", "key": "walker.js", "settings": { "bucket": "my-assets", "prefix": "public" } } ``` Out ``` get("public/walker.js", "Bytes<...>") ``` ### Read from S3 Read object from S3 and receive its raw bytes byte-exact Event ``` { "operation": "get", "key": "walker.js" } ``` Out ``` get("walker.js", "Bytes<(function(){...})()>") ``` ## Provider examples[​](#provider-examples "Direct link to Provider examples") | Provider | Endpoint | Notes | | ------------- | -------------------------------------------- | ---------------------------------- | | AWS S3 | `https://s3..amazonaws.com` | Set `region` to your actual region | | Cloudflare R2 | `https://.r2.cloudflarestorage.com` | No egress fees | | Scaleway | `https://s3..scw.cloud` | EU hosting | | DigitalOcean | `https://.digitaloceanspaces.com` | Simple pricing | | Backblaze B2 | `https://s3..backblazeb2.com` | Cheapest storage | | MinIO | `http://localhost:9000` | Self-hosted | ## Credentials[​](#credentials "Direct link to Credentials") Use `$env.` references in your flow config to avoid hardcoding secrets: ``` "accessKeyId": "$env.S3_ACCESS_KEY", "secretAccessKey": "$env.S3_SECRET_KEY" ``` Unlike the AWS SDK, `s3mini` has no implicit credential chain: `accessKeyId` and `secretAccessKey` are always required. ## File serving pattern[​](#file-serving-pattern "Direct link to File serving pattern") The primary use case is serving static files via the file transformer. This is the recommended pattern for managed deployments (Mode D) where files live in a bucket rather than needing to be baked into a Docker image. Set `file: true` so the store persists and returns bytes byte-exact: ``` { "stores": { "assets": { "package": "@walkeros/server-store-s3", "config": { "settings": { "bucket": "my-assets", "endpoint": "https://s3.eu-west-1.amazonaws.com", "accessKeyId": "$env.S3_ACCESS_KEY", "secretAccessKey": "$env.S3_SECRET_KEY", "prefix": "public" }, "file": true } } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.assets" } } } } ``` A request to `/static/walker.js` looks up `public/walker.js` in the `my-assets` bucket. Omit `file` for a structured key-value store instead. ## Provisioning[​](#provisioning "Direct link to Provisioning") The package ships an idempotent `setup()` lifecycle, invoked only by the explicit operator command: ``` walkeros setup store. ``` It never runs automatically. It checks whether the bucket exists and creates it if not. Re-running setup is a no-op when the bucket already exists in your account. ### Setup options[​](#setup-options "Direct link to Setup options") | Option | Type | Default | Notes | | -------- | -------- | -------------- | ----------------------------------------------------------------------------------------------------------------- | | `region` | `string` | `eu-central-1` | Region the bucket is created in (LocationConstraint). Falls back to `settings.region` when concrete (not `auto`). | `bucket` is taken from `settings.bucket` and is NOT duplicated under `setup`. ### Enable provisioning[​](#enable-provisioning "Direct link to Enable provisioning") Set `setup: true` in the component config to enable provisioning with defaults, or pass an object to override: ``` { "stores": { "assets": { "package": "@walkeros/server-store-s3", "config": { "settings": { "bucket": "my-assets", "endpoint": "https://s3.eu-central-1.amazonaws.com", "accessKeyId": "$env.S3_ACCESS_KEY", "secretAccessKey": "$env.S3_SECRET_KEY", "region": "eu-central-1" }, "setup": true } } } } ``` ### What setup does NOT apply[​](#what-setup-does-not-apply "Direct link to What setup does NOT apply") `s3mini` is a minimal S3 client that exposes `createBucket` and `bucketExists` only. It does NOT expose `PutBucketEncryption`, `PutPublicAccessBlock`, `PutBucketVersioning`, `PutBucketLifecycleConfiguration`, or `PutBucketTagging`. Configure those once via the AWS Console or `aws s3api`. ### Behavior[​](#behavior "Direct link to Behavior") * **Idempotent create:** `BucketAlreadyOwnedByYou` (concurrent caller, your account) is treated as success and returns `{ bucketCreated: false }`. `BucketAlreadyExists` (different AWS account owns the global name) fails loud with an actionable message so you pick a different name. * **Region resolution:** explicit `setup.region` wins; otherwise `settings.region` is used when concrete (not `auto`); otherwise the EU default `eu-central-1`. ### Runtime hard-fail[​](#runtime-hard-fail "Direct link to Runtime hard-fail") `storeS3Init` probes `bucketExists()` once when the collector wires the store. On a missing bucket it throws with an actionable message: ``` S3 bucket not found: my-assets at https://s3.eu-central-1.amazonaws.com. Run "walkeros setup store.assets" to create it. ``` Run `walkeros setup store.` once to provision the bucket, then redeploy. ## Security[​](#security "Direct link to Security") * **Key validation**: Path traversal attempts (`..`, absolute paths) are rejected * **Prefix scoping**: The `prefix` setting restricts all operations to a subdirectory * **No credential chain**: Credentials must be explicitly provided (no ambient AWS credentials) ## Structured vs file mode[​](#structured-vs-file-mode "Direct link to Structured vs file mode") The S3 store has two modes, decided once at init by `config.file`: * **Structured (default):** values are structured `StoreValue` data, serialized by the shared core codec and stored with `Content-Type: application/json`. Use for session state, lookups, and cached responses. * **File (`file: true`):** values persist as raw bytes byte-exact, stored with the real mime derived from the key (or `application/octet-stream` when unknown). `set()` accepts a `Uint8Array` or `string`; `get()` hands the exact bytes back. Use for serving assets such as walker.js. One store instance is exactly one mode. ## API[​](#api "Direct link to API") ``` const file = await store.get('walker.js'); // StoreValue | undefined await store.set('data.json', { ok: true }); // structured value await store.set('walker.js', new Uint8Array([/*…*/])); // file mode: raw bytes await store.delete('old-file.txt'); // void ``` In file mode, `get()` returns the bytes as a `Uint8Array` leaf and `set()` rejects non-`Uint8Array`, non-`string` values with a clear error. --- # Sheets [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/sheets) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/stores/sheets)[ ](https://www.npmjs.com/package/@walkeros/server-store-sheets) [Package](https://www.npmjs.com/package/@walkeros/server-store-sheets) Google Sheets store with zero runtime dependencies. Uses raw `fetch` against the Sheets v4 REST API with built-in auth: Application Default Credentials (ADC) on Cloud Run / GKE, or explicit service account JWT for non-GCP environments. Designed for demos and small-scale prototyping where the spreadsheet itself is the operator-facing UI for tweaking lookup data. Not a production CRM substitute. ## Caveats and quotas (read first)[​](#caveats-and-quotas-read-first "Direct link to Caveats and quotas (read first)") The Sheets API is rate-limited and slow. Wiring this store directly into a high-throughput pipeline burns quota in seconds. * **Quota:** 60 read requests / minute / user / project, 60 write requests / minute / user / project. * **Latency:** 200 to 800 ms per HTTP round-trip. * **No internal cache:** the package does NOT cache reads. Caching is the consumer's responsibility, see "Wiring with the core cache" below. * **Concurrency:** last writer wins on the same cell. There is no transactional `getAndSet`. * **Single-writer model:** if two pipeline instances both write to the same sheet, their `keyToRow` indexes diverge. * **Demo and small-prototype grade only.** Not a production CRM substitute. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-store-sheets ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { storeSheetsInit } from '@walkeros/server-store-sheets'; await startFlow({ stores: { crm: { code: storeSheetsInit, config: { // Omit config.credentials for ADC on Cloud Run/GKE settings: { id: '1AbCdEfGhIjKlMnOpQrStUvWxYz', sheet: 'Customers', }, }, }, }, }); ``` Add to your `flow.json`: ``` "stores": { "crm": { "package": "@walkeros/server-store-sheets", "config": { "settings": { "id": "1AbCdEfGhIjKlMnOpQrStUvWxYz", "sheet": "Customers" } } } } ``` ## Configuration[​](#configuration "Direct link to Configuration") This store uses the standard store config wrapper (consent, data, env, id, ...). For the shared fields see [store configuration](/docs/stores/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ------------- | ------------------ | ----------------------------------------------------------------------------------------------------------- | ---- | | `id*` | `string` | Spreadsheet ID, the segment between /d/ and /edit in the URL | | | `sheet` | `string` | Sheet (tab) name within the spreadsheet | | | `key` | `string` | Column letter for keys (the lookup column) | | | `value` | `string` | Column letter for values (JSON-serialized blob) | | | `headerRows` | `integer` | Number of header rows to skip when reading the key column | | | `credentials` | `string \| object` | Service account JSON (string or object). Omit for ADC on Cloud Run/GKE (deprecated: use config.credentials) | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Read with ADC Read a value from the Sheets store using ADC, no credentials field needed on Cloud Run or GKE Event ``` { "operation": "get", "key": "alice" } ``` Out ``` get("alice", "{ tier: \"gold\" }") ``` ### Write with service account Append or update a JSON value in the configured sheet using an explicit service account JSON Event ``` { "operation": "set", "key": "bob", "value": { "tier": "silver" } } ``` Out ``` set("bob", "{ tier: \"silver\" }") ``` ## Wiring with the built-in cache[​](#wiring-with-the-built-in-cache "Direct link to Wiring with the built-in cache") To absorb the Sheets quota, wire the store with the built-in cache via `Flow.Store.cache`. The cache is on by default with an in-memory tier and requires no extra store declaration: ``` { "stores": { "crm": { "package": "@walkeros/server-store-sheets", "config": { "settings": { "id": "1AbCdEfGhIjKlMnOpQrStUvWxYz", "sheet": "Customers" } }, "cache": { "rules": [{ "ttl": 60 }] } } }, "transformers": { "enrich": { "package": "@walkeros/transformer-enrich", "env": { "store": "$store.crm" } } } } ``` Use `ttl: 5000` for demos (fast iteration), `ttl: 60000` or higher for production-ish loads. Without a cache, every event hits Sheets directly and trips the 60 req/min quota in under one second. The cache config lives at `packages/core/src/types/cache.ts`. See the [`understanding-stores`](https://github.com/elbwalker/walkerOS/tree/main/skills/understanding-stores) skill for the broader store/cache pattern. ## Provisioning[​](#provisioning "Direct link to Provisioning") The package ships an idempotent `setup()` lifecycle, invoked only by the explicit operator command: ``` walkeros setup store. ``` It never runs automatically. It verifies the spreadsheet exists and (if configured) writes the `setup.headers` row. ### Setup options[​](#setup-options "Direct link to Setup options") | Option | Type | Default | Notes | | --------- | ---------- | ------- | ---------------------------------------------------------------------------------------------- | | `headers` | `string[]` | (none) | Header values written to row 1 of the configured sheet. Idempotent overwrite, no drift detect. | `id` is taken from `settings.id` and is NOT duplicated under `setup`. ### Behavior[​](#behavior "Direct link to Behavior") * **Existence probe:** setup issues `GET /spreadsheets/?fields=spreadsheetId` and throws an actionable error on 404. * **Header write:** when `setup.headers` is provided, setup issues `PUT /values/!A1:1?valueInputOption=RAW` with the headers as the row values. Re-running with the same headers is a no-op overwrite. * **No `shareWith`:** Drive API integration is intentionally out of scope in this version (it requires a separate OAuth scope). Share the spreadsheet manually with the service account email before running setup. ### Runtime hard-fail[​](#runtime-hard-fail "Direct link to Runtime hard-fail") The first call to `init()` issues a single `GET /spreadsheets/?fields=spreadsheetId` per process per spreadsheet ID. On 404 it throws with an actionable message: ``` Spreadsheet not found: . Run "walkeros setup store." to ensure the sheet exists and is shared with the service account. ``` Operators see the error pointing at the exact command to fix it. Subsequent calls in the same process skip the check via an in-memory cache. ## Authentication[​](#authentication "Direct link to Authentication") ### Cloud Run / GKE (ADC)[​](#cloud-run--gke-adc "Direct link to Cloud Run / GKE (ADC)") When running on GCP infrastructure, omit `credentials`. The store fetches access tokens from the metadata server automatically. Required OAuth scope: `https://www.googleapis.com/auth/spreadsheets`. ### Non-GCP (service account)[​](#non-gcp-service-account "Direct link to Non-GCP (service account)") Pass a service account JSON via `config.credentials` with a `$env.` reference. The store signs JWTs locally using `node:crypto` and exchanges them for access tokens via Google's OAuth2 endpoint. ``` "config": { "credentials": "$env.SHEETS_SA_KEY", "settings": { "id": "...", "sheet": "Customers" } } ``` The `SHEETS_SA_KEY` environment variable should contain the full service account JSON (with `client_email` and `private_key` fields). Share the spreadsheet with the service account email before running setup. ## Structured only[​](#structured-only "Direct link to Structured only") The Sheets store is structured-only. Cells hold structured `StoreValue` JSON; there is no byte-native mode. Setting `file: true` is a hard error at init (use fs, S3, or GCS for byte-exact serving), and a value containing a binary (`Uint8Array`) leaf is rejected, since a spreadsheet cell cannot hold raw bytes. ## API[​](#api "Direct link to API") ``` const value = await store.get('alice'); // StoreValue | undefined (parsed JSON) await store.set('bob', { tier: 'silver' }); // void await store.delete('charlie'); // void ``` Each value is JSON-stringified into one cell (the `value` column). Reads JSON-parse the cell back. A non-parseable cell logs a debug line and returns `undefined`. `set()` for an unknown key appends a new row, capturing the row index from the API response. `set()` for a known key updates the existing value cell. `delete()` blanks the value cell, the row stays in place to keep `keyToRow` indexes stable. ## Limitations[​](#limitations "Direct link to Limitations") * **Single-cell value shape.** Multi-column structured rows are out of scope, ship a richer schema in a later phase if customers ask. * **No drift detection on header content.** If an operator manually edits row 1, the next `walkeros setup store.` overwrites it without warning. * **No transactional updates.** `set` is two HTTP calls (read index, write cell). Concurrent writers can interleave. * **Not a request-cache backend.** The Sheets store is a key-value state store, not a request-cache backend. It stores a single JSON cell per key and cannot persist the encoded request-cache value, so wiring it as a request cache throws. Cache structured or binary HTTP responses in a filesystem, S3, GCS, or in-memory store instead. --- # Transformers Transformers are middleware for **validating**, **enriching**, and **redacting** events in the walkerOS pipeline. Together with [Mapping](/docs/mapping/.md), they give you full control over how events are shaped before reaching destinations. A transformer step can either run custom logic (via `code` or `package`) or act as a **pass-through step**: a step with no `code` and no `package`, where the runtime synthesizes the push. Pass-through steps cover three patterns: chain-only hops (`before` / `next`), cache-only checks (`cache`), and mapping-only mutations (`mapping`). See [Create your own](/docs/transformers/create-your-own.md#pass-through-steps) for the full vocabulary. ## Use cases[​](#use-cases "Direct link to Use cases") | Use Case | Example | | ------------ | ----------------------------------------- | | **Validate** | Ensure events match JSON Schema contracts | | **Enrich** | Add server-side data (user segments, geo) | | **Redact** | Remove PII before sending to destinations | ## Basic setup[​](#basic-setup "Direct link to Basic setup") Add a transformer to the `transformers` block of your flow and wire it into the pipeline via a source's `next` or a destination's `before`. ``` import { startFlow } from '@walkeros/collector'; import { transformerFingerprint } from '@walkeros/server-transformer-fingerprint'; await startFlow({ transformers: { fingerprint: { code: transformerFingerprint, config: { settings: { /* ... */ } } } } }); ``` ## Configuration[​](#configuration "Direct link to Configuration") These fields are available on every transformer, regardless of package. They wrap the package-specific `settings` field, which is documented on each transformer's page. | Property | Type | Description | More | | ------------ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `settings` | `Transformer.Settings` | Implementation-specific configuration | | | `env` | `Transformer.Env` | Environment dependencies (platform-specific) | | | `id` | `string` | Transformer instance identifier (defaults to transformer key) | | | `logger` | `Logger.Config` | | | | `before` | `Route` | Pre-transformer chain that runs before this transformer pushes | | | `next` | `Route` | Graph wiring to the next transformer in the chain | | | `cache` | `EventCache.Config` | Step-level cache configuration for this transformer | | | `state` | `State.Config \| State.Config[]` | Declarative store get/set operations applied around this transformer | | | `init` | `boolean` | Whether to initialize immediately | | | `disabled` | `boolean` | Completely skip this transformer in chains | | | `mock` | `Transformer.Mock` | Return this value instead of calling push(). Global mock for all chains. Dev/testing only. | | | `chainMocks` | `Record` | Path-specific mock values keyed by chain path. Takes precedence over global mock. Dev/testing only. | | | `mapping` | `Mapping.Config` | Declarative event-to-event mapping applied when this transformer step has no code. At this position, only event-mutating fields apply (policy, mapping\[].policy, mapping\[].name, mapping\[].ignore, mapping\[].consent, include); vendor-payload fields are ignored. | | Beyond `code`, `package`, `before`, `next`, and `cache`, transformer entries also accept a `mapping` field that takes a `Mapping.Config` value, and a [`state`](/docs/collector/state.md) field for declarative store get/set. Dual semantic of `mapping` The `mapping` field uses the same `Mapping.Config` shape on destinations and transformer steps, but the semantic differs by position. On a destination, `mapping` shapes the vendor payload. On a transformer step, it mutates the event itself. Vendor-payload fields (`data`, per-rule `data`, `silent`) are ignored at the transformer position with a one-time warning at init. Closed schema Transformer entries use a **closed schema**: unknown top-level keys are errors. This catches typos like placing `rules` or `stop` at the top of a step (forgot the `cache:` wrapper). ## Next steps[​](#next-steps "Direct link to Next steps") * **[Cache](/docs/collector/cache.md)** - Cache pure handler results (integrated collector mode) * **[State](/docs/collector/state.md)** - Declarative store get/set without `$code:` * **[Create your own](/docs/transformers/create-your-own.md)** - Custom transformer guide --- # Bot detection [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/bot) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/bot)[ ](https://www.npmjs.com/package/@walkeros/server-transformer-bot) [Package](https://www.npmjs.com/package/@walkeros/server-transformer-bot) Annotates events with `user.botScore` (0-99, higher = more bot) and `user.agentScore` (0-99, higher = more AI agent). Optionally writes `user.agentProduct` (matched UA substring). Never drops events — destinations filter via mapping. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-transformer-bot ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { transformerBot } from '@walkeros/server-transformer-bot'; await startFlow({ transformers: { bot: { code: transformerBot }, }, }); ``` ``` "transformers": { "bot": { "package": "@walkeros/server-transformer-bot" } } ``` ## Configuration[​](#configuration "Direct link to Configuration") This transformer uses the standard transformer config wrapper (consent, data, env, id, ...). For the shared fields see [transformer configuration](/docs/transformers/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `input` | `input` | Input signal sources, resolved via getMappingValue against { event, ingest }. v1 only reads userAgent; other fields reserved for v1.1 header heuristics. | | | `userAgent` | `any \| array` | | | | `ip` | `any \| array` | | | | `acceptLanguage` | `any \| array` | | | | `acceptEncoding` | `any \| array` | | | | `secFetchSite` | `any \| array` | | | | `secFetchMode` | `any \| array` | | | | `secFetchDest` | `any \| array` | | | | `secFetchUser` | `any \| array` | | | | `secChUa` | `any \| array` | | | | `secChUaMobile` | `any \| array` | | | | `secChUaPlatform` | `any \| array` | | | | `output` | `output` | Output paths for bot/agent annotations. | | | `botScore` | `string` | Path for bot score (0-99, higher = more bot). Default: "user.botScore". Use "ingest.\*" to route to pipeline scratch instead of the event. Empty string or omit = skip. | | | `agentScore` | `string` | Path for AI agent score (0-99). v1 emits 0 (no match) or 95 (UA-map match). Default: "user.agentScore". | | | `agentProduct` | `string` | Path for matched UA substring (e.g. "ChatGPT-User"). Off by default — set to enable. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### ChatGPT-User (user-action AI) A real human routed an AI to fetch this page. botScore high but lower than crawlers — agentProduct lets destinations keep this traffic. Event ``` { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000602", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } ``` Out ``` return { "event": { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000602", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" }, "user": { "botScore": 90, "agentScore": 95 } } } ``` ### GPTBot training crawler OpenAI training crawler. Both botScore and agentScore are high. Event ``` { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000601", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } ``` Out ``` return { "event": { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000601", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" }, "user": { "botScore": 95, "agentScore": 95 } } } ``` ### Human visitor (Chrome) Modern Chrome UA. No bot or agent signals. Event ``` { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000600", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } ``` Out ``` return { "event": { "name": "page view", "data": { "title": "Home", "id": "/" }, "id": "ev-1700000600", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" }, "user": { "botScore": 0, "agentScore": 0 } } } ``` ## Source prerequisite[​](#source-prerequisite "Direct link to Source prerequisite") The transformer reads `userAgent` from `ctx.ingest` (default path `ingest.userAgent`). The upstream server source must populate it via `config.ingest`. Without it, every event scores 70 (missing-UA baseline). ``` { "sources": { "express": { "package": "@walkeros/server-source-express", "config": { "ingest": { "map": { "userAgent": { "key": "headers.user-agent" } } } } } } } ``` ## Detection layers (v1)[​](#detection-layers-v1 "Direct link to Detection layers (v1)") * **isbot** — catches curl, wget, python-requests, headless Chrome defaults, well-known crawlers. * **Curated AI agent map** — vendor self-declared UAs across OpenAI (GPTBot, ChatGPT-User, ChatGPT-Agent, OAI-SearchBot), Anthropic (ClaudeBot, Claude-User, Claude-SearchBot, Claude-Code, legacy anthropic-ai), Perplexity, Mistral, Meta (Meta-ExternalAgent, Meta-ExternalFetcher), Google (Google-CloudVertexBot, Google-Extended), Apple (Applebot-Extended), Amazon (Amazonbot), DuckDuckGo (DuckAssistBot), ByteDance (Bytespider), Common Crawl (CCBot). ## Destination filtering recipes[​](#destination-filtering-recipes "Direct link to Destination filtering recipes") Drop all bots: `event.user.botScore > 50` Drop crawlers, keep user-action AI: `event.user.botScore > 50 AND event.user.agentProduct NOT LIKE '%-User'` AI traffic report: `event.user.agentScore > 50`, grouped by `event.user.agentProduct` ## Not in v1[​](#not-in-v1 "Direct link to Not in v1") Header consistency heuristics (Sec-Fetch / Sec-CH-UA / Accept-Language), ASN / datacenter-IP, reverse DNS verification, web-side runtime checks, behavioral signals, TLS / JA4. See the README's "Not in v1" section for the full roadmap. ## Limits[​](#limits "Direct link to Limits") Will not catch residential-proxy + stealth Chrome, CAPTCHA-solver farms, or real-browser-as-a-service. For that threat model use a commercial vendor (Cloudflare Bot Management, DataDome, HUMAN). --- # Create your own transformer Build custom transformers for validation, enrichment, or redaction. ## The transformer interface[​](#the-transformer-interface "Direct link to The transformer interface") Transformers are functions that receive a context object and return a transformer instance: ``` type Transformer.Init = ( context: Transformer.Context, ) => Transformer.Instance; ``` The context contains: ``` interface Transformer.Context { config: Partial>; // Settings, mapping, next chain env: Types['env']; // Environment dependencies logger: Logger; // Logging functions id: string; // Transformer identifier collector: Collector.Instance; // Collector reference ingest?: Ingest; // Request metadata from source } ``` The returned instance must implement: ``` interface Transformer.Instance { type: string; // Unique identifier config: Transformer.Config; push(event, context): Transformer.Result | false | void; } ``` ## Return values[​](#return-values "Direct link to Return values") | Return | Behavior | | -------------------- | -------------------------------------------- | | `{ event }` | Continue chain with modified event | | `void` | Continue chain, event unchanged | | `false` | Stop chain, event dropped | | `{ event, next }` | Redirect to a different chain (fan-out) | | `{ event, respond }` | Continue chain with wrapped respond function | ## Minimal example[​](#minimal-example "Direct link to Minimal example") ``` import type { Transformer } from '@walkeros/core'; interface Settings { removeFields?: string[]; } interface Types extends Transformer.Types {} export const transformerRedact: Transformer.Init = (context) => { // Destructure what you need from context const { config = {} } = context; const settings = config.settings || {}; return { type: 'redact', config, push(event, pushContext) { const { logger } = pushContext; for (const field of settings.removeFields || []) { if (event.data?.[field]) { delete event.data[field]; logger.debug('Redacted field', { field }); } } return { event }; } }; }; ``` ## Push context[​](#push-context "Direct link to Push context") The `push` function receives an event and a push context: * **`config`**: Transformer configuration * **`env`**: Environment dependencies * **`logger`**: Scoped logger for output * **`id`**: Transformer identifier * **`collector`**: Access to collector instance * **`ingest`**: Request metadata from source (optional) ``` push(event, context) { const { logger, id, ingest } = context; logger.debug('Processing event', { transformer: id, event: event.name }); // Access request metadata if available if (ingest?.ip) { event.data = { ...event.data, clientIp: ingest.ip }; } return { event }; } ``` ## Examples[​](#examples "Direct link to Examples") ### Enrich: add server-side data[​](#enrich-add-server-side-data "Direct link to Enrich: add server-side data") ``` push(event, context) { event.data = { ...event.data, enrichedAt: Date.now(), region: 'eu-west-1', }; return { event }; } ``` ### Validate: custom check[​](#validate-custom-check "Direct link to Validate: custom check") ``` push(event, context) { if (!event.data?.id) { context.logger.error('Missing required id'); return false; // Stop chain } return { event }; } ``` ## Using your transformer[​](#using-your-transformer "Direct link to Using your transformer") ``` import { startFlow } from '@walkeros/collector'; import { transformerRedact } from './transformerRedact'; await startFlow({ transformers: { redact: { code: transformerRedact, config: { settings: { removeFields: ['email', 'phone'] } } } } }); ``` ## Pass-through steps[​](#pass-through-steps "Direct link to Pass-through steps") A **path** is the multi-step chain through a flow's transformer section. A **pass-through step** (short: **pass**) is a single step within a path that has no `code` and no `package`. The runtime synthesizes its push function for you, so the step exists purely to wire chain hops, run a cache check, or apply a declarative mapping. Three variants exist: ### 1. Before/next chain only[​](#1-beforenext-chain-only "Direct link to 1. Before/next chain only") A named hop in the chain. Useful for reusing a chain prefix across destinations. ``` "enrichServer": { "before": ["filterBots", "sessionLookup"] } ``` ### 2. Cache only[​](#2-cache-only "Direct link to 2. Cache only") A step that runs a cache check, often for dedup or response caching. ``` "dedup": { "cache": { "stop": true, "rules": [{ "key": ["event.id"], "ttl": 60 }] } } ``` ### 3. Mapping only[​](#3-mapping-only "Direct link to 3. Mapping only") A step that runs a declarative event-to-event mapping using `Mapping.Config` primitives (`policy`, per-rule `policy`, `mapping[].name`, `mapping[].ignore`). ``` "redactPII": { "mapping": { "policy": { "user.email": { "value": "[redacted]" } } } } ``` Dual semantic of `mapping` The `mapping` field uses the same `Mapping.Config` shape on destinations and transformer steps, but the semantic differs by position. On a destination, `mapping` shapes the vendor payload. On a transformer step, it mutates the event itself. Vendor-payload fields (`data`, per-rule `data`, `silent`) are ignored at the transformer position with a one-time warning at init. Closed schema Transformer entries use a **closed schema**: unknown top-level keys are errors. This catches typos like `{ "rules": [], "stop": true }` at the top of a step (forgot the `cache:` wrapper). ## Testing[​](#testing "Direct link to Testing") ``` import { transformerRedact } from '../index'; import type { Transformer, Collector } from '@walkeros/core'; import { createMockLogger } from '@walkeros/core'; // Helper to create transformer context for testing function createTransformerContext( config: Partial> = {}, ): Transformer.Context { return { config, env: {} as Types['env'], logger: createMockLogger(), id: 'test-redact', collector: {} as Collector.Instance, }; } // Helper to create push context for testing function createPushContext(): Transformer.Context { return { config: {}, env: {} as Types['env'], logger: createMockLogger(), id: 'test-redact', collector: {} as Collector.Instance, }; } describe('Redact Transformer', () => { it('removes specified fields', () => { const transformer = transformerRedact( createTransformerContext({ settings: { removeFields: ['email'] } }) ); const event = { data: { id: '123', email: 'test@example.com' } }; const pushContext = createPushContext(); const result = transformer.push(event, pushContext); expect(result.event.data.email).toBeUndefined(); expect(result.event.data.id).toBe('123'); }); }); ``` ## Package convention[​](#package-convention "Direct link to Package convention") Every walkerOS package includes machine-readable metadata for tooling and discovery. ### `walkerOS` field in package.json[​](#walkeros-field-in-packagejson "Direct link to walkeros-field-in-packagejson") ``` { "walkerOS": { "type": "transformer" }, "keywords": ["walkeros", "walkeros-transformer"] } ``` | Field | Required | Description | | ---------- | -------- | --------------------------------------------------------------------- | | `walkerOS` | Yes | Object with `type` metadata (and `platform` for sources/destinations) | ### Build-time generation[​](#build-time-generation "Direct link to Build-time generation") Use `buildDev()` from the shared tsup config to auto-generate `walkerOS.json`: ``` import { buildDev } from '@walkeros/config/tsup'; ``` This file contains your package's JSON Schemas and examples, enabling MCP tools and the CLI to validate configurations without installing your package. ### Optional: Hints[​](#optional-hints "Direct link to Optional: Hints") Packages can export a `hints` record from `src/dev.ts` to provide lightweight, actionable context beyond schemas and examples, such as validation behavior, enrichment patterns, or troubleshooting tips. Hints are serialized into `walkerOS.json` and surfaced via MCP tools. See the `walkeros-create-transformer` skill for details. ## Publishing checklist[​](#publishing-checklist "Direct link to Publishing checklist") * [ ] `walkerOS` field in package.json * Keywords include `walkeros` and `walkeros-transformer` * [ ] `buildDev()` in tsup.config.ts * [ ] `dist/walkerOS.json` generated on build * [ ] `npm run test` passes * [ ] `npm run lint` passes ## Next steps[​](#next-steps "Direct link to Next steps") * Learn about [creating sources](/docs/sources/create-your-own.md) * Learn about [creating destinations](/docs/destinations/create-your-own.md) --- # File [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/file) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/file)[ ](https://www.npmjs.com/package/@walkeros/server-transformer-file) [Package](https://www.npmjs.com/package/@walkeros/server-transformer-file) Serves static files from a [store](/docs/getting-started/stores.md) backend. Reads the file by `ingest.path`, derives the Content-Type from the file extension, and responds directly. Returns `false` to stop the transformer chain so no further transformers or destinations run for the request. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-transformer-file ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { transformerFile } from '@walkeros/server-transformer-file'; import { storeFsInit } from '@walkeros/server-store-fs'; await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './public' } }, }, }, transformers: { file: { code: transformerFile, config: { settings: { prefix: '/static' }, }, env: { store: '$store.files' }, }, }, }); ``` Add to your `flow.json`: ``` "stores": { "files": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./public" } } } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.files" } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This transformer uses the standard transformer config wrapper (consent, data, env, id, ...). For the shared fields see [transformer configuration](/docs/transformers/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ----------- | ------------------------ | ------------------------------------------------------------------------------------------------- | ---- | | `prefix` | `string` | URL prefix to strip before store lookup. E.g., "/static" → /static/walker.js looks up "walker.js" | | | `headers` | `Record` | Default response headers (e.g., Cache-Control, X-Frame-Options) | | | `mimeTypes` | `Record` | Extension → Content-Type overrides. Keys include dot: { ".wasm": "application/wasm" } | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Serve static file Serve a static JavaScript file from store. Config: prefix: "/static", headers: { "Cache-Control": "public, max-age=3600" } Event ``` { "name": "page view", "data": { "path": "/static/walker.js" }, "id": "ev-1700000600", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } ``` Out ``` respond({ "status": 200, "headers": { "Content-Type": "application/javascript", "Cache-Control": "public, max-age=3600" } }) ``` ## How it works[​](#how-it-works "Direct link to How it works") 1. Reads `ingest.path` from the request context 2. Strips the configured `prefix` (if set), e.g. `/static/image.png` becomes `image.png` 3. Looks up the remaining path in the store via `store.get(key)` 4. If found, derives the Content-Type from the file extension and responds with the content 5. Returns `false` to stop the chain. No further transformers or destinations run. 6. If the file is not found or the path doesn't match the prefix, the event passes through unchanged ## Environment[​](#environment "Direct link to Environment") | Key | Type | Required | Description | | ------- | ---------------- | -------- | ------------------------------------------------------------------------------------ | | `store` | `Store.Instance` | Yes | Store providing file content. Without one, the transformer warns and passes through. | ## Cache and file chain[​](#cache-and-file-chain "Direct link to Cache and file chain") Use native cache on the file transformer to serve cached files from memory, avoiding repeated store reads: ``` "sources": { "http": { "package": "@walkeros/server-source-express", "next": [ { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "next": "file" } ] } }, "transformers": { "file": { "package": "@walkeros/server-transformer-file", "config": { "settings": { "prefix": "/static" } }, "env": { "store": "$store.files" }, "cache": { "rules": [ { "match": "*", "key": ["ingest.method", "ingest.path"], "ttl": 300 } ] } } } ``` ## Custom MIME types[​](#custom-mime-types "Direct link to Custom MIME types") ``` { settings: { mimeTypes: { '.wasm': 'application/wasm', '.mjs': 'application/javascript', }, }, } ``` ## Behavior notes[​](#behavior-notes "Direct link to Behavior notes") * **Requires a store**: without `env.store`, every request logs a warning and passes through * **Prefix matching**: if `prefix` is set and the path doesn't start with it, the event passes through (not an error) * **Returns `false`**: a served file stops the chain; no destinations receive the event * **Content-Length**: automatically set for string and Buffer content ## Next steps[​](#next-steps "Direct link to Next steps") * **[Cache](/docs/collector/cache.md)** - Add response caching before file serving (integrated collector mode) * **[Stores](/docs/getting-started/stores.md)** - Configure store backends * **[Docker: including files](/docs/apps/docker.md#including-files)** - Bake files into Docker images --- # Fingerprint [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/fingerprint) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/server/transformers/fingerprint)[ ](https://www.npmjs.com/package/@walkeros/server-transformer-fingerprint) [Package](https://www.npmjs.com/package/@walkeros/server-transformer-fingerprint) Hashes configurable request fields (IP address, user agent, date, etc.) into a deterministic identifier and stores it on the event. No cookies, no PII stored. The same combination of inputs always produces the same hash, enabling server-side session continuity, cookie-free analytics, and cross-domain stitching without client-side IDs. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/server-transformer-fingerprint ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { transformerFingerprint } from '@walkeros/server-transformer-fingerprint'; await startFlow({ transformers: { fingerprint: { code: transformerFingerprint, config: { settings: { fields: ['ingest.ip', 'ingest.userAgent'], output: 'user.hash', length: 16, }, }, }, }, }); ``` Add to your `flow.json`: ``` "transformers": { "fingerprint": { "package": "@walkeros/server-transformer-fingerprint", "config": { "settings": { "fields": ["ingest.ip", "ingest.userAgent"], "output": "user.hash", "length": 16 } } } } ``` [See bundled mode setup](/docs/getting-started/modes/bundled.md) | [CLI reference](/docs/apps/cli.md) ## Configuration[​](#configuration "Direct link to Configuration") This transformer uses the standard transformer config wrapper (consent, data, env, id, ...). For the shared fields see [transformer configuration](/docs/transformers/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | --------- | ------------ | ----------------------------------------------------------------------------------------------------------- | ---- | | `fields*` | `Array` | Fields to include in hash (order matters). Each resolved via getMappingValue with source { event, ingest }. | | | `output` | `string` | Dot-notation path where hash is stored on the event. Default: "user.hash" | | | `length` | `integer` | Truncate hash to this length. Default: full 64-char SHA-256 hash | | \* Required fields ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### IP anonymization Privacy-preserving fingerprint using key+fn pattern: fn truncates IP to /24 subnet before hashing, so 10.0.42.\* users share a hash. Config: fields: \[{ key: "ingest.ip", fn: ip => ip.replace(/\\.\d+$/, ".0") }, "ingest.userAgent"] Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "Privacy Policy", "id": "/privacy" }, "id": "ev-1700000602", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000602, "source": { "type": "express", "platform": "server" } } ``` Out ``` return { "event": { "name": "page view", "data": { "domain": "www.example.com", "title": "Privacy Policy", "id": "/privacy" }, "user": { "hash": "44d9154b9a9b3792" }, "id": "ev-1700000602", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000602, "source": { "type": "express", "platform": "server" } } } ``` ### Server fingerprint Standard server fingerprint using ingest.ip and ingest.userAgent. Requires source config.ingest. Event ``` { "name": "page view", "data": { "domain": "www.example.com", "title": "Getting Started", "id": "/docs/getting-started" }, "id": "ev-1700000600", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } ``` Out ``` return { "event": { "name": "page view", "data": { "domain": "www.example.com", "title": "Getting Started", "id": "/docs/getting-started" }, "user": { "hash": "158f99cc06e33fd6" }, "id": "ev-1700000600", "trigger": "load", "entity": "page", "action": "view", "timestamp": 1700000600, "source": { "type": "express", "platform": "server" } } } ``` Fields are resolved from `{ event, ingest }` using walkerOS mapping. String values use dot notation (`"ingest.ip"`). Function values compute dynamically (`{ fn: () => new Date().getDate() }`). ## Daily rotation[​](#daily-rotation "Direct link to Daily rotation") Without rotation, the same IP + user agent always produce the same hash, indefinitely. To limit persistence, add a daily rotation field: ``` settings: { fields: [ 'ingest.ip', 'ingest.userAgent', { fn: () => new Date().toISOString().slice(0, 10) }, // "2024-01-15" ], output: 'user.hash', length: 16, } ``` The hash resets each day, limiting cross-day tracking while maintaining session continuity within a day. ## IP anonymization[​](#ip-anonymization "Direct link to IP anonymization") To avoid including the raw IP in the hash input, transform it before hashing: ``` settings: { fields: [ { key: 'ingest.ip', fn: (ip) => ip?.split('.').slice(0, 3).join('.') }, // drops last octet 'ingest.userAgent', ], output: 'user.hash', } ``` ## Result[​](#result "Direct link to Result") The hash is stored at the configured `output` path on the event: ``` // Input event (before fingerprint transformer) { name: 'page view', entity: 'page', action: 'view', ... } // Output event (after fingerprint transformer) { name: 'page view', entity: 'page', action: 'view', user: { hash: '158f99cc06e33fd6' }, ... } ``` If any field is missing (e.g., `ingest.ip` is undefined), it is treated as an empty string. The transformer never throws. --- # GA4 [Server](#)[ ](https://github.com/elbwalker/walkerOS/tree/main/packages/transformers/ga4) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/transformers/ga4)[ ](https://www.npmjs.com/package/@walkeros/transformer-ga4) [Package](https://www.npmjs.com/package/@walkeros/transformer-ga4) Decoder transformer that turns Google Analytics 4 Measurement Protocol v2 hits (`/g/collect`, `/mp/collect`) into walkerOS events. Drop it in a server source's `before` chain to ingest existing `gtag`/Google Tag traffic without changing the front-end. One HTTP request can carry many GA4 events; the transformer returns one walkerOS event per GA4 event in the hit. This is the v1 release (`0.1.0`) with an explicit scope: server-side decoding via `source-express`, GA4 v2 only, replace-not-merge mapping semantics. See [Caveats](#caveats) and [Roadmap](#roadmap) for the boundaries. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/transformer-ga4 ``` ## Wire it up[​](#wire-it-up "Direct link to Wire it up") The transformer reads `ctx.ingest.url` (required) and `ctx.ingest.body` (optional) from the source it sits in front of. The recommended pairing is `@walkeros/server-source-express`: * Bundled * Integrated ``` { "version": 4, "flows": { "default": { "config": { "platform": "server" }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "ingest": { "map": { "url": { "key": "url" }, "path": { "key": "path" }, "method": { "key": "method" }, "body": { "key": "body" } } } }, "before": "ga4" } }, "transformers": { "ga4": { "package": "@walkeros/transformer-ga4" } }, "destinations": { "log": { "package": "@walkeros/destination-demo" } } } } } ``` ``` import { startFlow } from '@walkeros/collector'; import { sourceExpress } from '@walkeros/server-source-express'; import { transformerGa4 } from '@walkeros/transformer-ga4'; import { destinationDemo } from '@walkeros/destination-demo'; await startFlow({ sources: { http: { code: sourceExpress, config: { ingest: { map: { url: { key: 'url' }, path: { key: 'path' }, method: { key: 'method' }, body: { key: 'body' }, }, }, }, before: 'ga4', }, }, transformers: { ga4: { code: transformerGa4 }, }, destinations: { log: { code: destinationDemo }, }, }); ``` ### Ingest contract[​](#ingest-contract "Direct link to Ingest contract") The transformer expects the source to populate `ctx.ingest` with these keys: | Key | Type | Required | Notes | | ------ | -------- | -------- | ------------------------------------------------------------ | | `url` | `string` | yes | Full request URL including the query string. | | `path` | `string` | yes | Request path, drives the `/g/collect` `before` match. | | `body` | `string` | no | Raw POST body. Multi-event batches are `\n`-separated lines. | `config.ingest` must use the `map` operator with **direct `req` field paths** (no `req.` prefix), as shown above. A bare object like `{ "url": "req.url" }` is silently inert: without an operator the source returns `req` itself, `ctx.ingest` stays empty, and raw GA4 params reach the collector as a nameless event. If `url` is missing or not a string the transformer drops the event silently. If `body` is JSON-parsed by the source before reaching the transformer, pass the original raw string through or skip the transformer. ## Configuration[​](#configuration "Direct link to Configuration") This transformer uses the standard transformer config wrapper (consent, data, env, id, ...). For the shared fields see [transformer configuration](/docs/transformers/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") This package has no package-specific settings. ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Add to cart A GA4 add\_to\_cart hit decoded to a walkerOS product add event with currency and value. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=add_to_cart&ep.currency=EUR&epn.value=129.99" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "product add", "entity": "product", "action": "add", "data": { "currency": "EUR", "value": 129.99 } } ``` ### Batched POST (fan-out) A single POST request carrying two newline-separated events fans out into two walkerOS events. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000", "body": "en=add_to_cart&ep.currency=EUR&epn.value=19.99\nen=add_to_cart&ep.currency=EUR&epn.value=29.99" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "product add", "entity": "product", "action": "add", "data": { "currency": "EUR", "value": 19.99 } }; return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "product add", "entity": "product", "action": "add", "data": { "currency": "EUR", "value": 29.99 } } ``` ### Begin checkout A GA4 begin\_checkout hit decoded to a walkerOS order start event with currency, value, and coupon. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=begin_checkout&ep.currency=EUR&epn.value=149.97&ep.coupon=WELCOME10" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "order start", "entity": "order", "action": "start", "data": { "currency": "EUR", "value": 149.97, "coupon": "WELCOME10" } } ``` ### Consent denied (gcs=G100) A page\_view hit with gcs=G100 still maps, with consent.{marketing,analytics} both false on the resulting event. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&gcs=G100&en=page_view&dl=https%3A%2F%2Fx&dt=X" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": { "marketing": false, "analytics": false }, "name": "page view", "entity": "page", "action": "view", "data": { "id": "https://x", "title": "X" } } ``` ### Custom event (\* fallback) Unknown GA4 event names hit the \* fallback rule and surface as a ga4 track event carrying the original name. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=newsletter_subscribe&ep.source=footer" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "ga4 track", "entity": "ga4", "action": "track", "data": { "event_name": "newsletter_subscribe" } } ``` ### Login A GA4 login hit decoded to a walkerOS session login event with the auth method. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=login&ep.method=google" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "session login", "entity": "session", "action": "login", "data": { "method": "google" } } ``` ### Page view A standard GA4 page\_view hit decoded to a walkerOS page view with id, title, and referrer. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=page_view&dl=https%3A%2F%2Fshop.example.com%2Fproducts%2Fsku-123&dt=Trail%20Runner%20Pro&dr=https%3A%2F%2Fshop.example.com%2F" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "page view", "entity": "page", "action": "view", "data": { "id": "https://shop.example.com/products/sku-123", "title": "Trail Runner Pro", "referrer": "https://shop.example.com/" } } ``` ### Purchase (canary) A GA4 purchase hit decoded to a walkerOS order complete event with id, currency, total, tax, shipping, and coupon. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=purchase&ep.transaction_id=T-9001&ep.currency=EUR&epn.value=149.97&epn.tax=23.97&epn.shipping=4.95&ep.coupon=WELCOME10" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "order complete", "entity": "order", "action": "complete", "data": { "id": "T-9001", "currency": "EUR", "total": 149.97, "tax": 23.97, "shipping": 4.95, "coupon": "WELCOME10" } } ``` ### Scroll A GA4 scroll hit decoded to a walkerOS page scroll event with the percent\_scrolled value. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=scroll&epn.percent_scrolled=90" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "page scroll", "entity": "page", "action": "scroll", "data": { "percent": 90 } } ``` ### Search A GA4 search hit decoded to a walkerOS search submit event carrying the search term. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=search&ep.search_term=trail%20runner" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "search submit", "entity": "search", "action": "submit", "data": { "term": "trail runner" } } ``` ### user\_engagement (ignored) Auto-fired GA4 user\_engagement events are dropped by default — the transformer returns false. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=user_engagement&_et=1500" } ``` Out ``` return false ``` ### View item A GA4 view\_item hit decoded to a walkerOS product view event with currency and value. Event ``` { "url": "https://www.google-analytics.com/g/collect?v=2&tid=G-EXAMPLE&_p=p1&cid=cid-1&sid=1700000000&en=view_item&ep.currency=EUR&epn.value=129.99" } ``` Out ``` return { "id": "p1", "timestamp": 1700000000000, "timing": 0, "trigger": "ga4", "user": { "device": "cid-1", "session": "1700000000" }, "globals": {}, "source": { "type": "ga4" }, "consent": {}, "name": "product view", "entity": "product", "action": "view", "data": { "currency": "EUR", "value": 129.99 } } ``` ## Default mappings[​](#default-mappings "Direct link to Default mappings") `transformer-ga4` ships with default mappings for 33 standard GA4 event names. Out of the box, you get pageviews, ecommerce, list/promotion, engagement, and auth events mapped to walkerOS's [entity-action naming](/docs/getting-started/event-model.md). ### Page / scroll / click[​](#page--scroll--click "Direct link to Page / scroll / click") | GA4 (`en`) | walkerOS (`name`) | Fields | | --------------- | ----------------- | --------------------------- | | `page_view` | `page view` | `id`, `title`, `referrer` | | `scroll` | `page scroll` | `percent` | | `click` | `link click` | `url`, `domain`, `outbound` | | `file_download` | `file download` | `name`, `extension`, `url` | ### Ecommerce[​](#ecommerce "Direct link to Ecommerce") | GA4 (`en`) | walkerOS (`name`) | Fields | | ------------------- | ----------------- | -------------------------------------------- | | `view_item` | `product view` | `currency`, `value` | | `add_to_cart` | `product add` | `currency`, `value` | | `remove_from_cart` | `product remove` | `currency`, `value` | | `view_cart` | `cart view` | `currency`, `value` | | `begin_checkout` | `order start` | `currency`, `value`, `coupon` | | `add_shipping_info` | `order shipping` | `currency`, `value`, `tier` | | `add_payment_info` | `order payment` | `currency`, `value`, `type` | | `purchase` | `order complete` | `id`, `currency`, `total`, `tax`, `shipping` | | `refund` | `order refund` | `id`, `currency`, `total` | | `add_to_wishlist` | `wishlist add` | `currency`, `value` | ### List / promotion[​](#list--promotion "Direct link to List / promotion") | GA4 (`en`) | walkerOS (`name`) | Fields | | ------------------ | ----------------- | ---------------------- | | `view_item_list` | `list view` | `id`, `name` | | `select_item` | `product click` | `list_id`, `list_name` | | `view_promotion` | `promotion view` | reads from `items[0]` | | `select_promotion` | `promotion click` | reads from `items[0]` | | `select_content` | `content select` | `type`, `id` | ### Video / form / search[​](#video--form--search "Direct link to Video / form / search") | GA4 (`en`) | walkerOS (`name`) | Fields | | ---------------- | ----------------- | ----------------------------------------- | | `video_start` | `video start` | `title`, `duration`, `current`, `percent` | | `video_progress` | `video progress` | same as `video_start` | | `video_complete` | `video complete` | same as `video_start` | | `form_start` | `form start` | `id`, `name`, `destination` | | `form_submit` | `form submit` | `id`, `name`, `destination` | | `search` | `search submit` | `term` | ### Auth / lead / share[​](#auth--lead--share "Direct link to Auth / lead / share") | GA4 (`en`) | walkerOS (`name`) | Fields | | --------------- | ----------------- | ---------------------- | | `login` | `session login` | `method` | | `sign_up` | `session signup` | `method` | | `generate_lead` | `lead generate` | `currency`, `value` | | `share` | `content share` | `method`, `type`, `id` | ### Auto-fired noise (dropped by default)[​](#auto-fired-noise-dropped-by-default "Direct link to Auto-fired noise (dropped by default)") | GA4 (`en`) | Behavior | | ----------------- | -------------- | | `user_engagement` | `ignore: true` | | `session_start` | `ignore: true` | | `first_visit` | `ignore: true` | These events are emitted automatically by `gtag` and rarely carry analytics intent. Override the rule if you need them. ### Fallback[​](#fallback "Direct link to Fallback") | GA4 (`en`) | walkerOS (`name`) | Fields | | ---------- | ----------------- | --------------------------------- | | `'*'` | `ga4 track` | `data.event_name` = original `en` | Any GA4 event name not listed above falls through to `'*'` and produces a generic `ga4 track` walkerOS event. Override `'*'` to change the fallback rule globally. ## Override a default field[​](#override-a-default-field "Direct link to Override a default field") User config **replaces** the matching default rule per event name. Other events keep their defaults. To swap a field on `purchase`: ``` { "transformers": { "ga4": { "package": "@walkeros/transformer-ga4", "config": { "settings": { "mapping": { "purchase": { "name": "order complete", "data": { "map": { "id": "params.ep.transaction_id", "total": "params.epn.value", "currency": "params.ep.currency", "coupon": "params.ep.promo_code" } } } } } } } } } ``` Because v1 uses replace semantics, the entire `purchase` rule is taken from user config: copy any default fields you want to keep. Additive per-field merge is on the [roadmap](#roadmap). ## Drop an event[​](#drop-an-event "Direct link to Drop an event") Set `ignore: true` on any key to prevent it from being emitted: ``` "settings": { "mapping": { "click": { "ignore": true } } } ``` This is how `user_engagement`, `session_start`, and `first_visit` are silenced by default. ## Custom events[​](#custom-events "Direct link to Custom events") Two patterns: **1. Override `'*'`** to change the global fallback for unknown GA4 event names: ``` "settings": { "mapping": { "*": { "name": "custom event", "data": { "map": { "event_name": "name" } } } } } ``` **2. Add a specific key** for an event you fire via `gtag('event', '', ...)`: ``` "settings": { "mapping": { "newsletter_subscribe": { "name": "newsletter signup", "data": { "map": { "source": "params.ep.source" } } } } } ``` ## Tracking ID filtering[​](#tracking-id-filtering "Direct link to Tracking ID filtering") By default only Measurement IDs starting with `G-` are accepted; Ads (`AW-`) and DC (`DC-`) hits are dropped. Widen via a string regex in `settings.tidPattern`: ``` "settings": { "tidPattern": "^(G|AW|DC)-" } ``` The string is compiled to a `RegExp` at init time. ## Caveats[​](#caveats "Direct link to Caveats") * **Replace semantics, not merge.** A user mapping rule fully replaces the matching default rule. There is no per-field merge inside `data.map` in v1. * **GA4 v2 only.** Assumes the v2 Measurement Protocol layout (`ep.`, `epn.`, `up.`, `upn.`, `prN`, `gcs`). v1 is out of scope. * **`G-` tids only by default.** Override `tidPattern` to capture Ads and DC traffic. * **Basic `gcs` only.** Maps `G1XX` to `marketing`/`analytics` booleans. Functional/preferences flags and the newer `gcd` parameter are not decoded. * **Body must be raw text.** The transformer parses POST bodies as URL-encoded form lines. Pre-parsed JSON bodies will not decode. * **Ingest contract is required.** Source wiring must populate `ctx.ingest.url` (required) and `ctx.ingest.body` (optional) for batched hits. ## Roadmap[​](#roadmap "Direct link to Roadmap") * **Additive per-field merge** so partial overrides extend the default rule instead of replacing it * **Web ingest via interception sources** for capturing `gtag` traffic from the browser * **More vendor decoders** (Segment, Snowplow, Adobe) following the same `before`-chain pattern * **Richer consent decoding** (`gcd`, functional/preferences flags) ## Next steps[​](#next-steps "Direct link to Next steps") * **[Create your own](/docs/transformers/create-your-own.md)** - Build custom transformers * **[Server source: express](/docs/sources/server/express.md)** - Pair the decoder with the HTTP source --- # Validate [](#) [Source code](https://github.com/elbwalker/walkerOS/tree/main/packages/transformers/validate)[ ](https://www.npmjs.com/package/@walkeros/transformer-validate) [Package](https://www.npmjs.com/package/@walkeros/transformer-validate) Checks events against JSON Schema [contracts](/docs/getting-started/flow/contract.md) and records a verdict. In `pass` mode it annotates the event and continues so a downstream step can route on the result. In `strict` mode it drops invalid events by stopping the chain. The transformer runs on both web and server. ## Installation[​](#installation "Direct link to Installation") ``` npm install @walkeros/transformer-validate ``` * Integrated * Bundled ``` import { startFlow } from '@walkeros/collector'; import { transformerValidate } from '@walkeros/transformer-validate'; await startFlow({ transformers: { validate: { code: transformerValidate, config: { settings: { contract: [contractWeb], mode: 'strict' } }, }, }, }); ``` ``` "transformers": { "validate": { "package": "@walkeros/transformer-validate", "config": { "settings": { "contract": ["$contract.web"], "mode": "strict" } }, "next": "ga4" } } ``` ## How it works[​](#how-it-works "Direct link to How it works") The transformer reads the `{ ingest, event }` context and validates the canonical event. Each `contract` entry is a constraint: a `$contract.` reference resolves to the entity-action schemas for the event, an inline JSON Schema applies to the whole event. All entries are AND-ed and every error is aggregated. Set `format: true` to additionally check the canonical partial event shape (field types and structure); required-field rules come from `contract`. `mode` decides what happens to an invalid event: * **`pass`** (default): write the verdict to the event and continue. A downstream destination or transformer routes on `event.source.valid`. * **`strict`**: record the errors, then stop the chain so the event never reaches downstream steps. The verdict and the error list are written to two different places. The boolean verdict goes onto the **event** (`output.isValid`, default `source.valid`) as analytics-grade data that travels with the event. The issue list goes onto the **ingest** (`output.errors`, default `validation`) as observer-visible diagnostics, never event data, so it survives even a strict-mode drop. Set either path to an empty string to skip that write. ## Configuration[​](#configuration "Direct link to Configuration") This transformer uses the standard transformer config wrapper (consent, data, env, id, ...). For the shared fields see [transformer configuration](/docs/transformers/.md#configuration). Package-specific fields live under `config.settings` and are listed below. ## Settings[​](#settings "Direct link to Settings") | Property | Type | Description | More | | ---------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `contract` | `Array` | Validation constraints. Each entry is a resolved $contract.\* rule (with entity-action \`events\` schemas and/or a full-event \`schema\`) or an inline whole-event JSON Schema. All entries are AND-ed; every error is aggregated. | | | `format` | `boolean` | When true, also validate the canonical WalkerOS.Event structural shape (correct field types, no unknown fields). All fields are optional, so this checks structure and types, not presence. | | | `mode` | `'strict' \| 'pass'` | \`strict\` drops invalid events (chain-stop) after recording errors; \`pass\` annotates and continues. Default \`pass\`. | | | `output` | `output` | Where the verdict (on the event) and the issue list (on the ingest) are written. | | | `isValid` | `string` | Event dot-path for the boolean verdict. Default \`source.valid\`. Empty string = skip. | | | `errors` | `string` | Ingest dot-path for the issue list. Default \`validation\`. Empty string = skip. | | ## Mapping[​](#mapping "Direct link to Mapping") This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see [mapping](/docs/mapping/.md). ## Examples ### Filter gtm.\* via a contract pattern (real event passes) The same ^gtm\\. rejection contract leaves a real "page view" untouched: it passes and is annotated source.valid:true. Event ``` { "name": "page view", "entity": "page", "action": "view" } ``` Out ``` return { "event": { "name": "page view", "entity": "page", "action": "view", "source": { "valid": true } } } ``` ### Strict validate against a contract (valid) A "page view" with the required data.title passes the inline contract. The verdict source.valid:true is written to the event; the chain continues. Event ``` { "name": "page view", "entity": "page", "action": "view", "data": { "title": "Home" } } ``` Out ``` return { "event": { "name": "page view", "entity": "page", "action": "view", "data": { "title": "Home" }, "source": { "valid": true } } } ``` ## Reference a contract[​](#reference-a-contract "Direct link to Reference a contract") The canonical place for event shapes is the top-level `contract` block. Reference it with `$contract.`; the bundler resolves the reference to a concrete schema before deploy, so the runtime transformer only ever sees resolved schemas. ``` { "version": 4, "contract": { "web": { "events": { "order": { "complete": { "properties": { "data": { "required": ["total", "currency"] } } } } } } }, "flows": { "default": { "transformers": { "validate": { "package": "@walkeros/transformer-validate", "config": { "settings": { "contract": ["$contract.web"], "format": true, "mode": "strict", "output": { "isValid": "source.valid", "errors": "validation" } } }, "next": "ga4" } } } } } ``` ## Filter unwanted events[​](#filter-unwanted-events "Direct link to Filter unwanted events") There is no separate `ignore` or `filter` setting. To drop unwanted events (for example GTM lifecycle pings like `gtm.js` / `gtm.dom`), author an inline schema that rejects them and run `mode: "strict"`. A schema where `name` must NOT match `^gtm\.` fails those events while real events pass. ``` "validate": { "package": "@walkeros/transformer-validate", "config": { "settings": { "contract": [{ "type": "object", "properties": { "name": { "not": { "pattern": "^gtm\\." } } } }], "mode": "strict" } } } ``` ## Limitations[​](#limitations "Direct link to Limitations") * The error list may contain an extra parent entry pointing at a `properties` wrapper per failure, a quirk of the underlying JSON Schema engine. The `isValid` verdict is unaffected: it is `false` exactly when there is at least one failure. * Issues are emitted at `level: "error"`; there is no warn level. ## Next steps[​](#next-steps "Direct link to Next steps") * **[Contract](/docs/getting-started/flow/contract.md)** - named, inheritable event schemas referenced via `$contract` * **[Create your own](/docs/transformers/create-your-own.md)** - build custom transformers --- # Upgrading walkerOS clients The walkerOS app (`app.walkeros.io`) enforces minimum client versions per endpoint. When an endpoint is updated in a way that is not backwards-compatible with older clients, the app starts returning **`426 Upgrade Required`** to any client below the new minimum version. This page explains how to recognise it and how to recover. ## How clients identify themselves[​](#how-clients-identify-themselves "Direct link to How clients identify themselves") Every request from an official walkerOS client carries three headers: | Header | Example | | --------------------------- | ------------------------------------------ | | `User-Agent` | `walkeros-cli/1.4.0 (node/18.19.0; linux)` | | `X-WalkerOS-Client` | `cli`, `mcp`, or `runner` | | `X-WalkerOS-Client-Version` | `1.4.0` | The server uses these to: * Route tailored upgrade instructions per client type. * Decide whether the caller meets the minimum version for the endpoint. * Distinguish interactive CLI sessions from long-lived flow runners. Set the env var `WALKEROS_CLIENT_TYPE=runner` to override the default client type to `runner`. The official runner Docker image does this automatically. ## The 426 response[​](#the-426-response "Direct link to The 426 response") When the client is below the required version, the app returns `426 Upgrade Required` with a JSON body: ``` { "error": "upgrade_required", "client": "cli", "current": "1.4.0", "minimum": "1.5.0", "upgrade": "npm install -g @walkeros/cli@latest", "docs": "https://walkeros.io/docs/upgrading" } ``` The CLI prints this as a one-line error and exits with code `2`. The MCP server surfaces the same info in its tool error responses so AI assistants can relay the instruction. ## Upgrade checklist[​](#upgrade-checklist "Direct link to Upgrade checklist") * **CLI** (`@walkeros/cli`): `npm install -g @walkeros/cli@latest` (or bump the version in your project's `package.json` if you installed locally). * **MCP server** (`@walkeros/mcp`): update the pinned version in your MCP client config, or bump `@walkeros/mcp` in your project and restart the client. * **Runner image**: pull the latest `walkeros/flow` (or your self-hosted tag) and redeploy. The image ships with a current `@walkeros/cli` and sets `WALKEROS_CLIENT_TYPE=runner`. After upgrading, re-run the failing command. If the `426` persists, your client is still below the minimum version. Double-check the installed version against the `minimum` field the server reported. ## Why this exists[​](#why-this-exists "Direct link to Why this exists") The wire contract between clients and the app evolves. Rather than silently breaking older clients with a `400`, the app sends `426` with a precise required version and the exact upgrade command, so the fix is always one copy-paste away. --- # walkerOS Skills Skills are the source of truth for working with walkerOS. Each page below is generated from the matching `skills//SKILL.md` in the repository. ## Understanding[​](#understanding "Direct link to Understanding") * [Understanding Destinations](/skills/walkeros-understanding-destinations.md) — Use when working with walkerOS destinations, understanding the destination interface, or learning about env pattern and configuration. Covers interface, lifecycle, env mocking, and paths. * [Understanding Development](/skills/walkeros-understanding-development.md) — Use when contributing to walkerOS, before writing code, or when unsure about project conventions. Covers build/test/lint workflow, XP principles, folder structure, and package usage. * [Understanding Events](/skills/walkeros-understanding-events.md) — Use when creating walkerOS events, understanding event structure, or working with event properties. Covers entity-action naming, event properties, statelessness, and vendor-agnostic design. * [Understanding Flow](/skills/walkeros-understanding-flow.md) — Use when learning walkerOS architecture, understanding data flow, or designing composable event pipelines. Covers Source→Collector→Destination pattern and separation of concerns. * [Understanding Mapping](/skills/walkeros-understanding-mapping.md) — Use when transforming walkerOS events in the flow (source→collector or collector→destination), configuring data/map/loop/set/condition/policy, or using $code: syntax in JSON configs. * [Understanding Sources](/skills/walkeros-understanding-sources.md) — Use when working with walkerOS sources, understanding event capture, or learning about the push interface. Covers browser, dataLayer, and server source patterns. * [Understanding Stores](/skills/walkeros-understanding-stores.md) — Use when working with walkerOS stores, understanding key-value storage in flows, or learning about store injection via env. Covers interface, lifecycle, $store. wiring, and available store packages. * [Understanding Transformers](/skills/walkeros-understanding-transformers.md) — Use when working with walkerOS transformers, understanding event validation/enrichment/redaction, or learning about transformer chaining. Covers interface, return values, and pipeline integration. ## Using[​](#using "Direct link to Using") * [Using Cli](/skills/walkeros-using-cli.md) — Use when bundling walkerOS flows, testing events with simulate/push, running local servers, validating configs, or configuring Flow JSON files. * [Using Logger](/skills/walkeros-using-logger.md) — Use when working with walkerOS sources/destinations to understand standard logging patterns, replace console.log, or add logging to external API calls. Covers DRY principles, when to log, and migration patterns. * [Using Step Examples](/skills/walkeros-using-step-examples.md) — Use when writing, simulating, validating, or testing with walkerOS step examples. Covers the complete lifecycle from authoring examples to CI integration. * [Using Store Cache](/skills/walkeros-using-store-cache.md) — Use when adding read-through caching to a walkerOS store, memoizing a slow API/Sheets backing, composing multi-tier cache chains, or deduplicating concurrent store reads. Covers recipes, TTL choice, error policy, and observability counters. * [Using Transformer Ga4](/skills/walkeros-using-transformer-ga4.md) — Use when wiring `@walkeros/transformer-ga4` into a server flow, overriding default GA4 event mappings, dropping events, adding custom event keys, or troubleshooting GA4 Measurement Protocol decoding. Covers the `before`-chain wiring contract, configuration recipes, and per-field patching with extend/remove. ## Creating[​](#creating "Direct link to Creating") * [Create Cmp Source](/skills/walkeros-create-cmp-source.md) — Use when creating a new walkerOS CMP (consent management platform) source. Structured fill-in-the-blanks workflow that turns any CMP's consent API into a walkerOS source package. Covers CookieFirst, Usercentrics, CookiePro/OneTrust patterns and generalizes to any CMP. * [Create Destination](/skills/walkeros-create-destination.md) — Use when creating a new walkerOS destination to send events to a vendor or API (GA4/gtag, Meta/Facebook Pixel, Mixpanel, Amplitude, a custom HTTP API, Measurement Protocol), web or server-side. Example-driven workflow: research the vendor SDK and define step examples before implementing the destination interface, env pattern, and mappings. * [Create Source](/skills/walkeros-create-source.md) — Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern. * [Create Transformer](/skills/walkeros-create-transformer.md) — Use when creating a new walkerOS transformer to modify events in the pipeline (validate, enrich, or redact) at a before/next chain position. Example-driven workflow for the transformer interface, return values, and chaining. ## Tasks[​](#tasks "Direct link to Tasks") * [Debugging](/skills/walkeros-debugging.md) — Use when walkerOS events aren't reaching destinations, debugging event flow, or troubleshooting mapping issues. Covers common problems and debugging strategies. * [Mapping Configuration](/skills/walkeros-mapping-configuration.md) — Use when configuring walkerOS event mappings for specific use cases. Provides recipes for GA4, Meta, custom APIs, and common transformation patterns. * [Mcp Actions](/skills/walkeros-mcp-actions.md) — Use when an AI agent calls walkerOS MCP tools from code (code execution / Code Mode, a Worker, or an app sandbox) to validate, simulate, bundle, or inspect flows and packages, and wants to filter large results in code instead of issuing many separate tool calls. * [Testing Strategy](/skills/walkeros-testing-strategy.md) — Use when writing tests, reviewing test code, or discussing testing approach for walkerOS packages. Covers env pattern, dev examples, and package-specific strategies. * [Writing Documentation](/skills/walkeros-writing-documentation.md) — Use when writing or updating walkerOS documentation - README, website docs, or skills. Covers quality standards, example validation, and DRY patterns. --- # Create a CMP Source A CMP source is a specialized walkerOS source that listens to a consent management platform's events and translates consent states into `elb('walker consent', state)` calls. Every CMP source follows the same skeleton with only 5-6 decision points that vary per CMP. This skill turns "build a new CMP source" into a structured fill-in-the-blanks workflow. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, read these skills: * [understanding-flow](/skills/walkeros-understanding-flow.md) - How sources fit in architecture * [understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface * [create-source](/skills/walkeros-create-source.md) - General source creation workflow * [testing-strategy](/skills/walkeros-testing-strategy.md) - How to test * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards (for Phase 8) ## Supporting files[​](#supporting-files "Direct link to Supporting files") This skill includes reference files you can copy: * **[examples/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples)** - Generic CMP consent examples to adapt for your CMP * [inputs.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/inputs.ts) - Consent input scenarios (full, partial, minimal, revocation) * [outputs.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/outputs.ts) - Expected walkerOS consent states after mapping * [env.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/env.ts) - Mock environment factories (createMockElb, createMockWindow) * **[templates/cmp/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp)** - Complete CMP source implementation template * [index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/index.ts) - Source implementation with detection paths + handleConsent * [types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/types.ts) - Type definitions (Settings, CMP API, Types bundle) * [test-utils.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/test-utils.ts) - Test utilities (MockWindow, createMockElb, createCmpSource) * [index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/index.test.ts) - Test suite skeleton (8 describe blocks) **Note:** These are generic templates for the skill. The actual CMP package's examples go in `packages/web/sources/cmps/[name]/src/examples/`. ## Canonical template[​](#canonical-template "Direct link to Canonical template") **CookieFirst** at `packages/web/sources/cmps/cookiefirst/` is the canonical template. Copy its structure for every new CMP source. **Note:** The CookieFirst README and package.json predate this skill and are missing some sections required by mandatory check #10 (walkerOS.json, Type definitions, Related, Timing considerations) and the `walkerOS-source` keyword. New CMP sources MUST include all required sections. The CookieFirst package will be updated to match. ## Process overview[​](#process-overview "Direct link to Process overview") ``` 1. Research -> Fill in the CMP research template 2. Examples -> Create input/output examples (full, partial, minimal, revocation, edge cases) 3. Mapping -> Define category map with sensible defaults 4. Scaffold -> Copy from CookieFirst template 5. Convention -> walkerOS.json, buildDev 6. Test -> Write tests FIRST (TDD): 25-32 tests across 8-9 describe blocks 7. Implement -> Wire up detection paths + handleConsent + destroy 8. Document -> README + update existing consent guide page ``` *** ## Phase 1: Research the CMP[​](#phase-1-research-the-cmp "Direct link to Phase 1: Research the CMP") **Goal:** Fill in every field of this template before writing any code. ### CMP research template[​](#cmp-research-template "Direct link to CMP research template") Fill in ALL of these fields for the target CMP: | Field | Description | Example (CookieFirst) | | --------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | | **CMP name** | Official product name | CookieFirst | | **Global window object** | `window.X` shape and key properties | `window.CookieFirst.consent` (boolean map) | | **SDK events** | Event names, `CustomEvent` detail structure | `cf_init` (Event), `cf_consent` (CustomEvent with consent detail) | | **Callbacks/hooks** | Function wrapping patterns | None | | **Category naming** | Human-readable vs opaque IDs | Human-readable (`necessary`, `functional`, `performance`, `advertising`) | | **Consent access pattern** | Is consent read from a property (`window.CMP.consent`) or an API method (`CMP.getConsent()`)? | Property: `window.CookieFirst.consent` | | **Explicit consent detection** | API or mechanism to check if user actively chose | `consent === null` means no explicit choice | | **"Already loaded" detection** | How to detect CMP loaded before source | `window.CookieFirst.consent` exists and is non-null | | **Official docs URL** | Link to CMP's developer/API documentation | CookieFirst Public API docs | | **npm packages / TS types** | Available type packages | None (define own types) | | **Timing constraints** | Does the source need to load before/after the CMP? Any `require` config needed? | No `require` -- consent sources should init immediately | | **Event registration mechanism** | How does the CMP register event listeners? `addEventListener`, callback assignment, SDK method? | `addEventListener` (standard DOM events) | | **Cleanup/unsubscribe mechanism** | How to remove listeners on destroy? `removeEventListener`, nullify callback, SDK unsubscribe method? | `removeEventListener` (standard DOM cleanup) | | **SDK readiness pattern** | How does the CMP signal its SDK is ready? DOM event, callback array, global flag, Promise? | `cf_init` DOM event | ### Detection paths[​](#detection-paths "Direct link to Detection paths") Every CMP source needs up to 3 detection paths: | Path | Purpose | Questions to answer | | ------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------- | | **Already loaded** | CMP loaded before source | Is there a global object to check? What state does it expose? Is consent read from a property or an API method? | | **Init listener** | CMP loads after source | What event/callback fires on SDK init? Is it a DOM event, callback assignment, or SDK readiness array? | | **Change listener** | User updates consent | What event fires on consent change? Is it the same as init? | ### Event registration patterns[​](#event-registration-patterns "Direct link to Event registration patterns") Not all CMPs use `addEventListener`. Fill in "Event registration mechanism" in the research template to determine which pattern applies: | Pattern | CMPs | Registration | Cleanup | | ----------------------- | ------------------------ | ------------------------------------ | ------------------------------------ | | **DOM events** | CookieFirst, CookiePro | `addEventListener(name, handler)` | `removeEventListener(name, handler)` | | **Callback assignment** | Cookiebot (`onaccept`) | `window.CMP.onaccept = handler` | `window.CMP.onaccept = original` | | **SDK readiness array** | Didomi (`didomiOnReady`) | `window.didomiOnReady.push(handler)` | No unsubscribe (fires once) | | **SDK method** | Didomi (`on`) | `CMP.on('consent.changed', handler)` | Vendor-specific (check docs) | This affects the source skeleton (Phase 7), MockWindow shape (Phase 6), and destroy implementation. ### Decision matrix[​](#decision-matrix "Direct link to Decision matrix") Fill in this matrix for your CMP. Reference implementations for comparison: | Decision | CookieFirst | Usercentrics | CookiePro | | ----------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------------------ | | Already loaded? | `window.CookieFirst.consent` | None (events only) | `window.OneTrust` + `window.OptanonActiveGroups` | | Init listener | `cf_init` event | Same as change event (`ucEvent`) | `OptanonWrapper` callback (self-unwrap) | | Change listener | `cf_consent` event | `ucEvent` / `UC_SDK_EVENT` | `OneTrustGroupsUpdated` event | | Consent shape | Boolean map `{ category: bool }` | Mixed object (groups + services) | Comma-separated IDs `,C0001,C0003,` | | Category naming | Human-readable | Admin-configured | Opaque IDs (C0001-C0005) | | Explicit check | `consent === null` | Official consent metadata (V3 `consent.type`; V2 an `EXPLICIT` entry in service consent history) | `IsAlertBoxClosed()` | | Default categoryMap | Populated (human names to walkerOS) | Empty (pass-through) | Populated (opaque IDs need mapping) | | Number of change events | 1 (`cf_consent`) | 1 (`ucEvent`) | 2 (`OptanonWrapper` + `OneTrustGroupsUpdated`) | | Consent layers | Single (categories only) | Dual (groups + services) | Single (categories only) | | Consent access | Property (`window.CookieFirst.consent`) | Event detail (`event.detail`) | Property (`window.OptanonActiveGroups`) | | Event registration | `addEventListener` | `addEventListener` | Callback assignment + `addEventListener` | ### Gate: Research complete[​](#gate-research-complete "Direct link to Gate: Research complete") * All fields in the research template filled * Detection paths identified (which of the 3 apply) * Decision matrix row completed for the new CMP * Official docs URL captured *** ## Phase 2: Create examples[​](#phase-2-create-examples "Direct link to Phase 2: Create examples") **Goal:** Define realistic consent data BEFORE writing implementation. ### Required example inputs[​](#required-example-inputs "Direct link to Required example inputs") Create at minimum these scenarios in `src/examples/inputs.ts`. See [inputs.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/inputs.ts) for the generic template. | Example | Purpose | Description | | ----------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `fullConsent` | All categories accepted | User clicked "Accept All" | | `partialConsent` | Some categories accepted | User customized consent | | `minimalConsent` | Only essential/necessary | User clicked "Deny All" or similar | | `implicitConsent` | Page-load defaults (if applicable) | CMP loaded but user hasn't chosen. **Note:** behavior varies by CMP -- some grant nothing by default, others grant functional/necessary. Research the specific CMP's default consent state. | | `noConsent` | No consent data available | CMP hasn't loaded yet | | `revocationInput` | Consent withdrawal | User goes from full to partial | Add CMP-specific edge cases: * **Case sensitivity:** Uppercase/lowercase variants of consent fields * **Service-level consent:** If CMP supports individual service booleans * **Custom categories:** Non-standard category IDs * **Empty/malformed data:** Edge cases for the consent shape ### Required example outputs[​](#required-example-outputs "Direct link to Required example outputs") Create expected walkerOS consent states in `src/examples/outputs.ts`. See [outputs.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/outputs.ts) for the generic template. Key rule: denied groups MUST have explicit `false`, not be omitted (see mandatory check #1). ### Example env[​](#example-env "Direct link to Example env") Create mock factories in `src/examples/env.ts`. See [env.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples/env.ts) for the generic template. ### Export via dev.ts[​](#export-via-devts "Direct link to Export via dev.ts") ``` // src/dev.ts export * as examples from './examples'; ``` ### Gate: Examples valid[​](#gate-examples-valid "Direct link to Gate: Examples valid") * All example files compile * Can trace: each input -> expected output for each example * Edge cases included (case sensitivity, empty data, revocation) *** ## Phase 3: Define category mapping[​](#phase-3-define-category-mapping "Direct link to Phase 3: Define category mapping") **Goal:** Decide default `categoryMap` and document mapping rationale. ### When to use a populated default map[​](#when-to-use-a-populated-default-map "Direct link to When to use a populated default map") Use a populated default when the CMP uses **opaque or non-standard category names** that are meaningless without mapping: ``` // CookiePro: opaque IDs require mapping export const DEFAULT_CATEGORY_MAP: Record = { C0001: 'functional', // Strictly Necessary C0002: 'analytics', // Performance C0003: 'functional', // Functional C0004: 'marketing', // Targeting C0005: 'marketing', // Social Media }; ``` ### When to use an empty default map[​](#when-to-use-an-empty-default-map "Direct link to When to use an empty default map") Use an empty default when the CMP uses **human-readable, admin-configured category names** that can pass through as-is: ``` // Usercentrics: admin-configured names pass through const settings: Settings = { categoryMap: config?.settings?.categoryMap ?? {}, }; ``` ### Merging behavior[​](#merging-behavior "Direct link to Merging behavior") Custom entries merge with (and override) defaults: ``` const mergedCategoryMap = { ...DEFAULT_CATEGORY_MAP, ...(config?.settings?.categoryMap ?? {}), }; ``` ### OR logic for many-to-one mappings[​](#or-logic-for-many-to-one-mappings "Direct link to OR logic for many-to-one mappings") When multiple CMP categories map to the same walkerOS group, use OR logic: if ANY source category is `true`, the target group is `true`. ``` // OR logic: once true, stays true state[mapped] = state[mapped] || value; ``` ### Dual-layer consent (categories + services/vendors)[​](#dual-layer-consent-categories--servicesvendors "Direct link to Dual-layer consent (categories + services/vendors)") Some CMPs expose consent at multiple layers (e.g., Usercentrics: groups + services; Didomi: purposes + vendors). When the decision matrix shows "Consent layers: Dual," decide how to handle: **Option A: Map primary layer only (recommended for most cases).** Use the category/purpose layer and ignore the service/vendor layer. This matches walkerOS's category-level `WalkerOS.Consent` model directly. **Option B: Expose both layers via settings.** Add a setting like `consentLayer: 'categories' | 'services'` and map whichever the user chooses. Use this when the CMP's service-level consent is meaningfully different from its category-level consent. Document the chosen approach in the README under "How it works." ### Gate: Mapping defined[​](#gate-mapping-defined "Direct link to Gate: Mapping defined") * Default categoryMap chosen (populated or empty, with rationale) * OR logic for many-to-one mappings documented * Merge behavior with user overrides documented * Dual-layer strategy decided (if applicable per decision matrix) *** ## Phase 4: Scaffold[​](#phase-4-scaffold "Direct link to Phase 4: Scaffold") **Goal:** Create package structure mirroring CookieFirst. ### Directory structure[​](#directory-structure "Direct link to Directory structure") ``` packages/web/sources/cmps/[name]/ ├── package.json ├── tsconfig.json ├── tsup.config.ts ├── jest.config.mjs ├── README.md └── src/ ├── index.ts # Main source export ├── dev.ts # Dev exports (examples) ├── types/ │ └── index.ts # Types, Settings, CMP API interface, declare global ├── examples/ │ ├── index.ts # Re-exports │ ├── inputs.ts # CMP consent input examples │ ├── outputs.ts # Expected walkerOS consent outputs │ └── env.ts # Mock factories └── __tests__/ ├── index.test.ts # Full test suite └── test-utils.ts # createMockElb, createMockWindow, createSource ``` ### package.json template[​](#packagejson-template "Direct link to package.json template") ``` { "name": "@walkeros/web-source-cmp-[name]", "description": "[CMP Name] consent management source for walkerOS", "version": "1.0.0", "license": "MIT", "walkerOS": { "type": "source", "platform": "web" }, "main": "./dist/index.js", "module": "./dist/index.mjs", "types": "./dist/index.d.ts", "exports": { ".": { "types": "./dist/index.d.ts", "import": "./dist/index.mjs", "require": "./dist/index.js" }, "./examples": { "types": "./dist/examples/index.d.ts", "import": "./dist/examples/index.mjs", "require": "./dist/examples/index.js" }, "./dev": { "types": "./dist/dev.d.ts", "import": "./dist/dev.mjs", "require": "./dist/dev.js" } }, "files": ["dist/**"], "scripts": { "build": "tsup --silent", "clean": "rm -rf .turbo && rm -rf node_modules && rm -rf dist", "dev": "jest --watchAll --colors", "lint": "tsc && eslint \"**/*.ts*\"", "test": "jest", "update": "npx npm-check-updates -u && npm update" }, "dependencies": { "@walkeros/core": "1.0.0", "@walkeros/collector": "1.0.0" }, "repository": { "url": "git+https://github.com/elbwalker/walkerOS.git", "directory": "packages/web/sources/cmps/[name]" }, "author": "elbwalker ", "keywords": [ "walker", "walkerOS", "walkerOS-source", "source", "web", "[name]", "consent", "cmp" ] } ``` ### Config files[​](#config-files "Direct link to Config files") Copy these exactly from CookieFirst, updating only the `globalName`: * `tsconfig.json` - extends `@walkeros/config/tsconfig/web.json` * `tsup.config.ts` - uses `buildModules`, `buildExamples`, `buildBrowser`, `buildES5` * `jest.config.mjs` - extends `@walkeros/config/jest/web.config` *** ## Phase 5: walkerOS.json convention[​](#phase-5-walkerosjson-convention "Direct link to Phase 5: walkerOS.json convention") ### Add `walkerOS` field to package.json[​](#add-walkeros-field-to-packagejson "Direct link to add-walkeros-field-to-packagejson") ``` { "walkerOS": { "type": "source", "platform": "web" } } ``` ### Use `buildDev()` in tsup.config.ts[​](#use-builddev-in-tsupconfigts "Direct link to use-builddev-in-tsupconfigts") Use the standard `buildDev()` helper from `@walkeros/config/tsup` (consistent with `create-source` and `create-destination` skills): ``` import { buildDev } from '@walkeros/config/tsup'; // In defineConfig array: buildDev(), ``` **Note:** The CookieFirst template uses `buildModules({ entry: ['src/dev.ts'] })` instead. New CMP sources should prefer `buildDev()` for consistency. ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") If your CMP source has non-obvious behaviors or troubleshooting patterns, add hints. See `walkeros-writing-documentation` skill for full guidelines and `walkeros-create-source` skill for the hints pattern. ### Gate: Convention met[​](#gate-convention-met "Direct link to Gate: Convention met") * [ ] `walkerOS` field in package.json with type and platform * Dev build configured in tsup.config.ts * Keywords include `walkerOS` and `walkerOS-source` *** ## Phase 6: Test (TDD -- write tests BEFORE implementation)[​](#phase-6-test-tdd----write-tests-before-implementation "Direct link to Phase 6: Test (TDD -- write tests BEFORE implementation)") **Goal:** Write the full test suite first. Watch it fail. Then implement. ### Test structure (8-9 describe blocks)[​](#test-structure-8-9-describe-blocks "Direct link to Test structure (8-9 describe blocks)") | Describe block | Tests | What it covers | | ----------------------------- | ----- | --------------------------------------------------------------------------------------------- | | `initialization` | 4-6 | No errors, correct type, default settings, custom settings, listener registration | | `explicit consent filtering` | 3-4 | Explicit events processed, implicit ignored/processed based on setting, case-insensitive type | | `non-consent event filtering` | 2 | Non-consent events ignored, events without detail ignored | | `category mapping` | 5-8 | Full/partial/minimal consent, custom mapping, unmapped categories, OR logic | | `[CMP-specific parsing]` | 2-4 | CMP-specific consent format parsing (service-level, string parsing, etc.) | | `event handling` | 3 | Consent changes, multiple changes, revocation | | `consent revocation` | 2 | Full->partial, full->minimal (explicit false values verified) | | `cleanup` | 2 | Destroy removes listeners, restores wrapped functions | | `no window environment` | 1 | Handles missing window gracefully | ### Test utilities pattern (MockWindow)[​](#test-utilities-pattern-mockwindow "Direct link to Test utilities pattern (MockWindow)") See [test-utils.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/test-utils.ts) for the complete template including `MockWindow` interface, `createMockElb`, `createMockWindow`, and `createCmpSource` factories. For CMPs that use callback assignment or SDK methods instead of `addEventListener` (e.g., Didomi's `onReady`, Cookiebot's `onaccept`), adapt `MockWindow` to expose those callbacks as testable properties. ### Test template[​](#test-template "Direct link to Test template") Use the test template: [index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/index.test.ts) ### Gate: Tests fail for the right reason[​](#gate-tests-fail-for-the-right-reason "Direct link to Gate: Tests fail for the right reason") * Tests fail with "Cannot find module '../index'" (module doesn't exist yet) * Tests use example outputs for assertions (not hardcoded values) * Consent revocation test included (full grant -> revoke -> verify explicit `false`) *** ## Phase 7: Implement[​](#phase-7-implement "Direct link to Phase 7: Implement") **Goal:** Wire up detection paths, handleConsent, and destroy. Make tests pass. ### v4 source identity[​](#v4-source-identity "Direct link to v4 source identity") CMP sources register their `type` literal via `SourceMap` in `src/types.ts`. CMPs are web by default - `platform` is optional in the augmentation: ``` import type { Source } from '@walkeros/core'; declare module '@walkeros/core' { interface SourceMap { // Replace with the CMP's package-level identifier. cookiefirst: { type: 'cookiefirst'; platform?: 'web'; // optional: CMP runs on web by default }; } } ``` The source returns `{ type: 'cookiefirst', config, push }` from `Source.Init` * the `type` literal must match the `SourceMap` key. **CMP sources do NOT set `source.url` or `source.referrer`.** A CMP only knows about consent state, not the page that produced it. Setting page context belongs to a future web-context transformer that runs in the source's `next` chain. Don't reach for `window.location` in CMP source code, leave those fields untouched. ### Settings interface pattern[​](#settings-interface-pattern "Direct link to Settings interface pattern") See [types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/types.ts) for the complete type definitions including `Settings`, `InitSettings`, `Types` bundle, CMP API interfaces, and `declare global` window augmentation. Every CMP source has these core settings: * `categoryMap?: Record` -- map CMP categories to walkerOS consent groups * `explicitOnly?: boolean` -- only process explicit consent (default: `true`) * `globalName?: string` -- CMP-specific: global object name ### Source skeleton[​](#source-skeleton "Direct link to Source skeleton") Every CMP source follows this skeleton with 5-6 decision points. See [index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp/index.ts) for the complete template. Key implementation steps: 1. Resolve window (`env.window` fallback to `globalThis`) 2. Merge settings with defaults 3. Track listener references for cleanup 4. `handleConsent` function (explicitOnly, categoryMap with OR logic, `elb('walker consent', state)`) 5. Detection path: Already loaded 6. Detection path: Init listener 7. Detection path: Change listener **The factory MUST be side-effect-free; do the detection paths in `init()`.** Steps 5-7 attach listeners and perform the "already loaded" static consent read, which emits `elb('walker consent', state)`. Put them inside an `init()` method on the returned instance, NOT in the factory body. The collector runs the factory in Pass 1 (before all sources are merged) and `init()` in Pass 2. Emitting consent from the factory races source merge order and can leave a later `require:["consent"]` source parked. Return `{ type, config, push, init, destroy }`; `init` runs the static read + attaches listeners, `destroy` removes them. (The collector also matches `require` against current recorded state, so a consent read from `init()` still activates dependent steps regardless of order — but a side-effect-free factory is the contract.) **Note on init listeners:** Some init listeners (like CookieFirst's `cf_init`) read consent from the global window object, not from `event.detail`. Others (like `cf_consent`) receive consent via `event.detail`. Check which pattern your CMP uses for each detection path. ### Gate: All tests pass[​](#gate-all-tests-pass "Direct link to Gate: All tests pass") * [ ] `npm run verify:touched -- ` passes (L1: typecheck + lint + test) * [ ] `npm run build` passes *** ## Phase 8: Document[​](#phase-8-document "Direct link to Phase 8: Document") ### README structure[​](#readme-structure "Direct link to README structure") Follow this structure (sentence case headings, imports in code examples): ``` # @walkeros/web-source-cmp-[name] [CMP Name] consent management source for walkerOS. Source Code | NPM | Documentation ## Installation ## Usage (with imports) ## Configuration ### Settings (table) ### Default category mapping (if applicable) ### Custom mapping example ## How it works (numbered list of detection paths) ### Timing considerations ## [CMP] API reference (links to CMP docs) ## walkerOS.json ## Type definitions ## Related (links to consent guide) ## License ``` ### Update consent guide page[​](#update-consent-guide-page "Direct link to Update consent guide page") Update `website/docs/guides/consent/examples/[cmp-name].mdx` to recommend the source package first, preserving the manual snippet as a fallback. Follow the Usercentrics page (`usercentrics.mdx`) as the reference pattern: 1. **"Using the source package (recommended)"** heading first -- install command, usage code with imports, bullet list of what the source handles 2. **"Manual event listener"** heading -- preserve the existing manual snippet as a fallback option *** ## Mandatory checks[​](#mandatory-checks "Direct link to Mandatory checks") These are non-negotiable patterns every CMP source MUST follow. Violating any of these is a privacy compliance issue or a correctness bug. ### 1. Ensure every consent update includes explicit `false` for denied groups[​](#1-ensure-every-consent-update-includes-explicit-false-for-denied-groups "Direct link to 1-ensure-every-consent-update-includes-explicit-false-for-denied-groups") The collector uses merge semantics (`assign()`), so omitting a key means "no change," NOT "denied." Every consent state passed to `elb('walker consent', state)` must include explicit `false` for denied groups, not just `true` for granted ones. **Boolean-map CMPs** (CookieFirst, Usercentrics group-level): The CMP's consent object already contains explicit `false` values (e.g., `{ marketing: false, functional: true }`). These flow through naturally via iteration -- no extra code needed. **Presence-based CMPs** (CookiePro): Only active groups are listed (e.g., `",C0001,C0003,"`). Absence means denied. You MUST initialize ALL mapped groups to `false`, then set active ones to `true`: ``` // Presence-based CMPs: initialize all groups to false, then set active to true allMappedGroups.forEach((group) => { state[group] = false; }); activeIds.forEach((id) => { if (map[id]) state[map[id]] = true; }); ``` ### 2. Case-insensitive comparison for all CMP string fields[​](#2-case-insensitive-comparison-for-all-cmp-string-fields "Direct link to 2. Case-insensitive comparison for all CMP string fields") Use `.toLowerCase()`. CMP docs are inconsistent on casing. ``` // Explicit type check if (settings.explicitOnly && detail.type?.toLowerCase() !== 'explicit') return; // Category ID lookup const mapped = normalizedMap[id.toLowerCase()]; ``` ### 3. Apply `categoryMap` consistently in ALL code paths[​](#3-apply-categorymap-consistently-in-all-code-paths "Direct link to 3-apply-categorymap-consistently-in-all-code-paths") If there are multiple parsing branches (group-level vs service-level, or already-loaded vs event-listener), mapping MUST work identically in each. ### 4. Prevent dual-firing on consent events[​](#4-prevent-dual-firing-on-consent-events "Direct link to 4. Prevent dual-firing on consent events") Many CMPs fire multiple signals for the same consent action. If you don't guard against this, `handleConsent` fires twice per user action. Three known patterns: The collector's exactly-once delivery does not solve this for you: it guarantees each subscriber sees a given state version once, but two upstream CMP signals (a callback plus a DOM event) are two distinct consent commands, so both are delivered legitimately. Coalescing duplicate upstream signals is the source's job, as below. **Pattern A: Callback + event (CookiePro)** CMP fires both a callback (`OptanonWrapper`) and a DOM event (`OneTrustGroupsUpdated`) on the same action. Use the callback for init only and self-unwrap after first call: ``` actualWindow.OptanonWrapper = () => { if (originalWrapper) originalWrapper(); handleConsent(); actualWindow.OptanonWrapper = originalWrapper; // Self-unwrap }; ``` **Pattern B: Multiple change events (Cookiebot)** CMP fires separate events for accept, decline, and revoke. Each event carries the full consent state. Register the SAME handler for all change events -- no special dedup needed, but be aware of the multiplicity. **Pattern C: Init event re-fires on change (Usercentrics)** Single event type (`ucEvent`) fires for both init and change. No dual-firing risk, but the init/change distinction must come from event detail (e.g., `detail.type`), not event name. **Research step:** During Phase 1, fill in "Number of change events" in the decision matrix. If >1, determine which pattern applies and plan accordingly. ### 5. Document timing/race conditions in the README[​](#5-document-timingrace-conditions-in-the-readme "Direct link to 5. Document timing/race conditions in the README") What if the CMP loads before the source? What about `explicitOnly: false`? Each CMP has different timing behavior -- document it explicitly under "Timing considerations." For "previous-choice" detection (was this an active user decision or a first-visit default?), read the CMP's official consent metadata, the consent type or decision history (e.g. Usercentrics V3 `consent.type`, V2 the service consent history), not a per-pageload event `type` field. A per-pageload event `type` conflates first-visit defaults with a returning visitor's restored choice, so a returning visitor's prior consent would be dropped under `explicitOnly`. ### 6. Test consent revocation end-to-end[​](#6-test-consent-revocation-end-to-end "Direct link to 6. Test consent revocation end-to-end") Full grant -> revoke -> verify denied (explicit `false` values). This is the most common source of bugs. ``` test('handles consent withdrawal', async () => { // Initial: full consent mockWindow.__dispatchEvent('event', inputs.fullConsent); expect(consentCalls[0].consent).toEqual(outputs.fullConsentMapped); // User revokes marketing mockWindow.__dispatchEvent('event', inputs.partialConsent); expect(consentCalls[1].consent.marketing).toBe(false); // explicit false }); ``` ### 7. Use `MockWindow` interface in tests[​](#7-use-mockwindow-interface-in-tests "Direct link to 7-use-mockwindow-interface-in-tests") Properly typed, not `as unknown as` casts scattered through tests. Define one `MockWindow` interface in `test-utils.ts` that extends the source's **narrowed** `CmpWindow` (the surface the source actually touches), not the global `Window`. Because `Env.window` is narrowed to `CmpWindow`, the mock literal satisfies it directly — no `as unknown as Window` anywhere. ### 8. Store category keys in user-expected format[​](#8-store-category-keys-in-user-expected-format "Direct link to 8. Store category keys in user-expected format") Normalize during init for case-insensitive lookup, but store the original keys in the config so users see what they configured. ``` // Store original casing in config const mergedCategoryMap = { ...DEFAULT_CATEGORY_MAP, ...userMap }; // Build normalized lookup for internal use const normalizedMap: Record = {}; Object.entries(mergedCategoryMap).forEach(([key, value]) => { normalizedMap[key.toLowerCase()] = value; }); ``` ### 9. `walkerOS.json` convention[​](#9-walkerosjson-convention "Direct link to 9-walkerosjson-convention") Add `"walkerOS": { "type": "source", "platform": "web" }` to `package.json`. ### 10. README requirements[​](#10-readme-requirements "Direct link to 10. README requirements") Must include: Source Code/NPM/Documentation links, walkerOS.json section, Type definitions section, Related section, License section, sentence case headings, imports in all code examples, timing considerations section. *** ## Validation checklist[​](#validation-checklist "Direct link to Validation checklist") Beyond [understanding-development](/skills/walkeros-understanding-development.md) requirements (build, test, lint, no `any`): * All 10 mandatory checks pass * Research template fully filled * Decision matrix row complete * Examples include revocation and edge cases * Tests: 25-32 tests across 8-9 describe blocks * [ ] `MockWindow` interface in test-utils (not scattered casts) * Category mapping uses OR logic for many-to-one * Consent state always includes explicit `false` for denied groups * [ ] `destroy()` cleans up ALL listeners and restores wrapped functions * README follows required structure * Consent guide page updated *** ## Known limitations[​](#known-limitations "Direct link to Known limitations") The skill's source skeleton and code templates are based on DOM-event CMPs (CookieFirst, Usercentrics, CookiePro). CMPs that deviate significantly from this pattern will require adaptation: | Limitation | Affected CMPs | Workaround | | ---------------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Source skeleton assumes `addEventListener`** | Didomi (SDK array/method), Cookiebot (callback assignment) | Use the "Event registration patterns" table in Phase 1 to identify the correct pattern, then adapt the skeleton's listener setup and `destroy()` accordingly. | | **Consent read via property, not API method** | Didomi (`getCurrentUserStatus()`) | If consent is accessed via an API method call rather than a window property, wrap the call in `handleConsent` and adjust the "already loaded" detection path. | | **`destroy()` may not be possible** | CMPs with SDK readiness arrays (fire-once, no unsubscribe) | Document in README that the init callback cannot be removed. Only the change listener needs cleanup. | | **No vendor-level consent model** | Didomi (purposes + vendors as separate consent layers) | Use the "Dual-layer consent" guidance in Phase 3. walkerOS `Consent` is category-level; vendor-level consent requires flattening or a `consentLayer` setting. | These are research-phase decisions -- the skill's phases, mandatory checks, and validation checklist still apply. The research template and decision matrix capture these variations so they are identified early. *** ## Reference files[​](#reference-files "Direct link to Reference files") | What | Where | | ------------------ | ----------------------------------------------------------------------------------------------------------------- | | Skill examples | [examples/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/examples) | | Skill templates | [templates/cmp/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-cmp-source/templates/cmp) | | Canonical template | `packages/web/sources/cmps/cookiefirst/` | | Source types | `packages/core/src/types/source.ts` | | Consent guide | `website/docs/guides/consent/` | | Usercentrics plan | `docs/plans/2026-02-15-usercentrics-source.md` | | CookiePro plan | `docs/plans/2026-02-15-cookiepro-source.md` | ## Related skills[​](#related-skills "Direct link to Related skills") * [walkeros-create-source](/skills/walkeros-create-source.md) - General source creation workflow * [walkeros-understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface and push pattern * [walkeros-testing-strategy](/skills/walkeros-testing-strategy.md) - Testing with env mocking * [walkeros-writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards --- # Create a New Destination ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, read these skills: * [understanding-flow](/skills/walkeros-understanding-flow.md) - How destinations fit in architecture * [understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination interface * [understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer chaining to destinations * [understanding-mapping](/skills/walkeros-understanding-mapping.md) - Event transformation * [testing-strategy](/skills/walkeros-testing-strategy.md) - How to test with env pattern (Phase 3 + Phase 8) * [using-step-examples](/skills/walkeros-using-step-examples.md) - Authoritative pattern for `Flow.StepExample` structure and Three Type Zones * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards (for Phase 9) ## Choose Your Template[​](#choose-your-template "Direct link to Choose Your Template") | Complexity | Template | When to Use | | ---------- | ------------ | ----------------------------------- | | Simple | `plausible/` | Single SDK call, minimal config | | Complex | `gtag/` | Multiple services, sub-destinations | | Server | `gcp/` | Server-side, batching, SDK init | ## Process Overview[​](#process-overview "Direct link to Process Overview") ``` 1. Research → Deeply understand vendor SDK, API, and event taxonomy 2. Classify → Determine vendor taxonomy type and integration approach 3. Examples → Define in/out pairs FIRST (start with the end result) 4. Mapping → Define walkerOS → vendor transformation 5. Scaffold → Copy template and configure 6. Convention → Add walkerOS.json metadata and buildDev 7. Implement → Build using examples as test fixtures 8. Test → Verify against example variations 9. Document → Write README ``` *** ## Phase 1: Research[​](#phase-1-research "Direct link to Phase 1: Research") **Goal:** Deeply understand the vendor SDK before writing any code. Research quality determines implementation quality. ### 1.1 Find and Install Official SDK[​](#11-find-and-install-official-sdk "Direct link to 1.1 Find and Install Official SDK") Always prefer the vendor's official SDK package over raw HTTP API calls. The SDK handles transport, batching, retries, and plugin ecosystems — don't reinvent these. * [ ] **Install the SDK** — `npm install @vendor/sdk` and read the actual source * [ ] **Read TypeScript types** — Import types from the SDK package directly. Never duplicate type definitions. This ensures IntelliSense completeness and consistency with SDK updates. * [ ] **Understand the full API surface** — List every public method, not just `track()`. Identity methods, property operations, group management, specialized event types (revenue, etc.) ``` # Search npm for official packages npm search [vendor-name] npm search @[vendor] # Install and inspect actual types npm install @vendor/analytics-browser ls node_modules/@vendor/analytics-browser/lib/esm/ ``` ### 1.2 Understand SDK Architecture[​](#12-understand-sdk-architecture "Direct link to 1.2 Understand SDK Architecture") * [ ] **Init options** — What does `init()` accept? What can be configured? * [ ] **Call ordering** — Can you call methods before `init()`? Is there internal queuing? What are the race condition implications? * [ ] **Plugin system** — Does the SDK support plugins? How are they added? * [ ] **Identity management** — How does the SDK handle user/device/session IDs? Does it manage cookies/storage? Can this be disabled (since walkerOS manages identity)? * [ ] **Consent** — Does the SDK have a consent mode or just binary opt-out? ### 1.3 Identify All Event Methods[​](#13-identify-all-event-methods "Direct link to 1.3 Identify All Event Methods") Go beyond just `track()`. Most SDKs have specialized methods: | Method Category | Examples | walkerOS Handling | | ---------------- | ----------------------------------- | ------------------- | | Event tracking | `track()`, `logEvent()` | Default `push()` | | User properties | `identify()`, `setUserProperties()` | `mapping.settings` | | Revenue/purchase | `revenue()`, `purchase()` | `mapping.settings` | | Groups/accounts | `setGroup()`, `groupIdentify()` | `mapping.settings` | | Identity setters | `setUserId()`, `setDeviceId()` | `settings.identify` | | Opt-out | `setOptOut()`, `consent()` | `on('consent')` | | Cleanup | `flush()`, `reset()` | `destroy()` | ### 1.4 Check Existing Patterns[​](#14-check-existing-patterns "Direct link to 1.4 Check Existing Patterns") Review similar destinations in the codebase: ``` ls packages/web/destinations/ ``` ### Gate: Research Complete[​](#gate-research-complete "Direct link to Gate: Research Complete") * Official SDK installed and types inspected * All public methods listed with their purpose * Init options understood (what to disable, what to pass through) * Call ordering / race conditions understood * Identity, consent, and plugin patterns documented ### Checkpoint: Research Review (Optional)[​](#checkpoint-research-review-optional "Direct link to Checkpoint: Research Review (Optional)") If working with human oversight, pause here to confirm: * SDK integration approach correct? * Event mapping makes sense for the use case? * Any vendor quirks or limitations to handle? *** ## Phase 2: Classify Vendor Taxonomy[​](#phase-2-classify-vendor-taxonomy "Direct link to Phase 2: Classify Vendor Taxonomy") **Goal:** Understand what the vendor expects, which determines destination complexity. ### 2.1 Taxonomy Types[​](#21-taxonomy-types "Direct link to 2.1 Taxonomy Types") | Type | Description | Mapping Needed | Example Vendors | | ---------------- | ---------------------------------------------- | -------------------------------------- | --------------------- | | **Free-form** | Any event name accepted, no prescribed schema | Minimal — names pass through | Most modern analytics | | **Rigid** | Prescribed event names unlock specific reports | Essential — must map to exact names | Some legacy analytics | | **Schema-based** | Self-describing events with formal schemas | Structural — must build schema objects | Data warehouse tools | ### 2.2 Determine Integration Approach[​](#22-determine-integration-approach "Direct link to 2.2 Determine Integration Approach") | Approach | When to use | Pattern | | -------------------------- | ---------------------------------- | ---------------------------------------------------------- | | **Vendor SDK as host** | SDK has plugins, batching, retries | Load SDK, disable what walkerOS replaces, use as transport | | **Script + command queue** | SDK loaded via script tag | Load script, use global function queue | | **HTTP API** | No SDK available, or server-side | Direct HTTP calls via `sendWeb()` or `fetch` | **Prefer the vendor SDK** — it handles transport, retries, and plugin orchestration. HTTP API is a fallback when no SDK exists. When using the vendor SDK: * Disable features walkerOS replaces (autocapture, identity storage) **by default** — but allow users to re-enable via settings * Pass through all SDK init options for full configurability * Import types from the SDK package directly ### Gate: Classification Complete[​](#gate-classification-complete "Direct link to Gate: Classification Complete") * Taxonomy type identified (free-form / rigid / schema-based) * Integration approach chosen (SDK / script / HTTP) * Know which SDK features walkerOS replaces vs. uses *** ## Phase 3: Create Examples (BEFORE Implementation)[​](#phase-3-create-examples-before-implementation "Direct link to Phase 3: Create Examples (BEFORE Implementation)") **Mandatory.** Examples are the test fixtures for Phase 8. Define expected `in` / `out` / `mapping` triples FIRST — start with the end result in mind. Without examples, you cannot test. Even for free-form vendors where no mapping is strictly "required," step examples serve as the single source of truth for tests, simulations, and documentation. > **Authoritative pattern:** See [using-step-examples](/skills/walkeros-using-step-examples.md) for the Three Type Zones and lifecycle. This skill reuses that contract — do not diverge. ### 3.1 Scaffold Directory Structure[​](#31-scaffold-directory-structure "Direct link to 3.1 Scaffold Directory Structure") ``` mkdir -p packages/web/destinations/[name]/src/examples mkdir -p packages/web/destinations/[name]/src/{schemas,types} ``` ### 3.2 Required Files (3 files total)[​](#32-required-files-3-files-total "Direct link to 3.2 Required Files (3 files total)") All seven reference web destinations (gtag, meta, snowplow, plausible, piwikpro, api, demo) use exactly three files in `src/examples/`. Match this structure — no `events.ts`, `outputs.ts`, or standalone `mapping.ts`. | File | Purpose | | ------------------- | ----------------------------------------------------------------- | | `examples/env.ts` | Mock environment for testing (no real network calls) | | `examples/step.ts` | `Flow.StepExample` entries with `in` / `out` / `mapping?` triples | | `examples/index.ts` | Barrel exports: `env` and `step` | The `step.ts` file embeds the input event, the mapping config, and the expected vendor output together in one `Flow.StepExample` — subsuming what older skills described as separate `events.ts` / `outputs.ts` / `mapping.ts` files. ### 3.3 Typing Rules (strict)[​](#33-typing-rules-strict "Direct link to 3.3 Typing Rules (strict)") **No `any`.** Every example value must be explicitly typed. * **Inputs** use `WalkerOS.Event` (via `getEvent()` from `@walkeros/core`) — never hand-roll event literals. * **Outputs** use **vendor SDK types** imported from the official package whenever the vendor publishes them. Do not invent local output types for payloads the vendor already types (e.g. use Meta Pixel's `fbq` argument types, not a local `FbqCall` interface). * **Step entries** are typed `Flow.StepExample` from `@walkeros/core`. * **Mock env** is typed against the destination's local `Env` type from `../types`. No `as any`, no untyped `{}`. * Vendor SDK types come from the SDK you installed in Phase 1 — reuse them rather than duplicating shapes. ### 3.4 Code Template — `examples/step.ts`[​](#34-code-template--examplesstepts "Direct link to 34-code-template--examplesstepts") ``` import type { Flow, WalkerOS } from '@walkeros/core'; import { getEvent } from '@walkeros/core'; // One step example per supported feature / setting. // `in` is a WalkerOS.Event (use getEvent for deterministic fixtures). // `out` is the vendor-specific call we expect the destination to produce — // typed against the vendor SDK's published types where available. // `mapping` is the mapping rule under test (optional — omit for default push). // Set `title` + `description` for public examples; mark test-only fixtures // with `public: false`. See // [walkeros-using-step-examples](../walkeros-using-step-examples/SKILL.md). export const purchase: Flow.StepExample = { in: getEvent('order complete', { timestamp: 1700000100 }), mapping: { name: 'purchase', data: { map: { transaction_id: 'data.id', value: 'data.total', currency: { key: 'data.currency', value: 'EUR' }, }, }, }, out: [ 'event', 'purchase', { transaction_id: '0rd3r1d', value: 555, currency: 'EUR', }, ], }; export const pageView: Flow.StepExample = { in: getEvent('page view', { timestamp: 1700000102 }), mapping: undefined, out: ['event', 'page_view', { send_to: 'G-XXXXXX-1' }], }; // For destinations that handle consent updates, use the command field // to route `in` through elb('walker consent', in) instead of an event push: export const consentGranted: Flow.StepExample = { command: 'consent', in: { marketing: true, functional: true }, out: ['consent', 'update', { ad_storage: 'granted' }], }; ``` #### Init step example[​](#init-step-example "Direct link to Init step example") Every destination ships an `examples.step.init` entry — the init is a first-class step example, not a hidden side effect. * `in` mirrors the real `Destination.Config` shape users copy-paste — typically `{ loadScript: true, settings: { /* vendor-specific */ } }`. Whatever a user would configure in their flow goes here verbatim. * `out` is the ordered list of vendor calls the `init()` lifecycle produces (script tags, SDK initializers, queue setup). Each effect tuple follows the standard `[callable, ...args]` shape. **Test pattern.** Call `destination.init({ id, config, env, logger, collector })` directly in the test — no capture helpers, no `capture.ts`, no allowlists. Assert the captured vendor calls equal `examples.step.init.out`: ``` const calls: unknown[][] = []; const env = wrapEnv(examples.env.init, (call) => calls.push(call)); await destination.init({ id: 'test', config: examples.step.init.in as Destination.Config, env, logger, collector, }); expect(calls).toEqual(examples.step.init.out); ``` **Event step tests** bootstrap once with `examples.step.init.in`, then slice the shared capture buffer to isolate push effects from init effects: ``` const pushCalls = calls.slice(examples.step.init.out.length); expect(pushCalls).toEqual(example.out); ``` There are no hand-maintained allowlists or `isInitEffect` filters — the init example's `out.length` is the single source of truth for how many effects belong to init. **Multi-tool packages** (like `gtag`, which drives GA4, Google Ads, and GTM) ship **per-tool init examples** — `examples.step.ga4Init`, `adsInit`, `gtmInit` — instead of a single `init`. The docs render each on its own page via ``, and tests pick the right init per sub-tool. For destinations, the Three Type Zones collapse to: * `in` = walkerOS event (`WalkerOS.Event`) * `out` = vendor output (typed against vendor SDK) * `mapping` = rule under test (optional) ### 3.5 `examples/index.ts` (barrel)[​](#35-examplesindexts-barrel "Direct link to 35-examplesindexts-barrel") ``` export * as env from './env'; export * as step from './step'; ``` ### 3.6 `examples/env.ts`[​](#36-examplesenvts "Direct link to 36-examplesenvts") Mock the vendor SDK surface and any DOM touchpoints. Never reach real network, real cookies, or real globals. Type the exports against your local `Env`: ``` import type { Env } from '../types'; export const init: Env | undefined = { /* pre-init state (vendor SDK not yet loaded) */ }; export const push: Env = { /* post-init state used for push() tests */ }; ``` ### 3.7 Test Fixture Contract (hard rule)[​](#37-test-fixture-contract-hard-rule "Direct link to 3.7 Test Fixture Contract (hard rule)") The examples authored here **are** the Phase 8 test fixtures. No parallel fixtures allowed. * `src/index.test.ts` (or `src/__tests__/stepExamples.test.ts`) **MUST** iterate examples via `it.each(Object.entries(examples.step))`. * Tests **must NOT** contain hardcoded payloads, vendor configs, or expected outputs. * If a test needs a value that is not in `examples.step`, **add it to `step.ts` first**, then consume it from the test. Never inline test data. * The only per-test setup allowed is deriving destination `settings` from the example's `mapping.settings` (e.g. enabling the right sub-tool). See `packages/web/destinations/gtag/src/__tests__/stepExamples.test.ts` for a canonical reference. ### 3.8 Export via `dev.ts`[​](#38-export-via-devts "Direct link to 38-export-via-devts") ``` export * as schemas from './schemas'; export * as examples from './examples'; ``` ### Phase 3 Acceptance Checklist[​](#phase-3-acceptance-checklist "Direct link to Phase 3 Acceptance Checklist") * [ ] `src/examples/env.ts` — mock env, no real network, typed against local `Env` * [ ] `src/examples/step.ts` — one `Flow.StepExample` per supported feature / setting, typed `in` / `out` / `mapping?` * [ ] `src/examples/index.ts` — barrel exports `env` and `step` * No standalone `events.ts`, `outputs.ts`, or `mapping.ts` files * All vendor SDK types imported from the official package — no `any`, no reinvented local output types * [ ] `src/index.test.ts` (or `__tests__/stepExamples.test.ts`) iterates `examples.step` via `it.each(Object.entries(...))` * Tests contain zero hardcoded payloads / vendor configs / expected outputs — everything flows from `examples.step` * [ ] `npm run build` passes — examples compile against published types * Each example traces: `in` → apply `mapping` → matches `out` *** ## Phase 4: Define Mapping[​](#phase-4-define-mapping "Direct link to Phase 4: Define Mapping") **Goal:** Document transformation from walkerOS events to vendor format. Mapping rules live **inside** each `Flow.StepExample` entry in `step.ts` — no separate `mapping.ts` file. Each step example embeds the exact mapping rule under test alongside its `in` event and expected `out` output. ### Verify Mapping Logic[​](#verify-mapping-logic "Direct link to Verify Mapping Logic") For each entry in `step.ts`, trace: ``` Input: examples.step.purchase.in (WalkerOS.Event) ↓ Apply examples.step.purchase.mapping ↓ name transforms, data.map applied Output: Should match examples.step.purchase.out ``` ### Gate: Mapping Verified[​](#gate-mapping-verified "Direct link to Gate: Mapping Verified") * Step examples cover: page view + at least one conversion event * One step example per supported setting / sub-tool * Each `mapping` traces correctly from `in` to `out` *** ## Phase 5: Scaffold[​](#phase-5-scaffold "Direct link to Phase 5: Scaffold") **Template destination:** `packages/web/destinations/plausible/` ``` cp -r packages/web/destinations/plausible packages/web/destinations/[name] cd packages/web/destinations/[name] # Update package.json: name, description, repository.directory ``` **Directory structure:** ``` packages/web/destinations/[name]/ ├── src/ │ ├── index.ts # Main destination (init + push) │ ├── index.test.ts # Tests against examples │ ├── dev.ts # Exports schemas and examples │ ├── examples/ │ ├── schemas/ │ └── types/ ├── package.json ├── tsconfig.json ├── tsup.config.ts ├── jest.config.mjs └── README.md ``` ### Transformer Chain Integration[​](#transformer-chain-integration "Direct link to Transformer Chain Integration") Destinations can wire to transformer chains via `before` in the init config: ``` destinations: { myDestination: { code: destinationMyDestination, config: { settings: { /* ... */ } }, before: 'redact' // Events go through redactor before this destination } } ``` *** ## Phase 6: walkerOS.json Convention[​](#phase-6-walkerosjson-convention "Direct link to Phase 6: walkerOS.json Convention") Every walkerOS package ships a `walkerOS.json` file for CDN-based schema discovery. ### Add `walkerOS` field to package.json[​](#add-walkeros-field-to-packagejson "Direct link to add-walkeros-field-to-packagejson") ``` { "walkerOS": { "type": "destination", "platform": "web" }, "keywords": ["walkerOS", "walkerOS-destination", ...] } ``` ### Use `buildDev()` in tsup.config.ts[​](#use-builddev-in-tsupconfigts "Direct link to use-builddev-in-tsupconfigts") Replace `buildModules({ entry: ['src/dev.ts'] })` with `buildDev()`: ``` import { buildDev } from '@walkeros/config/tsup'; // In defineConfig array: buildDev(), ``` This auto-generates `dist/walkerOS.json` from your Zod schemas at build time. ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") If your destination has capabilities, behaviors, or troubleshooting patterns not obvious from schemas alone, add hints. See `walkeros-writing-documentation` skill for full guidelines. Create `src/hints.ts`: ``` import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'auth-methods': { text: 'Supports X, Y, and Z auth methods. See settings schema for all options.', code: [{ lang: 'json', code: '{ "settings": { ... } }' }], }, }; ``` Export from `src/dev.ts`: ``` export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Guidelines: * Expand awareness — describe capabilities ("supports X, Y, Z"), don't prescribe one path * Reference schemas and examples, don't duplicate them * Verify every claim against actual implementation before publishing * Key naming: kebab-case, group with prefixes (auth-\*, storage-\*, troubleshoot-\*) * Most destinations don't need hints — schemas and examples cover the common case ### Gate: Convention Met[​](#gate-convention-met "Direct link to Gate: Convention Met") * [ ] `walkerOS` field in package.json with type and platform * [ ] `buildDev()` in tsup.config.ts * Build generates `dist/walkerOS.json` * Keywords include `walkerOS` and `walkerOS-destination` ### Runtime-only npm dependencies[​](#runtime-only-npm-dependencies "Direct link to Runtime-only npm dependencies") If your package wraps a third-party npm dep that **cannot be ESM-bundled** (uses `__dirname`, ships a `.node` binary, etc.), declare it under `walkerOS.bundle.external` in your `package.json`. See [walkeros-using-cli → Bundle externals](/skills/walkeros-using-cli.md#bundle-externals-per-package-walkerosbundleexternal) for the complete contract. *** ## Phase 7: Implement[​](#phase-7-implement "Direct link to Phase 7: Implement") **Now write code to produce the outputs defined in Phase 3.** ### Template Files[​](#template-files "Direct link to Template Files") Use these templates as your starting point: | File | Purpose | Template | | ---------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------ | | `types/index.ts` | Type definitions | [types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-destination/templates/simple/types.ts) | | `index.ts` | Main destination | [index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-destination/templates/simple/index.ts) | ### Key Patterns[​](#key-patterns "Direct link to Key Patterns") 1. **Init receives context**: Destructure `config`, `env`, `logger`, `id` from context 2. **Push receives context**: Includes `data`, `rule` (renamed from `mapping`), `ingest` 3. **Use `getEnv(env)`**: Never access `window`/`document` directly, and never cast them. Declare your SDK global (`declare global { interface Window { vendorSdk?: ... } }`) and narrow `Env['window']` to the SDK shape in `types.ts`. The generic `getEnv(env)` then returns your narrowed env intersected with the real DOM globals, so `window.vendorSdk` and `document.createElement` are fully typed with zero casts. No `as any`, `as unknown`, `as Window`, `as Document`, `@ts-ignore`, or `@ts-expect-error` is allowed in production **or** test code, ever. If something will not type check, fix the types, do not cast. 4. **Return config from init**: Allows updating config during initialization 5. **Optional `destroy` method**: Implement if the destination holds resources (DB connections, SDK clients, timers) that need cleanup on shutdown. Call `flush()` or equivalent on the vendor SDK. 6. **Mapping-based settings** (recommended): Use standard walkerOS mapping values (`map`, `loop`, `key`, `value`, `condition`) in `mapping.settings.*` for vendor-specific operations. Resolve via `getMappingValue()` in `push()` and interpret the resolved object's keys as SDK method instructions. This keeps config agnostic and reuses the mapping engine. 7. **Consent two-layer**: `config.consent` gates walkerOS event delivery. `on('consent')` controls vendor SDK internals (opt-out, pause capture, etc.). Both needed for complete consent compliance. * For step-example tests, use `command: 'consent'` on `Flow.StepExample` to invoke the `on('consent')` handler. Do not push consent data as an event. ### Reserved fields[​](#reserved-fields "Direct link to Reserved fields") `config.setup` is reserved for the setup lifecycle (see [Adding setup (optional)](#adding-setup-optional)). Do not use the key `setup` for unrelated package metadata or hint keys. The framework wires this field to `SetupFn` via `resolveSetup`, and `walkeros setup destination.` reads it. Repurposing the name will collide with that wiring. ### Gate: Implementation Compiles[​](#gate-implementation-compiles "Direct link to Gate: Implementation Compiles") * [ ] `npm run build` passes * [ ] `npm run verify:touched -- ` passes (L1: typecheck + lint + test) *** ## Adding setup (optional)[​](#adding-setup-optional "Direct link to Adding setup (optional)") A destination package can implement an optional `setup()` function to provision external resources idempotently: BigQuery datasets and tables, Pub/Sub topics, SQLite tables, warehouse schemas, S3 buckets, webhook registrations on downstream platforms. Setup runs only when an operator explicitly types `walkeros setup destination.`. The runtime never auto-invokes it from `init()`, `push()`, or `destroy()`. The framework provides the slot, the CLI command, and a `resolveSetup` helper. The package owns: what setup means, idempotency, error handling, return value. For background on how setup fits the destination lifecycle, see [understanding-destinations](/skills/walkeros-understanding-destinations.md#setup-optional). ### Types[​](#types "Direct link to Types") ``` // types/index.ts import type { CoreDestination } from '@walkeros/core'; export interface Settings { /* runtime push settings */ } export interface InitSettings { /* one-time init settings */ } export interface Mapping { /* per-event mapping config */ } export interface Env { /* injected platform deps (SDK clients, etc.) */ } // The package's own setup options interface. // Becomes the U slot of Types; surfaces as `config.setup: boolean | Setup` for users. export interface Setup { // package-specific provisioning options // e.g. for BigQuery: location, partitioning, clustering, schema location?: string; partitioning?: { field: string; type: 'DAY' | 'HOUR' }; } // Service-account / key-pair credentials. Becomes the trailing C slot of Types; // surfaces as the optional top-level `config.credentials` (sibling of `settings`), // $env-resolvable and validated per package. export type Credentials = CoreDestination.Credential; export type Types = CoreDestination.Types< Settings, Mapping, Env, InitSettings, Setup, Credentials >; ``` **Where credentials belong.** Put service-account / key-pair credentials (the kind paired with `$env.NAME` to inject a secret) on the strictly-typed top-level `config.credentials`, not inside `settings`. Read them via `config.credentials ?? settings.credentials ?? ADC` so existing flows keep working, and omit them to fall back to Application Default Credentials. The raw `settings.` passthrough (e.g. `settings.bigquery`) stays as the lower-level escape hatch for SDK-specific auth options. The package-specific `settings.credentials` still works but is deprecated. ### Implementation[​](#implementation "Direct link to Implementation") ``` // setup.ts import type { CoreDestination, SetupFn } from '@walkeros/core'; import { resolveSetup } from '@walkeros/core'; import type { Setup, Types } from './types'; const DEFAULT_SETUP: Setup = { location: 'EU', }; export const setup: SetupFn< CoreDestination.Config, CoreDestination.Env > = async ({ config, env, logger }) => { const options = resolveSetup(config.setup, DEFAULT_SETUP); if (!options) return; // config.setup is false or unset // Package-specific provisioning, idempotent. // Returning a structured object (e.g. { datasetCreated: true }) // makes that data available to operators via `walkeros setup ... | jq`. }; ``` Wire it in your default export: ``` // index.ts import { setup } from './setup'; export default { type: 'my-destination', push: /* ... */, setup, }; ``` ### When to implement[​](#when-to-implement "Direct link to When to implement") Implement `setup()` when your destination needs first-time provisioning of external resources before events can flow: warehouse tables, Pub/Sub topics, S3 buckets, schema bindings, IAM roles, webhook registrations on downstream platforms. Skip it when your destination only consumes credentials or HTTP endpoints the user already provisioned. ### Contract[​](#contract "Direct link to Contract") * Triggered only by `walkeros setup .`. Never by runtime push, init, or destroy. * **Idempotency is your responsibility.** Re-running setup against a fully provisioned environment must be a safe no-op. Use try-create-catch-409 on REST APIs, `IF NOT EXISTS` on SQL, native idempotent operations where available. The framework does not retry, track state, or detect drift. * Return structured data from `setup()` when useful for operator scripting. The CLI emits non-undefined return values as JSON to stdout. * For packages where `setup: true` (boolean form) is meaningless because mandatory fields have no safe defaults (e.g., Kafka `numPartitions`, GitHub webhook `webhookUrl`), reject the boolean form with a clear runtime error listing required fields: ``` if (config.setup === true) { throw new Error( 'kafka destination setup requires explicit options: ' + '{ topic, numPartitions, replicationFactor }. There is no safe default.', ); } ``` *** ## Phase 8: Test Against Examples[​](#phase-8-test-against-examples "Direct link to Phase 8: Test Against Examples") > Tests verify implementation against the examples from Phase 3. If examples are incomplete, tests will be incomplete. > > See [testing-strategy](/skills/walkeros-testing-strategy.md) for the shared env / dev-examples conventions this phase depends on. **Verify implementation produces expected outputs.** ### Test Template[​](#test-template "Direct link to Test Template") Use the test template: [index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-destination/templates/simple/index.test.ts). Reference canonical implementation: `packages/web/destinations/gtag/src/__tests__/stepExamples.test.ts`. ### Key Test Patterns[​](#key-test-patterns "Direct link to Key Test Patterns") 1. **`it.each(Object.entries(examples.step))` is mandatory** — one iteration per step example. Do not write per-feature tests with hand-rolled payloads. 2. **Use `createPushContext()` helper** — standardizes context creation. 3. **Include `id` field** — required in context. 4. **Use `rule` instead of `mapping`** — property renamed in `PushContext`. 5. **Zero hardcoded payloads** — every input, vendor config, and expected output comes from `examples.step` or `examples.env`. If you need something new, add it to examples first. 6. **Clone env per test** — `clone(examples.env.push)` so vendor mocks don't leak across iterations. ### Gate: Tests Pass[​](#gate-tests-pass "Direct link to Gate: Tests Pass") * [ ] `npm run verify:touched -- ` passes (L1) * Tests iterate via `it.each(Object.entries(examples.step))` * Tests contain no hardcoded payloads, vendor configs, or expected outputs * Every assertion reads from `examples.step[...].out` *** ## Phase 9: Document[​](#phase-9-document "Direct link to Phase 9: Document") Follow the [writing-documentation](/skills/walkeros-writing-documentation.md) skill for: * README structure and templates * Example validation against `apps/quickstart/` * Quality checklist before publishing Key requirements for destination documentation: * Event mapping table (walkerOS → vendor format) * Configuration options table (use PropertyTable if schema exists) * Working code example with imports * Installation instructions *** ## Validation Checklist[​](#validation-checklist "Direct link to Validation Checklist") Beyond [understanding-development](/skills/walkeros-understanding-development.md) requirements (build, test, lint, no `any`): * Uses `getEnv(env)` pattern (never direct `window`/`document` access, never `as Window`/`as Document`/`as unknown` casts in src or tests) * [ ] `dev.ts` exports `schemas` and `examples` * Examples match type signatures * Tests use examples for assertions (not hardcoded values) * [ ] `walkerOS.json` generated at build time * [ ] `walkerOS` field in package.json *** ## Reference Files[​](#reference-files "Direct link to Reference Files") | What | Where | | --------------- | -------------------------------------------- | | Simple template | `packages/web/destinations/plausible/` | | Complex example | `packages/web/destinations/gtag/` | | Types | `packages/web/core/src/types/destination.ts` | ## Related Skills[​](#related-skills "Direct link to Related Skills") * [understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination interface and env pattern * [using-step-examples](/skills/walkeros-using-step-examples.md) — Authoritative `Flow.StepExample` pattern and Three Type Zones * [testing-strategy](/skills/walkeros-testing-strategy.md) — Testing with env mocking and dev-examples-as-fixtures conventions * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards --- # Create a New Source ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, read these skills: * [understanding-flow](/skills/walkeros-understanding-flow.md) - How sources fit in architecture * [understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface * [understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer chaining from sources * [understanding-events](/skills/walkeros-understanding-events.md) - Event structure sources emit * [understanding-mapping](/skills/walkeros-understanding-mapping.md) - Transform raw input to events * [testing-strategy](/skills/walkeros-testing-strategy.md) - How to test with env pattern and dev examples * [using-step-examples](/skills/walkeros-using-step-examples.md) - Authoritative `Flow.StepExample` pattern, `createTrigger`, Three Type Zones * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards (for Phase 9) ## Source Types[​](#source-types "Direct link to Source Types") A source's identity is split into two fields: | Field | Meaning | Examples | | ----------------- | --------------------------------------------------------------- | ---------------------------------------------- | | `source.type` | The kind of source (its role / mechanism) | `browser`, `dataLayer`, `cookiefirst`, `fetch` | | `source.platform` | The runtime that hosts the source (`web`, `server`, `app`, ...) | `web`, `server` | | Platform | Input | Example types | | -------- | ----------------------- | ----------------------------------- | | `web` | DOM events, dataLayer | `browser`, `dataLayer` | | `server` | HTTP requests, webhooks | `gcp`, `express`, `lambda`, `fetch` | ## Augmenting `SourceMap`[​](#augmenting-sourcemap "Direct link to augmenting-sourcemap") Every source registers its `type` literal and any source-specific `source.*` fields by augmenting `SourceMap` from `@walkeros/core`. This is how the collector and downstream consumers know about the new source kind without loosening the union to `string`. Add this to the source's `src/types.ts` (or `src/types/index.ts`): ``` import type { Source, Elb } from '@walkeros/core'; declare module '@walkeros/core' { interface SourceMap { // Replace `mySource` with the source's package-level identifier. mySource: { type: 'mySource'; // matches the literal you return from Source.Init platform?: 'web'; // 'web' | 'server' | 'app' | ... // Add any extra fields the source surfaces in `event.source.*` here. // e.g. `version?: string;` is already on the base Source - only add // truly source-specific keys. }; } } ``` Reference implementations: `packages/web/sources/browser/src/types/index.ts`, `packages/web/sources/demo/src/types.ts`. Conflicting declarations cause compile errors on purpose, this surfaces naming collisions early. ## Source Categories[​](#source-categories "Direct link to Source Categories") | Category | Purpose | Examples | Key Concern | | ------------------ | ----------------------------------------- | ----------------------- | -------------------- | | **Transformation** | Convert external format → walkerOS events | `dataLayer`, `fetch` | Mapping accuracy | | **Transport** | Receive events from specific platform | `gcp`, `aws`, `express` | Platform integration | ## Choose Your Template[​](#choose-your-template "Direct link to Choose Your Template") | Complexity | Template | When to Use | | --------------------- | -------------- | ------------------------------------- | | Simple transformation | `fetch/` | Generic HTTP handler, data conversion | | Platform transport | `gcp/`, `aws/` | Cloud platform integration | | Browser interception | `dataLayer/` | DOM events, array interception | ## Process Overview[​](#process-overview "Direct link to Process Overview") ``` 1. Research → Deeply understand external system, SDK, and data format 2. Classify → Determine source type and integration approach 3. Examples → Define in/out pairs FIRST (start with the end result) 4. Mapping → Define input → walkerOS event transformation 5. Scaffold → Copy template and configure 6. Convention → Add walkerOS.json metadata and buildDev 7. Implement → Build using examples as test fixtures 8. Test → Verify against example variations 9. Document → Write README ``` *** ## Phase 1: Research[​](#phase-1-research "Direct link to Phase 1: Research") **Goal:** Deeply understand the external system before writing any code. Research quality determines implementation quality. ### 1.1 Find and Install Official SDK[​](#11-find-and-install-official-sdk "Direct link to 1.1 Find and Install Official SDK") Always prefer the vendor's official SDK package over raw HTTP API calls. The SDK handles transport, data formatting, and platform specifics - don't reinvent these. * [ ] **Install the SDK** - `npm install @vendor/sdk` and read the actual source * [ ] **Read TypeScript types** - Import types from the SDK package directly. Never duplicate type definitions. This ensures IntelliSense completeness and consistency with SDK updates. * [ ] **Understand the full API surface** - List every public method and type export. What data structures does the platform provide? What request/event formats exist? ``` # Search npm for official packages npm search [vendor-name] npm search @[vendor] # Install and inspect actual types npm install @vendor/sdk ls node_modules/@vendor/sdk/lib/esm/ ``` ### 1.2 Understand SDK Architecture[​](#12-understand-sdk-architecture "Direct link to 1.2 Understand SDK Architecture") * [ ] **Init options** - What does the SDK expose? How is the platform connection established? * [ ] **Call ordering** - When does data arrive? Is it pushed (webhooks, callbacks) or pulled (polling, intercepting)? What are the timing implications? * [ ] **Data format** - What does the raw event/request look like? Headers, body structure, query params, authentication tokens? * [ ] **Identity signals** - Does the external system provide user IDs, session IDs, device IDs? How are they delivered (headers, cookies, body fields)? * [ ] **Consent** - Does the platform have consent signals? How are they communicated? ### 1.3 Identify All Data Entry Points[​](#13-identify-all-data-entry-points "Direct link to 1.3 Identify All Data Entry Points") Go beyond just the primary event payload. Most external systems provide multiple data channels: | Data Channel | Examples | walkerOS Handling | | ---------------- | ----------------------------------- | -------------------- | | Event payload | Request body, DOM event data | Default `push()` | | Headers/metadata | Auth tokens, content-type, origin | `context` or `user` | | Query params | UTM parameters, tracking IDs | `data` or `context` | | Platform context | Cloud function metadata, Lambda ctx | `source` or `custom` | | Identity | User ID, session ID, device ID | `user` | | Consent signals | Opt-in/out flags, consent string | `consent` | ### 1.4 Check Existing Patterns[​](#14-check-existing-patterns "Direct link to 1.4 Check Existing Patterns") Review similar sources in the codebase: ``` # List existing sources ls packages/web/sources/ ls packages/server/sources/ # Reference implementations # - dataLayer: DOM-based, array interception # - express: HTTP middleware # - fetch: Generic HTTP handler (simplest server pattern) # - gcp: Cloud Functions specific ``` ### Gate: Research Complete[​](#gate-research-complete "Direct link to Gate: Research Complete") * Official SDK installed and types inspected (or HTTP API documented if no SDK exists) * All data entry points listed with their format * Init options and call ordering understood * Identity and consent signal delivery documented * Raw event/request structure mapped to walkerOS event fields ### Checkpoint: Research Review (Optional)[​](#checkpoint-research-review-optional "Direct link to Checkpoint: Research Review (Optional)") If working with human oversight, pause here to confirm: * SDK integration approach correct? * Data capture strategy makes sense for the use case? * Any platform quirks or limitations to handle? *** ## Phase 2: Classify Source Type[​](#phase-2-classify-source-type "Direct link to Phase 2: Classify Source Type") **Goal:** Understand what the source captures and how it delivers data, which determines implementation complexity. ### 2.1 Source Categories[​](#21-source-categories "Direct link to 2.1 Source Categories") | Category | Description | Mapping Needed | Example Sources | | ------------------ | ------------------------------------------ | ------------------------------- | ------------------------ | | **Transformation** | Converts external event format to walkerOS | Essential - must map fields | `dataLayer`, `fetch` | | **Transport** | Receives events from a specific platform | Structural - platform unwrap | `gcp`, `aws`, `express` | | **Interception** | Intercepts existing data flows | Varies - depends on data format | `dataLayer`, CMP sources | ### 2.2 Determine Integration Approach[​](#22-determine-integration-approach "Direct link to 2.2 Determine Integration Approach") | Approach | When to use | Pattern | | --------------------------- | ----------------------------------- | --------------------------------------------------------- | | **Platform SDK as host** | SDK provides typed request/response | Use SDK types, wrap handler in walkerOS source | | **DOM interception** | Capture browser-side events | Listen to DOM events, intercept arrays/globals | | **HTTP handler** | Generic webhook/API receiver | Parse request, extract events, forward to collector | | **Callback/event listener** | Platform provides event emitter | Register listener, transform events, forward to collector | **Prefer the vendor SDK** - it provides typed request/response objects and handles platform specifics. Raw HTTP parsing is a fallback when no SDK exists. When using the vendor SDK: * Import types from the SDK package directly * Use SDK request/response types for handler signatures * Let the SDK handle platform-specific parsing (body parsing, header extraction) ### Gate: Classification Complete[​](#gate-classification-complete "Direct link to Gate: Classification Complete") * Source category identified (transformation / transport / interception) * Integration approach chosen (SDK / DOM / HTTP / callback) * Know what the source captures and how it delivers data *** ## Phase 3: Create Input Examples (BEFORE Implementation)[​](#phase-3-create-input-examples-before-implementation "Direct link to Phase 3: Create Input Examples (BEFORE Implementation)") **Mandatory.** Examples are the test fixtures for Phase 8. Define expected `trigger` / `in` / `out` triples FIRST - start with the end result in mind. Without examples, you cannot test. Even for simple sources, step examples are the single source of truth for tests, simulations, and documentation. > **Authoritative pattern:** See [using-step-examples](/skills/walkeros-using-step-examples.md) for the Three Type Zones, `createTrigger` contract, and CI integration. This skill reuses that contract - do not diverge. ### 3.1 Scaffold Directory Structure[​](#31-scaffold-directory-structure "Direct link to 3.1 Scaffold Directory Structure") ``` mkdir -p packages//sources/[name]/src/examples mkdir -p packages//sources/[name]/src/{schemas,types} ``` ### 3.2 Required Files (3-4 files)[​](#32-required-files-3-4-files "Direct link to 3.2 Required Files (3-4 files)") All reference sources in the monorepo use this exact layout in `src/examples/`. Match it - no `inputs.ts`, `outputs.ts`, `requests.ts`, or standalone `mapping.ts`. | File | Required? | Purpose | | --------------------- | --------- | ------------------------------------------------------------------- | | `examples/step.ts` | yes | `Flow.StepExample` entries with `trigger` / `in` / `out` triples | | `examples/trigger.ts` | yes | `createTrigger` implementation following `Trigger.CreateFn` | | `examples/index.ts` | yes | Barrel exports: `env` (if present), `step`, `createTrigger` | | `examples/env.ts` | if needed | Mock env for platform deps (browser window/document, express, etc.) | `env.ts` is included whenever the source touches platform globals or injected deps - all web sources and every server source that wraps a platform SDK ship one. Sources whose tests drive the collector entirely through `trigger.ts` (e.g. `web/sources/session`) may omit it. When in doubt, **include it**. The old `inputs.ts` / `outputs.ts` / `requests.ts` / `mapping.ts` files are gone * their contents now live inline in each `Flow.StepExample` entry in `step.ts`. ### 3.3 Three Type Zones for Sources[​](#33-three-type-zones-for-sources "Direct link to 3.3 Three Type Zones for Sources") Sources are the **inverse** of destinations in the Three Type Zones model: | Zone | Source semantics | | --------- | ------------------------------------------------------------------------------------- | | `trigger` | How to simulate the invocation (HTTP method, DOM event type, cloud event) | | `in` | External trigger content - HTTP request, DOM HTML, SDK payload (NOT a walkerOS event) | | `out` | The walkerOS event(s) the source should emit (`WalkerOS.Event`) | Where a destination does `WalkerOS.Event → vendor output`, a source does `external content → WalkerOS.Event`. Read [using-step-examples](/skills/walkeros-using-step-examples.md) before authoring entries. ### 3.4 Typing Rules (strict)[​](#34-typing-rules-strict "Direct link to 3.4 Typing Rules (strict)") **No `any`.** Every example value must be explicitly typed. * **`trigger`** uses the local source trigger type or a platform-native type (e.g. `'load' | 'click'` for DOM, HTTP method strings for server sources). * **`in`** uses the **vendor / platform SDK types** imported from the official package whenever available (Express `Request`, Fetch `Request`, API Gateway `APIGatewayProxyEvent`, Lambda `Context`, GCP `CloudEvent`, etc.). Do not invent local request types when the platform publishes them. * **`out`** uses `WalkerOS.Event` (or `DeepPartialEvent` for fragments). * **Step entries** are typed `Flow.StepExample` from `@walkeros/core`. * **Mock env** is typed against the source's local `Env` type from `../types`. * `createTrigger` is typed as `Trigger.CreateFn` - the `Content` and `Result` generics come from the source's own types module. ### 3.5 Code Template - `examples/step.ts`[​](#35-code-template---examplesstepts "Direct link to 35-code-template---examplesstepts") ``` import type { Flow } from '@walkeros/core'; // One step example per captured trigger / input shape. // `trigger` tells createTrigger how to simulate the invocation. // `in` is the platform-specific content (HTTP request, DOM HTML, SDK payload) - // typed against the platform SDK's published types where available. // `out` is the walkerOS event the source is expected to emit. // Set `title` + `description` for public examples; mark test-only fixtures // with `public: false`. See // [walkeros-using-step-examples](../walkeros-using-step-examples/SKILL.md). export const pageView: Flow.StepExample = { trigger: { type: 'load', options: { url: 'https://example.com/docs', title: 'Documentation', }, }, in: '', // no external content - DOM-driven trigger out: { name: 'page view', data: { domain: 'example.com', title: 'Documentation', id: '/docs' }, entity: 'page', action: 'view', trigger: 'load', source: { type: 'browser', platform: 'web', url: 'https://example.com/docs', }, }, }; // Server example: HTTP POST carrying a walker event payload. export const orderComplete: Flow.StepExample = { trigger: { type: 'POST' }, in: { method: 'POST', path: '/collect', body: { name: 'order complete', data: { id: 'ORD-123', total: 149.97 } }, }, out: { name: 'order complete', data: { id: 'ORD-123', total: 149.97 }, entity: 'order', action: 'complete', }, }; ``` ### 3.6 `examples/index.ts` (barrel)[​](#36-examplesindexts-barrel "Direct link to 36-examplesindexts-barrel") ``` export * as env from './env'; // omit if the source has no env.ts export * as step from './step'; export { createTrigger, trigger } from './trigger'; ``` ### 3.7 `examples/trigger.ts` - `createTrigger`[​](#37-examplestriggerts---createtrigger "Direct link to 37-examplestriggerts---createtrigger") Every source exports a `createTrigger` following the unified `Trigger.CreateFn` interface. It simulates real-world invocations from the outside - no source instance access, full blackbox. ``` import type { Trigger } from '@walkeros/core'; import { startFlow } from '@walkeros/collector'; export const createTrigger: Trigger.CreateFn = async ( config, ) => { let flow: Trigger.FlowHandle | undefined; const trigger: Trigger.Fn = (type?: string) => async (content) => { if (!flow) { const result = await startFlow(config); flow = { collector: result.collector, elb: result.elb }; } // Package-specific: make real HTTP request, inject DOM, dispatch SDK call. // Return the Result type declared by this source. return /* ... */; }; return { get flow() { return flow; }, trigger, }; }; ``` Reference implementations: * **Browser:** `packages/web/sources/browser/src/examples/trigger.ts` - DOM injection + native event dispatch * **Session:** `packages/web/sources/session/src/examples/trigger.ts` - no env.ts, trigger drives collector directly * **Express:** `packages/server/sources/express/src/examples/trigger.ts` - real HTTP `fetch()` to running server * **CMP (Usercentrics):** `packages/web/sources/cmps/usercentrics/src/examples/trigger.ts` - dispatches CMP events, asserts on collector consent state * **Fetch (function handler):** `packages/server/sources/fetch/src/examples/trigger.ts` - accesses source instance via `collector.sources`, calls `source.push()` with platform-native `Request` * **AWS Lambda:** `packages/server/sources/aws/src/lambda/examples/trigger.ts` - constructs API Gateway event + Lambda context * **GCP CloudFunction:** `packages/server/sources/gcp/src/cloudfunction/examples/trigger.ts` - synthesizes mock req/res (matching GCP Functions Framework) ### 3.8 Test Fixture Contract (hard rule)[​](#38-test-fixture-contract-hard-rule "Direct link to 3.8 Test Fixture Contract (hard rule)") The examples authored here **are** the Phase 8 test fixtures. No parallel fixtures allowed. * `src/index.test.ts` **MUST** iterate examples via `it.each(Object.entries(examples.step))`. * Tests **must NOT** contain hardcoded trigger payloads, HTTP requests, DOM HTML, or expected events. * If a test needs a value that is not in `examples.step`, **add it to `step.ts` first**, then consume it from the test. * Tests invoke `examples.createTrigger(config)` and dispatch each example's `trigger.type` + `in` content, asserting the collector receives `out`. See the canonical source tests under `packages/web/sources/browser/src/index.test.ts` and `packages/server/sources/express/src/index.test.ts`. ### 3.9 Export via `dev.ts`[​](#39-export-via-devts "Direct link to 39-export-via-devts") ``` export * as schemas from './schemas'; export * as examples from './examples'; ``` ### Phase 3 Acceptance Checklist[​](#phase-3-acceptance-checklist "Direct link to Phase 3 Acceptance Checklist") * [ ] `src/examples/step.ts` - one `Flow.StepExample` per captured trigger / input shape, typed `trigger` / `in` / `out` * [ ] `src/examples/trigger.ts` - exports `createTrigger` typed as `Trigger.CreateFn` * [ ] `src/examples/index.ts` - barrel exports `step`, `createTrigger`, and `env` (when present) * [ ] `src/examples/env.ts` - included whenever the source touches platform globals or injected deps; typed against local `Env`; no real network * No standalone `inputs.ts`, `outputs.ts`, `requests.ts`, or `mapping.ts` files * All platform / SDK types imported from the official package - no `any`, no reinvented request / response shapes * [ ] `src/index.test.ts` iterates `examples.step` via `it.each(Object.entries(...))` * Tests contain zero hardcoded payloads, requests, or expected events - everything flows from `examples.step` * Edge cases included (minimal input, invalid input) * [ ] `npm run build` passes - examples compile against published types * Each example traces: `trigger` + `in` → source push → matches `out` *** ## Phase 4: Define Mapping[​](#phase-4-define-mapping "Direct link to Phase 4: Define Mapping") **Goal:** Document transformation from input format to walkerOS events. Mapping lives **inside** each `Flow.StepExample` entry in `step.ts` - no separate `mapping.ts` file. Sources typically carry the mapping either in the source's own `settings` (see `dataLayer` for an example) or inline via the `trigger` → `in` → `out` relationship: the `in` content is the raw platform payload; the `out` is the walkerOS event after the source's transformation. ### Verify Mapping Logic[​](#verify-mapping-logic "Direct link to Verify Mapping Logic") For each entry in `step.ts`, trace: ``` Input: examples.step.pageView.trigger + examples.step.pageView.in ↓ createTrigger dispatches the trigger ↓ Source receives platform content, runs its transformation ↓ Source calls env.push / collector.push Output: Should match examples.step.pageView.out (a WalkerOS.Event) ``` ### Gate: Mapping Verified[​](#gate-mapping-verified "Direct link to Gate: Mapping Verified") * Step examples cover the main input event types * Each example name transforms to correct walkerOS event name * Each example traces correctly from `(trigger, in)` to `out` *** ## Phase 5: Scaffold[​](#phase-5-scaffold "Direct link to Phase 5: Scaffold") **Template sources:** * Web: `packages/web/sources/dataLayer/` * Server: `packages/server/sources/fetch/` (simplest pattern) ``` cp -r packages/server/sources/fetch packages/server/sources/[name] cd packages/server/sources/[name] # Update package.json: name, description, repository.directory ``` **Directory structure:** ``` packages/server/sources/[name]/ ├── src/ │ ├── index.ts # Main export │ ├── index.test.ts # Tests against examples │ ├── dev.ts # Exports schemas and examples │ ├── examples/ │ ├── schemas/ │ └── types/ ├── package.json ├── tsconfig.json ├── tsup.config.ts ├── jest.config.mjs └── README.md ``` ### Transformer Chain Integration[​](#transformer-chain-integration "Direct link to Transformer Chain Integration") Sources can wire to transformer chains via `next` in the init config: ``` sources: { mySource: { code: sourceMySource, config: { settings: { /* ... */ } }, next: 'validate' // Events go through validator before collector } } ``` *** ## Phase 6: walkerOS.json Convention[​](#phase-6-walkerosjson-convention "Direct link to Phase 6: walkerOS.json Convention") Every walkerOS package ships a `walkerOS.json` file for CDN-based schema discovery. ### Add `walkerOS` field to package.json[​](#add-walkeros-field-to-packagejson "Direct link to add-walkeros-field-to-packagejson") ``` { "walkerOS": { "type": "source", "platform": "web" }, "keywords": ["walkerOS", "walkerOS-source", ...] } ``` ### Use `buildDev()` in tsup.config.ts[​](#use-builddev-in-tsupconfigts "Direct link to use-builddev-in-tsupconfigts") Replace `buildModules({ entry: ['src/dev.ts'] })` with `buildDev()`: ``` import { buildDev } from '@walkeros/config/tsup'; // In defineConfig array: buildDev(), ``` This auto-generates `dist/walkerOS.json` from your Zod schemas at build time. ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") If your source has capabilities, behaviors, or troubleshooting patterns not obvious from schemas alone, add hints. See `walkeros-writing-documentation` skill for full guidelines. Create `src/hints.ts`: ``` import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'capture-timing': { text: 'Describes when events are captured. See settings schema for options.', code: [{ lang: 'json', code: '{ "settings": { ... } }' }], }, }; ``` Export from `src/dev.ts`: ``` export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Guidelines: * Expand awareness - describe capabilities ("supports X, Y, Z"), don't prescribe one path * Reference schemas and examples, don't duplicate them * Verify every claim against actual implementation before publishing * Key naming: kebab-case, group with prefixes (auth-\*, capture-\*, troubleshoot-\*) * Most sources don't need hints - schemas and examples cover the common case ### Gate: Convention Met[​](#gate-convention-met "Direct link to Gate: Convention Met") * [ ] `walkerOS` field in package.json with type and platform * [ ] `buildDev()` in tsup.config.ts * Build generates `dist/walkerOS.json` * Keywords include `walkerOS` and `walkerOS-source` ### Runtime-only npm dependencies[​](#runtime-only-npm-dependencies "Direct link to Runtime-only npm dependencies") If your package wraps a third-party npm dep that **cannot be ESM-bundled** (uses `__dirname`, ships a `.node` binary, etc.), declare it under `walkerOS.bundle.external` in your `package.json`. See [walkeros-using-cli → Bundle externals](/skills/walkeros-using-cli.md#bundle-externals-per-package-walkerosbundleexternal) for the complete contract. *** ## Phase 7: Implement[​](#phase-7-implement "Direct link to Phase 7: Implement") **Now write code to produce the outputs defined in Phase 3.** ### Template Files[​](#template-files "Direct link to Template Files") | File | Purpose | Template | | ------------------ | ---------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `types/index.ts` | Type definitions | [types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-source/templates/server/types.ts) | | `schemas/index.ts` | Zod validation schemas | [schemas.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-source/templates/server/schemas.ts) | | `index.ts` | Main source | [index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-source/templates/server/index.ts) | ### Key Patterns[​](#key-patterns "Direct link to Key Patterns") 1. **Context destructuring**: Extract `config`, `env`, `logger`, `id` from context 2. **Schema validation**: Use Zod schemas to validate settings and provide defaults 3. **Forward to collector**: Call `env.push()` to send events to the collector 4. **Platform deps via env**: All platform dependencies (window, document, express, cors) must go through `env` with fallback to globals/imports: `env.express ?? express`. This enables testing without mocking globals. 5. **Error logging**: Use `logger?.error()` for errors only, not routine operations 6. **Return Source.Instance**: Return `{ type, config, push }` object 7. **Optional `destroy` method**: Implement if the source holds resources (HTTP servers, timers, connections) that need cleanup on shutdown ### Gate: Implementation Compiles[​](#gate-implementation-compiles "Direct link to Gate: Implementation Compiles") * [ ] `npm run build` passes * [ ] `npm run verify:touched -- ` passes (L1: typecheck + lint + test) *** ## Adding setup (optional)[​](#adding-setup-optional "Direct link to Adding setup (optional)") A source package can implement an optional `setup()` function to provision external resources idempotently: Pub/Sub subscriptions, webhook registrations on upstream platforms, queue declarations, SQS queues, polling cursors, inbound API keys. Setup runs only when an operator explicitly types `walkeros setup source.`. The runtime never auto-invokes it from `init()`, push, or `destroy()`. The framework provides the slot, the CLI command, and a `resolveSetup` helper. The package owns: what setup means, idempotency, error handling, return value. For background on how setup fits the source lifecycle, see [understanding-sources](/skills/walkeros-understanding-sources.md#setup-optional). ### Types[​](#types "Direct link to Types") ``` // types/index.ts import type { CoreSource } from '@walkeros/core'; export interface Settings { /* runtime push settings */ } export interface InitSettings { /* one-time init settings */ } export interface Mapping { /* event extraction mapping */ } export interface Env { /* injected platform deps (SDK clients, request handlers, etc.) */ } // The package's own setup options interface. // Becomes the U slot of Types; surfaces as `config.setup: boolean | Setup` for users. export interface Setup { // package-specific provisioning options // e.g. for Pub/Sub source: subscriptionName, ackDeadlineSeconds, filter subscriptionName?: string; ackDeadlineSeconds?: number; } export type Types = CoreSource.Types< Settings, Mapping, Env, InitSettings, Setup >; ``` ### Implementation[​](#implementation "Direct link to Implementation") ``` // setup.ts import type { CoreSource, SetupFn } from '@walkeros/core'; import { resolveSetup } from '@walkeros/core'; import type { Setup, Types } from './types'; const DEFAULT_SETUP: Setup = { ackDeadlineSeconds: 60, }; export const setup: SetupFn< CoreSource.Config, CoreSource.Env > = async ({ config, env, logger }) => { const options = resolveSetup(config.setup, DEFAULT_SETUP); if (!options) return; // config.setup is false or unset // Package-specific provisioning, idempotent. // Returning a structured object (e.g. { subscriptionCreated: true }) // makes that data available to operators via `walkeros setup ... | jq`. }; ``` Wire it in your default export: ``` // index.ts import { setup } from './setup'; export default { type: 'my-source', init: /* ... */, setup, }; ``` ### When to implement[​](#when-to-implement "Direct link to When to implement") Implement `setup()` when your source needs first-time provisioning of upstream resources before events can be received: Pub/Sub subscriptions bound to a topic, webhook registrations on upstream SaaS platforms (Stripe, GitHub, Shopify), SQS queue declarations, message broker bindings, polling cursors. Skip it when your source only listens on an HTTP port the runtime already owns or intercepts data already present (DOM events, dataLayer pushes). ### Contract[​](#contract "Direct link to Contract") * Triggered only by `walkeros setup .`. Never by runtime push, init, or destroy. * **Idempotency is your responsibility.** Re-running setup against a fully provisioned environment must be a safe no-op. Use try-create-catch-409 on REST APIs, `IF NOT EXISTS` on SQL, native idempotent operations where available. The framework does not retry, track state, or detect drift. * Return structured data from `setup()` when useful for operator scripting. The CLI emits non-undefined return values as JSON to stdout. * For packages where `setup: true` (boolean form) is meaningless because mandatory fields have no safe defaults (e.g., GitHub webhook `webhookUrl`, Pub/Sub source `topicName`), reject the boolean form with a clear runtime error listing required fields: ``` if (config.setup === true) { throw new Error( 'github-webhook source setup requires explicit options: ' + '{ webhookUrl, repo, events }. There is no safe default.', ); } ``` *** ## Phase 8: Test Against Examples[​](#phase-8-test-against-examples "Direct link to Phase 8: Test Against Examples") > Tests verify implementation against the examples from Phase 3. If examples are incomplete, tests will be incomplete. > > See [testing-strategy](/skills/walkeros-testing-strategy.md) for the shared env / dev-examples conventions this phase depends on. **Verify implementation produces expected outputs.** ### Test Template[​](#test-template "Direct link to Test Template") Use the test template: [index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-source/templates/server/index.test.ts). Canonical references: * `packages/web/sources/browser/src/index.test.ts` * `packages/server/sources/express/src/index.test.ts` ### Key Test Patterns[​](#key-test-patterns "Direct link to Key Test Patterns") 1. **`it.each(Object.entries(examples.step))` is mandatory** - one iteration per step example. Do not write per-feature tests with hand-rolled payloads. 2. **Drive via `createTrigger`** - construct the trigger with `startFlow` config, then dispatch each example's `trigger.type` + `in` content. 3. **Use `createSourceContext()` helper** for any direct context construction. 4. **Zero hardcoded payloads** - every trigger type, request body, DOM HTML, and expected event comes from `examples.step` or `examples.env`. If you need something new, add it to examples first. 5. **Test error paths** - verify graceful error handling and logging for invalid input (add an error example to `examples.step` if needed). ### Gate: Tests Pass[​](#gate-tests-pass "Direct link to Gate: Tests Pass") * [ ] `npm run verify:touched -- ` passes (L1) * Tests iterate via `it.each(Object.entries(examples.step))` * Tests contain no hardcoded payloads, requests, or expected events * Every assertion reads from `examples.step[...].out` * Invalid input handled gracefully (no crashes) *** ## Phase 9: Document[​](#phase-9-document "Direct link to Phase 9: Document") Follow the [writing-documentation](/skills/walkeros-writing-documentation.md) skill for: * README structure and templates * Example validation against `apps/quickstart/` * Quality checklist before publishing Key requirements for source documentation: * Input format table documenting expected fields * Event name mapping table (source format → walkerOS format) * Configuration options table * Working code example with imports * Installation instructions *** ## Validation Checklist[​](#validation-checklist "Direct link to Validation Checklist") Beyond [understanding-development](/skills/walkeros-understanding-development.md) requirements (build, test, lint, no `any`): * [ ] `dev.ts` exports `schemas` and `examples` * Examples include edge cases (minimal, invalid input) * Invalid input returns gracefully (no crashes, clear error) * Tests use examples for assertions (not hardcoded values) * [ ] `walkerOS.json` generated at build time * [ ] `walkerOS` field in package.json *** ## Reference Files[​](#reference-files "Direct link to Reference Files") | What | Where | | --------------- | ----------------------------------- | | Web template | `packages/web/sources/dataLayer/` | | Server template | `packages/server/sources/fetch/` | | Source types | `packages/core/src/types/source.ts` | | Event creation | `packages/core/src/lib/event.ts` | ## Related Skills[​](#related-skills "Direct link to Related Skills") * [understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface and push pattern * [understanding-events](/skills/walkeros-understanding-events.md) - Event structure * [using-step-examples](/skills/walkeros-using-step-examples.md) - Authoritative `Flow.StepExample` + `createTrigger` pattern, Three Type Zones * [testing-strategy](/skills/walkeros-testing-strategy.md) - Testing with env mocking and dev-examples-as-fixtures conventions * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards --- # Create a New Transformer ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, read these skills: * [understanding-flow](/skills/walkeros-understanding-flow.md) - How transformers fit in architecture * [understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer interface * [understanding-events](/skills/walkeros-understanding-events.md) - Event structure * [testing-strategy](/skills/walkeros-testing-strategy.md) - How to test * [writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards (for Phase 7) ## Transformer Categories[​](#transformer-categories "Direct link to Transformer Categories") | Category | Purpose | Example | | ------------ | ----------------------------- | ---------------------------- | | **Validate** | Check event structure/content | JSON Schema, required fields | | **Enrich** | Add data to events | Server timestamps, geo data | | **Redact** | Remove/mask sensitive data | Strip PII, anonymize IPs | ## Process Overview[​](#process-overview "Direct link to Process Overview") ``` 1. Research → Understand use case (validate/enrich/redact) 2. Examples → Create event before/after examples FIRST 3. Scaffold → Copy template, configure package.json 4. Convention → Add walkerOS.json metadata and buildDev 5. Implement → Build transformer with TDD 6. Test → Verify against example transformations 7. Document → Write README ``` ## Supporting Files[​](#supporting-files "Direct link to Supporting Files") This skill includes reference files you can copy: * **[examples/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/examples)** - Example events and configs to adapt for your transformer * [events.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/examples/events.ts) - Before/after event examples * [config.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/examples/config.ts) - Configuration examples * **[templates/validation/](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation)** - Complete redact transformer template * [index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/index.ts) - Implementation with context pattern * [types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/types.ts) - Type definitions * [schemas.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/schemas.ts) - Zod validation schemas * [index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/index.test.ts) - Test suite with helpers *** ## Phase 1: Research[​](#phase-1-research "Direct link to Phase 1: Research") **Goal:** Understand what the transformer needs to do. ### 1.1 Define Use Case[​](#11-define-use-case "Direct link to 1.1 Define Use Case") * [ ] **Category**: Validate, Enrich, or Redact? * [ ] **Input**: What events will this process? * [ ] **Output**: What should change? What should be blocked? * [ ] **Configuration**: What settings does user need? ### 1.2 Check Existing Patterns[​](#12-check-existing-patterns "Direct link to 1.2 Check Existing Patterns") ``` # Reference implementation ls packages/transformers/fingerprint/ # Transformer types cat packages/core/src/types/transformer.ts ``` ### Gate: Research Complete[​](#gate-research-complete "Direct link to Gate: Research Complete") * Category identified (validate/enrich/redact) * Input/output transformation defined * Configuration options listed *** ## Phase 2: Create Examples (BEFORE Implementation)[​](#phase-2-create-examples-before-implementation "Direct link to Phase 2: Create Examples (BEFORE Implementation)") **Goal:** Define event transformations in `dev` entry FIRST. ### 2.1 Scaffold Directory Structure[​](#21-scaffold-directory-structure "Direct link to 2.1 Scaffold Directory Structure") ``` mkdir -p packages/transformers/[name]/src/{examples,schemas,types} ``` ### 2.2 Create Event Examples[​](#22-create-event-examples "Direct link to 2.2 Create Event Examples") Adapt [examples/events.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/examples/events.ts) for your transformer's use case. Each example file should include: * Events that should pass through (modified) * Expected output after processing * Events that should be blocked (for validators) ### 2.3 Create Config Examples[​](#23-create-config-examples "Direct link to 2.3 Create Config Examples") Adapt [examples/config.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/examples/config.ts) for your transformer's settings. ### 2.4 Step Examples[​](#24-step-examples "Direct link to 2.4 Step Examples") Add step examples with `{ in, out }` pairs for end-to-end step testing: ``` // examples/step.ts export const step = { 'order-passes': { in: { name: 'order complete', data: { id: 'ORD-123' } }, out: { name: 'order complete', data: { id: 'ORD-123' } }, }, 'debug-filtered': { in: { name: 'debug test', data: { message: 'noise' } }, out: false, // Transformer rejects this event }, }; ``` For transformers, both `in` and `out` are walkerOS events, except `out: false` which indicates the transformer filters (rejects) the event. Set `title` + `description` for public examples; mark test-only fixtures with `public: false`. See [using-step-examples](/skills/walkeros-using-step-examples.md) for the Three Type Zones. ### 2.5 Export via dev.ts[​](#25-export-via-devts "Direct link to 2.5 Export via dev.ts") ``` export * as schemas from './schemas'; export * as examples from './examples'; ``` ### Gate: Examples Valid[​](#gate-examples-valid "Direct link to Gate: Examples Valid") * All example files compile (`npm run build`) * Can trace: input event → expected output for each example *** ## Phase 3: Scaffold[​](#phase-3-scaffold "Direct link to Phase 3: Scaffold") **Template transformer:** `packages/transformers/fingerprint/` ``` cp -r packages/transformers/fingerprint packages/transformers/[name] cd packages/transformers/[name] # Update package.json: name, description, repository.directory ``` **Directory structure:** ``` packages/transformers/[name]/ ├── src/ │ ├── index.ts # Main export │ ├── transformer.ts # Transformer implementation │ ├── index.test.ts # Tests against examples │ ├── dev.ts # Exports schemas and examples │ ├── examples/ │ │ ├── index.ts # Re-exports │ │ ├── events.ts # Before/after event examples │ │ └── config.ts # Configuration examples │ ├── schemas/ │ │ └── index.ts # Zod schemas for settings │ └── types/ │ └── index.ts # Settings, Types interfaces ├── package.json ├── tsconfig.json ├── tsup.config.ts ├── jest.config.mjs └── README.md ``` *** ## Phase 4: walkerOS.json Convention[​](#phase-4-walkerosjson-convention "Direct link to Phase 4: walkerOS.json Convention") Every walkerOS package ships a `walkerOS.json` file for CDN-based schema discovery. ### Add `walkerOS` field to package.json[​](#add-walkeros-field-to-packagejson "Direct link to add-walkeros-field-to-packagejson") ``` { "walkerOS": { "type": "transformer" }, "keywords": ["walkerOS", "walkerOS-transformer", ...] } ``` ### Use `buildDev()` in tsup.config.ts[​](#use-builddev-in-tsupconfigts "Direct link to use-builddev-in-tsupconfigts") Replace `buildModules({ entry: ['src/dev.ts'] })` with `buildDev()`: ``` import { buildDev } from '@walkeros/config/tsup'; // In defineConfig array: buildDev(), ``` This auto-generates `dist/walkerOS.json` from your Zod schemas at build time. ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") If your transformer has capabilities, behaviors, or troubleshooting patterns not obvious from schemas alone, add hints. See `walkeros-writing-documentation` skill for full guidelines. Create `src/hints.ts`: ``` import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'validation-behavior': { text: 'Describes how validation works. See settings schema for options.', code: [{ lang: 'json', code: '{ "settings": { ... } }' }], }, }; ``` Export from `src/dev.ts`: ``` export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Guidelines: * Expand awareness — describe capabilities ("supports X, Y, Z"), don't prescribe one path * Reference schemas and examples, don't duplicate them * Verify every claim against actual implementation before publishing * Key naming: kebab-case, group with prefixes (validation-\*, enrichment-\*, troubleshoot-\*) * Most transformers don't need hints — schemas and examples cover the common case ### Gate: Convention Met[​](#gate-convention-met "Direct link to Gate: Convention Met") * [ ] `walkerOS` field in package.json with `type: "transformer"` * [ ] `buildDev()` in tsup.config.ts * Build generates `dist/walkerOS.json` * Keywords include `walkerOS` and `walkerOS-transformer` ### Runtime-only npm dependencies[​](#runtime-only-npm-dependencies "Direct link to Runtime-only npm dependencies") If your package wraps a third-party npm dep that **cannot be ESM-bundled** (uses `__dirname`, ships a `.node` binary, etc.), declare it under `walkerOS.bundle.external` in your `package.json`. See [walkeros-using-cli → Bundle externals](/skills/walkeros-using-cli.md#bundle-externals-per-package-walkerosbundleexternal) for the complete contract. *** ## Phase 5: Implement[​](#phase-5-implement "Direct link to Phase 5: Implement") **Now write code to transform examples as expected.** ### 5.1 Define Types[​](#51-define-types "Direct link to 5.1 Define Types") See [templates/validation/types.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/types.ts) for the pattern. Define `Settings` and `Types` interfaces. ### 5.2 Implement Transformer (Context Pattern)[​](#52-implement-transformer-context-pattern "Direct link to 5.2 Implement Transformer (Context Pattern)") Transformers use the **context pattern** - they receive a single `context` object containing `config`, `env`, `logger`, `id`, and `collector`. See [templates/validation/index.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/index.ts) for a complete implementation example. **Key patterns:** 1. **Context destructuring**: Extract `config`, `logger`, `id` from init context 2. **Schema validation**: Use Zod schemas to validate settings and provide defaults (see [templates/validation/schemas.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/schemas.ts)) 3. **Push receives pushContext**: The `push` function gets event + push context 4. **Return values**: `{ event }` (continue), `void` (passthrough), `false` (cancel) ### 5.3 Export[​](#53-export "Direct link to 5.3 Export") `src/index.ts`: ``` export { transformerRedact } from './transformer'; export type { Settings, Types } from './types'; ``` ### Gate: Implementation Compiles[​](#gate-implementation-compiles "Direct link to Gate: Implementation Compiles") * [ ] `npm run build` passes * [ ] `npm run verify:touched -- ` passes (L1: typecheck + lint + test) *** ## Phase 6: Test Against Examples[​](#phase-6-test-against-examples "Direct link to Phase 6: Test Against Examples") **Verify implementation produces expected outputs.** See [templates/validation/index.test.ts](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-create-transformer/templates/validation/index.test.ts) for a complete test suite showing: 1. **`createTransformerContext()` helper** - Standardizes init context creation 2. **`createPushContext()` helper** - Standardizes push context creation 3. **Examples for test data** - Don't hardcode test values 4. **Return value testing** - Verify `event`, `void`, or `false` returns ### Gate: Tests Pass[​](#gate-tests-pass "Direct link to Gate: Tests Pass") * [ ] `npm run verify:touched -- ` passes (L1) * Tests verify against example outputs *** ## Phase 7: Document[​](#phase-7-document "Direct link to Phase 7: Document") Follow the [writing-documentation](/skills/walkeros-writing-documentation.md) skill for: * README structure and templates * Quality checklist before publishing Key requirements for transformer documentation: * Use case description (validate/enrich/redact) * Configuration options table * Working code example with imports * Installation instructions * Link to website docs *** ## Transformer-Specific Validation[​](#transformer-specific-validation "Direct link to Transformer-Specific Validation") Beyond [understanding-development](/skills/walkeros-understanding-development.md) requirements (build, test, lint, no `any`): * [ ] `dev.ts` exports `schemas` and `examples` * Examples include before/after event pairs * Return values handle all cases (event, void, false) * Tests use examples for assertions (not hardcoded values) * [ ] `walkerOS.json` generated at build time * [ ] `walkerOS` field in package.json *** ## Reference Files[​](#reference-files "Direct link to Reference Files") | What | Where | | -------------- | ---------------------------------------- | | Template | `packages/transformers/fingerprint/` | | Types | `packages/core/src/types/transformer.ts` | | Chaining logic | `packages/collector/src/transformer.ts` | ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer interface and chaining * [walkeros-testing-strategy](/skills/walkeros-testing-strategy.md) - Testing patterns and env mocking * [walkeros-writing-documentation](/skills/walkeros-writing-documentation.md) - Documentation standards --- # Debugging walkerOS Events ## Quick Diagnosis[​](#quick-diagnosis "Direct link to Quick Diagnosis") | Symptom | Likely Cause | Check | | ---------------------------------- | ---------------------------- | ---------------------------------------- | | No events at all | Source not initialized | Console for errors, verify `startFlow()` | | Events fire but destination silent | Mapping mismatch | Event name matches mapping? | | Partial data missing | Path doesn't exist | Log event structure, check nested paths | | Consent blocking | Required consent not granted | Check `consent` config, grant consent | | Destination error | Vendor API issue | Check network tab, vendor console | ## Debugging Strategies[​](#debugging-strategies "Direct link to Debugging Strategies") ### 1. Console Logging[​](#1-console-logging "Direct link to 1. Console Logging") **Log all events at collector level:** ``` import { startFlow } from '@walkeros/collector'; const { elb } = await startFlow({ destinations: { debug: { push: async (event, context) => { console.log('[walkerOS Event]', { name: event.name, data: event.data, context: event.context, consent: event.consent, timestamp: event.timestamp, }); }, config: {}, }, // ... other destinations }, }); ``` ### 2. Network Tab Inspection[​](#2-network-tab-inspection "Direct link to 2. Network Tab Inspection") For destinations that make HTTP calls: 1. Open DevTools → Network tab 2. Filter by destination domain (e.g., `google-analytics.com`, `facebook.com`) 3. Trigger event 4. Inspect request payload **What to look for:** * Request being made at all? * Correct endpoint URL? * Payload structure matches vendor spec? ### 3. Vendor Debug Tools[​](#3-vendor-debug-tools "Direct link to 3. Vendor Debug Tools") | Vendor | Debug Tool | | --------- | --------------------------------------------------------------------------------------------- | | GA4 | [GA4 DebugView](https://support.google.com/analytics/answer/7201382) | | Meta | [Facebook Pixel Helper](https://developers.facebook.com/docs/meta-pixel/support/pixel-helper) | | Plausible | [Plausible Dashboard real-time](https://plausible.io/docs) | ### 4. Dry Run Mode[​](#4-dry-run-mode "Direct link to 4. Dry Run Mode") Test mapping without sending to vendor: ``` const destination = { ...actualDestination, config: { ...actualDestination.config, dryRun: true, // Events processed but not sent }, }; ``` ## Common Issues[​](#common-issues "Direct link to Common Issues") ### Event Name Mismatch[​](#event-name-mismatch "Direct link to Event Name Mismatch") **Problem:** Event fires but destination doesn't receive it. ``` // Event pushed elb('product view', { id: 'P123' }); // Mapping expects different name mapping: { Product: { // Wrong: capital P View: { // Wrong: capital V name: 'view_item'; } } } ``` **Fix:** Event names are case-sensitive. Use exact match: ``` mapping: { product: { view: { name: 'view_item'; } } } ``` ### Missing Nested Data[​](#missing-nested-data "Direct link to Missing Nested Data") **Problem:** `items` array is empty in destination. ``` // Event structure { name: 'order complete', data: { total: 100 }, nested: [ { entity: 'product', data: { id: 'P1' } } ] } // Mapping tries wrong path data: { map: { items: { loop: [ 'data.items', // Wrong: nested is at root, not in data { map: { id: 'data.id' } } ] } } } ``` **Fix:** Use correct path to nested array: ``` items: { loop: [ 'nested', // Correct: root-level nested { map: { item_id: 'data.id' } }, ]; } ``` ### Consent Blocking Events[​](#consent-blocking-events "Direct link to Consent Blocking Events") **Problem:** Events not reaching destination. **Check 1:** Does destination require consent? ``` // Destination config config: { consent: { marketing: true; } // Requires marketing consent } ``` **Check 2:** Is consent granted? ``` // Check current consent state console.log(event.consent); // Grant consent elb('walker consent', { marketing: true }); ``` ### Vendor SDK Not Loaded[​](#vendor-sdk-not-loaded "Direct link to Vendor SDK Not Loaded") **Problem:** `TypeError: env.window.gtag is not a function` **Cause:** Vendor script not loaded before push. **Fix:** Ensure init() loads script: ``` init: async (config, env) => { // Wait for script to load await loadScript('https://vendor.com/sdk.js'); // Verify SDK available if (!env.window.vendorSdk) { throw new Error('Vendor SDK failed to load'); } }, ``` ### Function Mapping Errors[​](#function-mapping-errors "Direct link to Function Mapping Errors") **Problem:** `Cannot read property 'price' of undefined` ``` // Mapping with unsafe access data: { map: { value: { fn: (e) => e.data.price * 100; } // Fails if data.price undefined } } ``` **Fix:** Add null checks: ``` value: { fn: (e) => (e.data?.price ?? 0) * 100; } ``` ## Debugging Checklist[​](#debugging-checklist "Direct link to Debugging Checklist") When events aren't working: 1. [ ] **Console errors?** Check browser console for exceptions 2. [ ] **Event pushed?** Add debug destination to log all events 3. [ ] **Mapping matched?** Verify entity/action names exactly match 4. [ ] **Data paths correct?** Log full event structure, verify paths exist 5. [ ] **Consent granted?** Check consent requirements and state 6. [ ] **SDK loaded?** Verify vendor script loaded before push 7. [ ] **Network request?** Check DevTools network tab 8. [ ] **Vendor receiving?** Use vendor debug tools ## Testing in Isolation[​](#testing-in-isolation "Direct link to Testing in Isolation") Test destination push directly: ``` import { push } from '@walkeros/web-destination-gtag'; import { mockEnv } from '@walkeros/core'; // Create test event const event = { name: 'product view', data: { id: 'P123', price: 99 }, // ... full event }; // Mock env to capture calls const calls = []; const testEnv = mockEnv(baseEnv, (path, args) => { calls.push({ path, args }); }); // Test push directly await push(event, { config: testConfig, env: testEnv }); // Inspect what was called console.log(calls); ``` ## MCP Tool Debugging[​](#mcp-tool-debugging "Direct link to MCP Tool Debugging") When using walkerOS MCP tools, check `_hints.warnings` in tool responses for diagnostic information: * **`flow_simulate`** warns when 0 destinations exist or none received the event. `step` is required (e.g. `"destination.gtag"`). Source steps take a `{ content, trigger? }` event where `content` is `{ name, data }`; sources, including `@walkeros/source-demo`, can be simulated this way. * **`flow_bundle`** warns when the build produces no output * **`flow_examples`** warns when no examples are found in the config * **`package_search`** returns the complete catalog and warns (via the `warnings` array) when it falls back to a partial source or omits packages, instead of silently returning a partial list * All error responses include a `hint` field with recovery suggestions When a request fails or behaves unexpectedly, call **`diagnostics`** (read-only, no parameters, works logged out). It reports the MCP and CLI versions, the resolved app URL and whether it came from `WALKEROS_APP_URL` or the default, app `/api/health` reachability, the bundled OpenAPI contract version, and which source served the last package lookup, so you can tell which backend and versions you are on. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Event flow architecture * [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination interface * [walkeros-mapping-configuration](/skills/walkeros-mapping-configuration.md) - Mapping recipes --- # Mapping Configuration Recipes ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Read [understanding-mapping](/skills/walkeros-understanding-mapping.md) first for core concepts. ## Quick Reference[​](#quick-reference "Direct link to Quick Reference") | I want to... | Use this pattern | | ------------------------------ | ----------------------------------------------------- | | Rename event | `{ name: 'new_name' }` | | Extract nested value | `'data.nested.value'` | | Set static value | `{ value: 'USD' }` | | Transform value | `{ fn: (e) => transform(e) }` | | Build object | `{ map: { key: 'source' } }` | | Process array | `{ loop: ['source', { map: {...} }] }` | | Gate by consent | `{ key: 'data.email', consent: { marketing: true } }` | | First defined value (fallback) | `[{ key: 'data.sku' }, { key: 'data.id' }]` | | Add a field to a default rule | `{ extend: { data: { map: { x: 'path' } } } }` | | Drop a field from output | `{ remove: ['field.path'] }` | ## Common Recipes[​](#common-recipes "Direct link to Common Recipes") ### GA4 / gtag[​](#ga4--gtag "Direct link to GA4 / gtag") **Product view → view\_item:** ``` product: { view: { name: 'view_item', data: { map: { currency: { value: 'USD' }, value: 'data.price', items: { loop: [ 'nested', { map: { item_id: 'data.id', item_name: 'data.name', item_category: 'data.category', price: 'data.price', quantity: { value: 1 }, }, }, ], }, }, }, }, } ``` **Order complete → purchase:** ``` order: { complete: { name: 'purchase', data: { map: { transaction_id: 'data.orderId', value: 'data.total', currency: 'data.currency', items: { loop: [ 'nested', { map: { item_id: 'data.id', item_name: 'data.name', price: 'data.price', quantity: 'data.quantity', }, }, ], }, }, }, }, } ``` ### Meta Pixel[​](#meta-pixel "Direct link to Meta Pixel") **Product view → ViewContent:** ``` product: { view: { name: 'ViewContent', data: { map: { content_ids: { fn: (e) => [e.data.id] }, content_type: { value: 'product' }, content_name: 'data.name', value: 'data.price', currency: { value: 'USD' }, }, }, }, } ``` **Order complete → Purchase:** ``` order: { complete: { name: 'Purchase', data: { map: { content_ids: { fn: (e) => e.nested?.map((n) => n.data.id) ?? [] }, content_type: { value: 'product' }, value: 'data.total', currency: 'data.currency', num_items: { fn: (e) => e.nested?.length ?? 0 }, }, }, }, } ``` ### Custom API Destination[​](#custom-api-destination "Direct link to Custom API Destination") **Transform to REST API format:** ``` '*': { '*': { name: { fn: (e) => `${e.entity}_${e.action}` }, // page_view data: { map: { eventName: 'name', eventData: 'data', userId: 'user.id', sessionId: 'user.session', timestamp: 'timestamp', metadata: { map: { consent: 'consent', globals: 'globals', }, }, }, }, }, } ``` ### Conditional Mapping[​](#conditional-mapping "Direct link to Conditional Mapping") **Different mapping based on event data:** ``` order: { complete: [ // High-value orders get extra tracking { condition: (e) => (e.data?.total ?? 0) > 500, name: 'high_value_purchase', data: { map: { value: 'data.total', priority: { value: 'high' }, notify: { value: true }, }, }, }, // Standard orders { name: 'purchase', data: { map: { value: 'data.total' } }, }, ], } ``` ### Patching Package-Shipped Rules (`extend` / `remove`)[​](#patching-package-shipped-rules-extend--remove "Direct link to patching-package-shipped-rules-extend--remove") Some packages (such as `@walkeros/transformer-ga4`) ship default mapping rules. Normally a user rule at the same key replaces the default in full. Use `extend` or `remove` to patch instead. Two layers, two keywords: * **`extend`** runs at init (config layer): deep-merges a partial rule onto the shipped default. A `null` value clears an inherited field. * **`remove`** runs after evaluation (output layer): strips dotted paths from the final data payload. Applied last, always wins. A rule with neither keyword keeps the existing replace behavior. ``` { "purchase": { "extend": { "data": { "map": { "affiliation": "params.ep.affiliation" } } }, "remove": ["currency"] } } ``` This keeps all fields the package ships for `purchase`, adds `affiliation`, and strips `currency` from the output. For the full reference and the two-layer model, see [Mapping.Rule docs](/docs/mapping/rule.md#patching-package-shipped-rules). ### Value\[] Fallback Chains[​](#value-fallback-chains "Direct link to Value\[] Fallback Chains") At any value position, an array of values is a fallback chain: the first entry that resolves to a defined value wins. ``` { "item_id": [ { "key": "data.sku" }, { "key": "data.id" }, { "value": "unknown" } ] } ``` Use this to try multiple source fields before falling back to a constant. ### Consent-Gated Fields[​](#consent-gated-fields "Direct link to Consent-Gated Fields") **Only include PII if consent granted:** ``` user: { login: { name: 'login', data: { map: { method: 'data.method', // Only include email if marketing consent email: { key: 'user.email', consent: { marketing: true }, }, // Only include user ID if functional consent userId: { key: 'user.id', consent: { functional: true }, }, }, }, }, } ``` ### Wildcard Patterns[​](#wildcard-patterns "Direct link to Wildcard Patterns") **Catch-all for unmatched events:** ``` // Any product action product: { '*': { name: { fn: (e) => `product_${e.action}` }, data: 'data', }, } // Any click on any entity '*': { click: { name: 'element_click', data: { map: { element_type: 'entity', element_id: 'data.id', }, }, }, } ``` ## Source-Side Mapping[​](#source-side-mapping "Direct link to Source-Side Mapping") **Transform HTTP input to walkerOS event:** ``` // In source config { mapping: { // Map incoming field names to walkerOS structure name: { fn: (input) => `${input.entity} ${input.action}` }, data: 'payload', user: { map: { id: 'userId', session: 'sessionId', }, }, }, } ``` ## Debugging Tips[​](#debugging-tips "Direct link to Debugging Tips") 1. **Event not mapping?** Check entity/action match exactly (case-sensitive) 2. **Data missing?** Verify source path exists: `'data.nested.field'` 3. **Function errors?** Add null checks: `e.data?.price ?? 0` 4. **Array empty?** Confirm `nested` array exists and has items ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-mapping](/skills/walkeros-understanding-mapping.md) - Core mapping concepts **Reference:** * [packages/core/src/mapping.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/mapping.ts) - Implementation * [apps/quickstart/src/](https://github.com/elbwalker/walkerOS/blob/main/apps/quickstart/src) - Validated examples --- # Calling walkerOS MCP Tools from Code ## When to Use This Skill[​](#when-to-use-this-skill "Direct link to When to Use This Skill") You are an AI agent running inside a **code-execution host**, not issuing tool calls one at a time through the model. Examples: * Claude Code in code-execution / "Code Mode" * a Cloudflare Worker or other serverless sandbox * an app's server-side sandbox that binds the walkerOS MCP transport The pattern: bind the MCP transport once, call only the tools you need from code, **filter the verbose `structuredContent` payload in code**, and return only the distilled answer to the model. This suppresses tokens that the model never needs to see (full catalogs, per-destination simulation dumps) and avoids a chain of one-by-one tool calls. This is the "Code Execution with MCP" / "Code Mode" pattern: treat MCP tools as functions you call in a sandbox, not as round-trips through the model context. ## The Seven Pure Tools[​](#the-seven-pure-tools "Direct link to The Seven Pure Tools") These walkerOS MCP tools are pure and composable: same input, same output, no auth, no cloud side effects. Safe to call freely from code. | Tool (fully-qualified) | Purpose | Verbose output worth filtering | | ------------------------- | --------------------------------------------------- | ------------------------------- | | `walkeros:flow_validate` | Validate a flow or step config; return issues | small | | `walkeros:flow_simulate` | Run a flow against an event, mock destination calls | **yes** (per-destination calls) | | `walkeros:flow_bundle` | Bundle a flow config to a JS artifact | medium (bundle/meta) | | `walkeros:flow_examples` | Fetch canonical flow/step examples | **yes** (many examples) | | `walkeros:package_search` | Find packages by type/platform/keyword | **yes** (catalog) | | `walkeros:package_get` | Read a package's schemas, hints, examples | **yes** (full schemas) | | `walkeros:diagnostics` | Report MCP/CLI versions, app URL, reachability | small | To test an event against a flow, use `walkeros:flow_simulate`: it runs the event through the flow with destination calls **mocked**, so nothing leaves the sandbox. Do not use `walkeros:flow_push` for this (see below). Always use the **fully-qualified `walkeros:`** name (Agent Skills best-practice) so the call is unambiguous when multiple MCP servers are bound. ### Out of scope for this pattern[​](#out-of-scope-for-this-pattern "Direct link to Out of scope for this pattern") The eight cloud/auth/side-effect tools (`walkeros:auth`, `walkeros:project_manage`, `walkeros:flow_manage`, `walkeros:deploy_manage`, `walkeros:secret_manage`, `walkeros:feedback`, `walkeros:flow_load`, `walkeros:flow_push`) carry authentication, cloud state, or side effects. `walkeros:flow_push` in particular sends a **real event to real destinations** (real API calls to live endpoints), so calling it "freely" would produce real, duplicated sends; use `walkeros:flow_simulate` to test without sending. Do **not** drive these tools from this code-execution filtering pattern; they belong in an interactive, authorized session. ## The Recommended Pattern[​](#the-recommended-pattern "Direct link to The Recommended Pattern") 1. **Bind the transport once.** Connect to the walkerOS MCP server at the top of your script, reuse the client for every call. 2. **Call only the tools you need.** Don't enumerate; pick the minimal set. 3. **Read `structuredContent`.** Every tool returns an MCP envelope; the typed payload lives in `structuredContent` (the `content` text block is for humans). 4. **Filter in code.** Reduce verbose payloads to the few fields that answer the question. 5. **Return the distilled answer.** Hand the model one line, not the full dump. ``` // Pseudo-code in a code-execution host. `mcp` is the bound walkerOS client. // (Typed `@walkeros/mcp-actions` wrappers are planned/forthcoming; until then // call the raw tools and read `structuredContent` yourself.) const flow = await readFile('flow.json', 'utf8'); // 1. Validate first; bail with one line if the config is broken. const v = await mcp.call('walkeros:flow_validate', { type: 'flow', input: flow, }); if (!v.structuredContent.valid) { return `invalid flow: ${v.structuredContent.errors.length} error(s)`; } // 2. Simulate one event through the flow; output is verbose (every destination). const sim = await mcp.call('walkeros:flow_simulate', { configPath: 'flow.json', event: 'product add', }); // 3. Filter in code: keep only destinations that failed. The model never sees // the full per-destination call dump, only the distilled verdict. const failed = Object.entries(sim.structuredContent.destinations) .filter(([, d]) => !d.received) .map(([name]) => name); // 4. Return one line. return failed.length ? `simulate failed for: ${failed.join(', ')}` : 'simulate ok for all destinations'; ``` The same shape applies to `walkeros:package_search` / `walkeros:package_get` (filter a catalog or a schema down to the one field the user asked about) and `walkeros:flow_examples` (pick the one matching example). ## Responsibility Boundary[​](#responsibility-boundary "Direct link to Responsibility Boundary") PII handling, secret redaction, and sandbox isolation are the **execution host's** responsibility, not walkerOS's: walkerOS MCP tools are pure functions over the config and event you pass them. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-using-cli](/skills/walkeros-using-cli.md) - The validate / simulate / bundle verbs these tools wrap * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Flow architecture you are validating and simulating * [walkeros-mapping-configuration](/skills/walkeros-mapping-configuration.md) - Mapping recipes for the configs you inspect with `package_get` --- # walkerOS Testing Strategy ## Overview[​](#overview "Direct link to Overview") walkerOS uses a layered testing approach with built-in patterns for mocking and documentation sync. This skill ensures tests are reliable, efficient, and maintainable. **Core principle:** Test real behavior using the `env` pattern, link to `dev` examples, verify before claiming complete. ## The Rules[​](#the-rules "Direct link to The Rules") ### Rule 1: Use `env` for Mocking, Not Jest[​](#rule-1-use-env-for-mocking-not-jest "Direct link to rule-1-use-env-for-mocking-not-jest") walkerOS has a built-in dependency injection pattern via `env` in context. This is lighter than Jest mocks, enables documentation generation, and keeps tests in sync with examples. **Wrong:** ``` jest.mock('../ga4', () => ({ initGA4: jest.fn() })); expect(initGA4).toHaveBeenCalledWith(...); ``` **Right:** ``` import { examples } from '../dev'; import { mockEnv } from '@walkeros/core'; const calls: Array<{ path: string[]; args: unknown[] }> = []; const testEnv = mockEnv(examples.env.push, (path, args) => { calls.push({ path, args }); }); await destination.push(event, { ...context, env: testEnv }); expect(calls).toContainEqual({ path: ['window', 'gtag'], args: ['event', 'page_view', { page_title: 'Home' }], }); ``` ### Rule 2: Link Tests to `dev` Examples[​](#rule-2-link-tests-to-dev-examples "Direct link to rule-2-link-tests-to-dev-examples") The `dev.ts` export provides `examples.env`, `examples.events`, `examples.mapping`, and `examples.step`. Using these in tests ensures documentation stays in sync. ``` import { examples } from '../dev'; // Use examples.env for mock environment const testEnv = mockEnv(examples.env.push, interceptor); // Assert against examples.events (documented expected output) expect(calls[0].args).toEqual(examples.events.ga4PageView()); // Test with examples.mapping configurations const config = { mapping: examples.mapping.ecommerce }; ``` #### Step Examples with `it.each`[​](#step-examples-with-iteach "Direct link to step-examples-with-iteach") Step examples (`examples.step`) provide `{ in, out }` pairs for each step. Use `it.each` to iterate over them: ``` import { examples } from '../dev'; describe('step examples', () => { it.each(Object.entries(examples.step))( '%s', async (name, { in: input, out: expected }) => { const result = await step.push(input, context); if (expected === false) { expect(result).toBe(false); } else { expect(result).toEqual(expected); } }, ); }); ``` See [using-step-examples](/skills/walkeros-using-step-examples.md) for the full lifecycle including the Three Type Zones and naming conventions. ### Rule 3: Test Real Behavior, Not Mock Behavior[​](#rule-3-test-real-behavior-not-mock-behavior "Direct link to Rule 3: Test Real Behavior, Not Mock Behavior") If you're asserting that a mock was called, you're testing the mock works, not the code. **Red flags:** * `expect(mockFn).toHaveBeenCalled()` without verifying the mock produces real effects * Assertions on `*-mock` test IDs * Tests that pass when mock is present, fail when removed **Fix:** Test what the code actually does. If external APIs must be mocked, verify the real API would receive correct data. ### Rule 4: Test First, Watch It Fail[​](#rule-4-test-first-watch-it-fail "Direct link to Rule 4: Test First, Watch It Fail") If you didn't see the test fail, you don't know it tests the right thing. **Process:** 1. Write failing test 2. Verify it fails for expected reason (missing feature, not typo) 3. Write minimal code to pass 4. Verify it passes 5. Refactor if needed **Red flags:** * Test passes immediately when written * Can't explain why test failed * "I'll add tests later" ### Rule 5: No Test-Only Methods in Production Code[​](#rule-5-no-test-only-methods-in-production-code "Direct link to Rule 5: No Test-Only Methods in Production Code") Production classes shouldn't have methods only tests use. **Wrong:** ``` class Session { destroy() { /* only used in tests */ } } ``` **Right:** ``` // In test-utils/ export function cleanupSession(session: Session) { ... } ``` ### Rule 6: Verify Before Claiming Complete[​](#rule-6-verify-before-claiming-complete "Direct link to Rule 6: Verify Before Claiming Complete") "Should pass now" is not verification. **Process:** 1. Run the actual test command 2. Read the output 3. Confirm pass/fail count 4. Only then claim status ## When to Use Each Test Type[​](#when-to-use-each-test-type "Direct link to When to Use Each Test Type") | Type | When to Add | Example | | --------------- | ------------------------------------------------------------------- | -------------------------------------------------------------- | | **Integration** | New usage pattern, new external API interaction, new data flow path | Collector → Destination → gtag() | | **Unit** | Combinatorics, edge cases, pure function logic | Mapping variations, core utilities | | **Contract** | Boundary validation | Destination output matches vendor API, source input validation | **Guideline:** Integration tests prove things work when stuck together. Unit tests efficiently cover variations. Contract tests catch API drift. ## Simulation Testing[​](#simulation-testing "Direct link to Simulation Testing") Simulation testing uses the CLI `push` command with `--simulate` flags. The collector does not export a `simulate()` function — simulation is a CLI concern that maps to `mock`/`disabled` config properties at runtime. **CLI usage:** ``` # Simulate a destination (mocks its push, captures API calls) walkeros push flow.json -e '{"entity":"page","action":"view"}' --simulate destination.ga4 # Simulate a source (captures events it pushes, disables all destinations) walkeros push flow.json --simulate source.browser # Mock a destination with a specific return value walkeros push flow.json -e event.json --mock destination.ga4='{"status":"ok"}' ``` **Programmatic usage:** ``` import { push } from '@walkeros/cli'; // Simulate a destination const result = await push( 'flow.json', { entity: 'page', action: 'view' }, { simulate: ['destination.ga4'], }, ); // result.usage = API call tracking data from wrapEnv // Simulate a source const result = await push('flow.json', undefined, { simulate: ['source.browser'], }); // result.captured = events captured from source env.push ``` **Key points:** * `--simulate destination.X` sets `config.mock = {}` on the target and `config.disabled = true` on all other destinations * `--simulate source.X` wraps `env.push` with a capture function and disables all destinations * Destination `/dev` env.push is auto-loaded to provide mock globals (fake `window.gtag`, etc.) * Returns `PushResult` with `result`, `captured` (source), and `usage` (destination) * The `mockEnv()` and env pattern examples above remain correct for unit testing individual step functions directly ## Package-Specific Approaches[​](#package-specific-approaches "Direct link to Package-Specific Approaches") | Package | Approach | | ----------------------- | ------------------------------------------------------------------------------- | | **core** | Unit tests only - pure functions, no env needed | | **collector** | Integration tests critical - input/output consistency is paramount | | **browser source** | Maintain walker algorithm coverage | | **web destinations** | Integration tests per unique pattern + unit tests for mappings, use env pattern | | **server destinations** | Same as web destinations | | **cli/docker** | Integration tests for spawn behavior, explore dev pattern to reduce duplication | | **sources** | Contract tests for input validation, integration tests for event capture | ## The env Pattern Deep Dive[​](#the-env-pattern-deep-dive "Direct link to The env Pattern Deep Dive") ### How env Works[​](#how-env-works "Direct link to How env Works") Each destination/source defines an `env` type that specifies external dependencies: ``` // Destination-specific env type export interface Env extends DestinationWeb.Env { window: { gtag: Gtag.Gtag; dataLayer: unknown[]; }; document: { createElement: (tagName: string) => HTMLElement; head: { appendChild: (node: unknown) => void }; }; } ``` ### mockEnv() Function[​](#mockenv-function "Direct link to mockEnv() Function") The `mockEnv()` function from `@walkeros/core` creates a Proxy that intercepts all function calls: ``` import { mockEnv } from '@walkeros/core'; const calls: Array<{ path: string[]; args: unknown[] }> = []; const testEnv = mockEnv(examples.env.push, (path, args) => { calls.push({ path, args }); // Optionally return a value }); // Now use testEnv in your destination context await destination.push(event, { ...context, env: testEnv }); // Assert on captured calls expect(calls).toContainEqual({ path: ['window', 'gtag'], args: ['event', 'purchase', expect.objectContaining({ value: 99.99 })], }); ``` ### dev.ts Structure[​](#devts-structure "Direct link to dev.ts Structure") Each package with external dependencies should have: ``` // src/dev.ts export * as schemas from './schemas'; export * as examples from './examples'; // src/examples/index.ts export * as env from './env'; export * as events from './events'; export * as mapping from './mapping'; export * as step from './step'; // Step examples { in, out } ``` ### Testing Sources with Injected env[​](#testing-sources-with-injected-env "Direct link to Testing Sources with Injected env") Sources accept platform dependencies via `env`. Mock `window`, `document`, or library imports by passing them through `env` instead of mocking globals. Type the mock against the source's own `Env` type, not against the global `Window`. A source narrows `Env.window` to only the members it touches, so the mock satisfies that narrowed shape directly with **no `as unknown as Window` cast**: ``` import type { Env } from '../types'; // the source's narrowed Env // Instead of mocking window.performance globally: const env: Env = { window: { performance: { getEntriesByType: jest.fn().mockReturnValue([{ type: 'navigate' }]), }, location: { href: 'https://test.com/' }, }, }; await createSessionSource(collector, undefined, env); // Instead of mocking express import, type the binding against the source's // injected dependency type so the mock is assignable without a cast: import type { SourceEnv } from './types'; const express: SourceEnv['express'] = Object.assign( jest.fn().mockReturnValue(mockApp), { json: jest.fn().mockReturnValue(middleware) }, ); await sourceExpress(createSourceContext({}, { express })); ``` This pattern avoids global state pollution between tests, enables simulation in non-browser environments, and stays cast-free because the source narrows its own `Env` to exactly the members it uses (the same declare-global + narrowed-`Env` approach destinations use — see [create-destination §3.3.1](/skills/walkeros-create-destination.md)). ## Red Flags - Stop and Fix[​](#red-flags---stop-and-fix "Direct link to Red Flags - Stop and Fix") * Using `jest.mock()` for internal modules when `env` pattern is available * Tests that don't import from `../dev` * Assertions only checking mock call counts * Tests with extensive mock setup (>50% of test is setup) * Test-only methods added to production classes * Claiming tests pass without running them ## Commands[​](#commands "Direct link to Commands") Verification tier (per `/workspaces/developer/AGENT.md` rule 11): ``` # L1, the default during a task: typecheck, lint, test for the touched package cd /workspaces/developer/walkerOS npm run verify:touched -- core npm run verify:touched -- web-destination-gtag # L2, at plan completion: only packages affected since origin/main git fetch origin main --depth=1 npm run verify:affected # L3, before pushing or marking PR-ready: critical path + affected npm run test:smoke # Single test file (still useful while iterating) cd packages/ && npm run test -- path/to/file.test.ts # Watch mode (single package) cd packages/ && npm run test -- --watch ``` Avoid bare `npm run test` at root inside per-task steps. That is L4 (full suite, 10-15 min) and is reserved for plan completion when the plan touched shared infra, or for explicit user request. ## Reading a Step Back from the Collector[​](#reading-a-step-back-from-the-collector "Direct link to Reading a Step Back from the Collector") When a test calls a step's raw push directly through the collector bag (`collector.sources.X.push`, `collector.destinations.X.push`, etc.), use the typed accessors from `@walkeros/core` instead of casting: ``` import { Source } from '@walkeros/core'; const src = Source.getSource(collector, 'testSource'); await src.push({ method: 'GET', path: '/api/data' }); ``` Available helpers: `Source.getSource`, `Destination.getDestination`, `Transformer.getTransformer`, `Store.getStore`. Each accepts an optional type parameter to recover the per-step generic that the bag's index signature erases on read. Each throws ` not found: ` when the id is unknown. Do not write `collector.sources.X.push as any` or `collector.sources.X.push as (rawData: ...) => Promise<...>`. The accessor exists exactly to remove that boundary cast. ## Destination Test Duplication, Pending Cleanup[​](#destination-test-duplication-pending-cleanup "Direct link to Destination Test Duplication, Pending Cleanup") As of 2026-04-29, web and server destination tests share substantial scaffolding (init validation, missing-settings rejection, mock setup, push assertions) with no shared harness. Audit found roughly 1500-2500 lines of repetition across 47 destinations. A typed shared harness is planned as a follow-on initiative. When writing a new destination test today: * Keep destination-specific behavior (mapping rules, vendor-specific batch shapes, edge cases) in the test file. * The boilerplate sections (env clone, `jest.clearAllMocks`, missing-required-settings, init returns valid contract) are candidates for extraction. Mirror the shape used by an existing destination of the same family (web vs server) so the future migration is mechanical. * Do NOT invent a new private harness inside the destination. If the missing harness blocks you, flag it. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-development](/skills/walkeros-understanding-development.md) - Development conventions and workflow **Reference:** * [AGENT.md](https://github.com/elbwalker/walkerOS/blob/main/AGENT.md) - Development guide --- # Understanding walkerOS Destinations ## Overview[​](#overview "Direct link to Overview") Destinations receive processed events from the collector and deliver them to third-party tools (analytics, marketing, data warehouses). **Core principle:** Destinations transform and deliver. They don't capture or process—that's sources and collector. ## Destination Interface[​](#destination-interface "Direct link to Destination Interface") See [packages/core/src/types/destination.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/destination.ts) for canonical interface. | Method | Purpose | Required | | --------------------------- | -------------------------- | ------------ | | `init(context)` | Load scripts, authenticate | Optional | | `push(event, context)` | Transform and send event | **Required** | | `pushBatch(batch, context)` | Batch processing | Optional | | `destroy(context)` | Cleanup on shutdown | Optional | | `config` | Settings, mapping, consent | **Required** | `destroy?: DestroyFn` — Optional cleanup method. Called during `command('shutdown')`. Use to close DB connections, flush buffers, or release SDK clients. Receives `{ id, config, env, logger }`. ## The env Pattern[​](#the-env-pattern "Direct link to The env Pattern") Destinations use dependency injection via `env` for external APIs. This enables testing without mocking. ``` // Destination defines its env type export interface Env extends DestinationWeb.Env { window: { gtag: Gtag.Gtag; dataLayer: unknown[]; }; } // Destination uses env, not globals async function push(event, context) { const { env } = context; env.window.gtag('event', mappedName, mappedData); } ``` ### Testing with env[​](#testing-with-env "Direct link to Testing with env") **REQUIRED SKILL:** See [testing-strategy](/skills/walkeros-testing-strategy.md) for full testing patterns. ``` import { mockEnv } from '@walkeros/core'; import { examples } from '../dev'; const calls: Array<{ path: string[]; args: unknown[] }> = []; const testEnv = mockEnv(examples.env.push, (path, args) => { calls.push({ path, args }); }); await destination.push(event, { ...context, env: testEnv }); expect(calls).toContainEqual({ path: ['window', 'gtag'], args: ['event', 'purchase', expect.any(Object)], }); ``` ### The env as observability seam[​](#the-env-as-observability-seam "Direct link to The env as observability seam") The env pattern is also the interception seam for trace-level vendor-call capture. At `trace` level the collector may attach an `observe` recorder under the `observe` key of the env, and `@walkeros/web-core`'s `getEnv` wraps the callables the destination declared as observable, recording each call. A destination that reaches its vendor through `getEnv(env)` is automatically observable; one that reaches a global directly, bypassing `env`, is not. This is one more reason to route every external call through `env`. ## Destination Config[​](#destination-config "Direct link to Destination Config") ``` config: { settings: { /* destination-specific */ }, mapping: { /* event transformation rules */ }, data: { /* global data mapping */ }, consent: { /* required consent states */ }, policy: { /* processing rules */ }, queue: boolean, // queue events dryRun: boolean, // test mode queueMax: number, // consent-queued events cap (default 1000) dlqMax: number, // dead-letter queue cap (default 100) } ``` ## Buffer bounds[​](#buffer-bounds "Direct link to Buffer bounds") Each destination keeps two internal buffers: `queuePush` (consent-denied events) and `dlq` (failed pushes). Both are size-bounded with FIFO drop-oldest eviction. Defaults: `queueMax: 1000`, `dlqMax: 100`. Set either on a destination's config to override per destination. Operators read drop counts from `collector.status.dropped[stepId('destination', id)]?.queue` (consent-denied evictions) and `?.dlq` (DLQ evictions). Build the key with `stepId()` from `@walkeros/core`. Point-in-time sizes stay on `collector.status.destinations[id].queuePushSize` / `dlqSize`. ## Batch scheduling[​](#batch-scheduling "Direct link to Batch scheduling") Set `config.batch` on a destination (with `pushBatch` implemented) to batch **all** of its events into one shared buffer. No `'* *'` wildcard mapping rule is needed. The configuration shape is `batch?: number | { wait?, size?, age? }` at both the destination-config layer and the mapping-rule layer. * `wait` (ms): debounce window. The timer resets on every push. Legacy form `batch: 1000` is shorthand for `{ wait: 1000 }`. * `size`: hard count cap. Default `1000`. Flushes immediately when reached. * `age` (ms): hard age cap since the first entry of the current window. Default `30000`. Prevents debounce starvation under sustained load. A mapping rule's own `batch` splits that entity-action into its own buffer and overrides `config.batch` per field (`rule ?? config ?? default`). To batch only specific events, omit `config.batch` and set `batch` on those rules. Pending batches flush on shutdown. ### Per-event metadata[​](#per-event-metadata "Direct link to Per-event metadata") `Batch.entries[]` carries per-event `{ event, ingest?, respond?, rule?, data? }`. Destinations that need per-event request IDs or HTTP responses (BigQuery, mParticle, HubSpot) should read `entries` instead of assuming all events in a batch share one `ingest`. `batch.events` and `batch.data` are derived views kept for backward compatibility. ### Failure handling[​](#failure-handling "Direct link to Failure handling") If `pushBatch` throws (or returns a rejected Promise), the entire batch is routed to the destination's `dlq` and `status.destinations[id].failed` is incremented by the batch size. Per-item retry is the destination SDK's responsibility (BigQuery, Kafka, HubSpot each have their own backoff semantics). Counters (`count`, `out`) are bumped only after a successful flush. Operators also see `status.destinations[id].inFlightBatch`: the number of events buffered but not yet delivered. ### Out-of-band errors from an EventEmitter SDK[​](#out-of-band-errors-from-an-eventemitter-sdk "Direct link to Out-of-band errors from an EventEmitter SDK") `tryCatchAsync` only catches errors on the awaited `push`/`pushBatch` path. If a destination owns an SDK object that is an EventEmitter (a BigQuery `StreamConnection`, a Redis client, a Kafka producer), that object can emit `'error'` on a detached tick with no awaiter. An EventEmitter that emits `'error'` with zero listeners throws synchronously and crashes the process. A destination that owns such an object MUST attach an `'error'` listener and route the error through `context.reportError`, which is available on every step context: * `context.reportError(err)` (no event): a connection-level error between pushes. Logs (redacted) and bumps `status.connectionErrors[stepId]`. Does not count as a failed event. * `context.reportError(err, event)`: a specific event was lost. Routes it to the DLQ and bumps `failed`, exactly like an in-band push failure. `reportError` is guarded and must never be called in a way that throws back into the emitter tick. The Pub/Sub pull source (`sources/gcp/src/pubsub/pull`) is the reference for attaching an `'error'` listener. The runner's process guards are a backstop, but the listener is what gives clean DLQ routing, attribution, and redaction. Optionally pair this with the `breaker` config so a persistently broken transport stops retrying every event. ## Require vs Consent[​](#require-vs-consent "Direct link to Require vs Consent") Two separate mechanisms control when destinations receive events: | Mechanism | Purpose | Scope | Effect | | --------- | -------------------- | ----------------- | -------------------------------------------------------------------------------------- | | `require` | Delay initialization | Whole destination | Destination stays in `pending` until all required events fire (e.g., `walker consent`) | | `consent` | Filter events | Per-event | Events without matching consent are silently skipped or queued | **Require** gates the destination *lifecycle*. A destination with `require: ["consent"]` does not exist in the collector until a `"walker consent"` event fires. Until then, events are queued internally. **Consent** gates *individual event delivery*. A destination with `consent: { marketing: true }` only receives events where the collector's consent state (or the event's own `consent` field) includes `{ marketing: true }`. **State refresh on flush:** When queued events are flushed to a destination, they receive the *current* collector state (`consent`, `user`, `globals`) — not the stale state from when they were originally captured. Any state-mutation command (`walker consent`, `walker user`, `walker globals`, etc.) triggers a flush attempt. The consent gate still applies: events without required consent simply return to the queue. Both can be combined: ``` { "config": { "require": ["consent"], "consent": { "marketing": true } } } ``` This means: don't initialize until consent fires, then only accept events with marketing consent. **Simulation impact:** `require` causes "destination not found" errors in `flow_simulate` because the destination stays pending. Remove `require` temporarily for simulation testing. ## Policy[​](#policy "Direct link to Policy") Policy modifies the event BEFORE mapping rules run. Defined at config level (applies to all events) or rule level (applies to specific events): ``` { "config": { "policy": { "user_data.email": { "key": "user.email", "consent": { "marketing": true } } } } } ``` Policy supports consent-gated field injection — fields are only added when the required consent is present in the event. ## Destination Paths[​](#destination-paths "Direct link to Destination Paths") | Type | Path | Examples | | ------ | ------------------------------- | ------------------------------------ | | Web | `packages/web/destinations/` | gtag, meta, api, piwikpro, plausible | | Server | `packages/server/destinations/` | aws, gcp, meta | ## Template Destination[​](#template-destination "Direct link to Template Destination") Use as starting point: `packages/web/destinations/plausible/` ## Transformer Wiring[​](#transformer-wiring "Direct link to Transformer Wiring") Destinations can wire to post-collector transformer chains via the `before` property: ``` destinations: { gtag: { code: destinationGtag, before: 'redact' // First transformer to run before this destination } } ``` The transformer chain runs after collector enrichment, before the destination receives events. Each destination can have its own chain. See [understanding-transformers](/skills/walkeros-understanding-transformers.md) for chain details. ## Consent: Two-Layer Pattern[​](#consent-two-layer-pattern "Direct link to Consent: Two-Layer Pattern") Destinations that integrate vendor SDKs typically need two consent layers: **Layer 1: `config.consent`** — gates walkerOS event delivery. If consent is not granted, events don't reach the destination. This is the primary barrier. **Layer 2: `on('consent')`** — controls vendor SDK internals. Even when walkerOS stops sending events, the vendor SDK may still run its own behaviors (DOM capture, polling, fetching configs). Use `on('consent')` to pause/resume these. ``` on(type, context) { if (type !== 'consent') return; const consent = context.data; // Derive from config.consent keys — don't hardcode consent names const granted = Object.keys(config.consent || {}).every(k => consent[k]); vendorSdk.setOptOut(!granted); } ``` Both layers are needed for complete consent compliance. `config.consent` prevents data flow. `on('consent')` prevents vendor SDK side effects. ## Response Delegation (env.respond)[​](#response-delegation-envrespond "Direct link to Response Delegation (env.respond)") Destinations can customize HTTP responses by calling `context.env.respond?.({ body, status?, headers? })`. This is useful for destinations that need to signal success/failure back to the HTTP caller. First call wins (idempotent). The respond function is optional — only present when the source provides one. ## Setup (optional)[​](#setup-optional "Direct link to Setup (optional)") Destinations can implement an optional `setup()` lifecycle to provision external resources, for example a BigQuery dataset and table, a Pub/Sub topic, or a warehouse schema. Setup is **never** invoked by the runtime, push, init, or deploy. It runs only when an operator explicitly types `walkeros setup destination.`. The signature is `(ctx: LifecycleContext, Env>) => Promise`, where `LifecycleContext` carries `{ id, config, env, logger }`. Idempotency is the package's responsibility: the framework adds no opinion. Use `resolveSetup(ctx.config.setup, DEFAULTS)` from `@walkeros/core` to normalize the `boolean | object` shape into a concrete options object. See [walkeros-create-destination](/skills/walkeros-create-destination.md), [walkeros-understanding-sources](/skills/walkeros-understanding-sources.md), [walkeros-understanding-stores](/skills/walkeros-understanding-stores.md), and the `walkeros setup` CLI documentation for the authoring template and operator workflow. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - How destinations fit in architecture * [walkeros-understanding-mapping](/skills/walkeros-understanding-mapping.md) - Configure transformations * [walkeros-create-destination](/skills/walkeros-create-destination.md) - Create new destination **Source Files:** * [packages/core/src/types/destination.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/destination.ts) - Interface **Package READMEs:** * [packages/web/destinations/gtag/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/destinations/gtag/README.md) - gtag example * [packages/web/destinations/plausible/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/destinations/plausible/README.md) - Plausible (template) **Documentation:** * [Website: Destinations](/docs/destinations/.md) - Overview * [Website: Create Your Own](/docs/destinations/create-your-own.md) - Guide --- # Understanding walkerOS Development ## Overview[​](#overview "Direct link to Overview") walkerOS follows extreme programming principles with strict conventions. This skill is your foundation before writing any code. **Core principle:** DRY, KISS, YAGNI. Test first. Verify before claiming complete. ## Commands[​](#commands "Direct link to Commands") | Command | Purpose | | ---------------- | --------------------------- | | `npm install` | Install all dependencies | | `npm run dev` | Watch mode for all packages | | `npm run build` | Build all packages | | `npm run format` | Prettier formatting | **Verification cheatsheet:** * While iterating (L1, touched package): `npm run verify:touched -- ` * At plan completion (L2, affected since `origin/main`): `npm run verify:affected` See `/workspaces/developer/AGENT.md` rule 11 for the full doctrine. ## XP Principles (Non-Negotiable)[​](#xp-principles-non-negotiable "Direct link to XP Principles (Non-Negotiable)") | Principle | In Practice | | ------------ | -------------------------------------------------------- | | **DRY** | Use `@walkeros/core` utilities, don't reimplement | | **KISS** | Minimal code to solve the problem | | **YAGNI** | Only implement what's requested | | **TDD** | Test first, watch it fail, then implement | | **No `any`** | Never use `any` in production code (tests are exception) | ## Folder Structure[​](#folder-structure "Direct link to Folder Structure") ``` packages/ ├── core/ # Platform-agnostic types, utilities, schemas ├── collector/ # Central event processing engine ├── config/ # Shared config (eslint, jest, tsconfig, tsup) ├── web/ │ ├── core/ # Web-specific utilities │ ├── sources/ # browser, dataLayer │ └── destinations/ # gtag, meta, api, piwikpro, plausible └── server/ ├── core/ # Server-specific utilities ├── sources/ # gcp └── destinations/ # aws, gcp, meta apps/ ├── walkerjs/ # Ready-to-use browser bundle ├── quickstart/ # Code examples (source of truth for patterns) └── demos/ # Demo applications ``` ## Core Package Usage[​](#core-package-usage "Direct link to Core Package Usage") **Always import from `@walkeros/core`:** ``` // Types import type { WalkerOS } from '@walkeros/core'; // Utilities import { getEvent, createEvent, // Event creation getMappingEvent, getMappingValue, // Transformations isString, isObject, isDefined, // Type checking assign, clone, // Object operations tryCatch, tryCatchAsync, // Error handling } from '@walkeros/core'; ``` **Config package for shared tooling:** * ESLint config: `@walkeros/config/eslint` * Jest config: `@walkeros/config/jest` * TSConfig: `@walkeros/config/tsconfig` * Tsup config: `@walkeros/config/tsup` ## Editing core Config types[​](#editing-core-config-types "Direct link to Editing core Config types") Core component configs live in two places: * TS interface: `packages/core/src/types/{destination,source,transformer,store,collector}.ts` * Zod schema: `packages/core/src/schemas/{destination,source,transformer,store,collector}.ts` Both are hand-written and mirror each other. TS stays authoritative because Zod's inferencer collapses recursive types (`Routes`, `MatchExpression`, `Value`) to `unknown`. Zod drives runtime validation, JSON Schema emission, and website Configuration reference tables. When adding, renaming, or removing a Config field, update BOTH files. A compile-time drift guard at `packages/core/src/schemas/__tests__/config-drift.test-d.ts` fails `tsc` if the key sets diverge. The guard checks keys only; value types may differ (recursion, generic slots). Run `npm run verify:touched -- core` to verify. ## Error visibility contract[​](#error-visibility-contract "Direct link to Error visibility contract") Top-level boundaries in the collector (`createPush` in `push.ts`, `createCommand` in `command.ts`) wrap their inner pipeline in `tryCatchAsync`. The `onError` callback MUST do two things: * log a structured error via `collector.logger.error(message, { ... })` with enough context to reproduce (event/ingest for push, command/data for command), * increment `collector.status.failed`. An empty `onError` is a defect: it swallows the exception, returns `{ ok: false }` silently, and leaves the operator blind. Use `packages/collector/src/push.ts` and `packages/collector/src/command.ts` as the canonical pattern. Two categories of caught error: 1. **Internal walkerOS pipeline failures** (push, command, mapping outer wrap in `mapping.ts`, source factory / init / queueOn flush in `source.ts`, transformer init in `transformer.ts`, destination init in `destination.ts`): log AND `status.failed++`. 2. **User-supplied callbacks** (mapping `condition` / `fn` / `validate`, `on` subscriptions in `on.ts`): log only. `status.failed` stays a pipeline-health signal; user-code visibility goes via logs. For invariant violations or operator-initiated aborts that must crash the host process, throw `FatalError` (exported from `@walkeros/core`). `FatalError` bypasses every boundary catch in both categories so a supervisor can terminate cleanly. Standard `Error` is absorbed, logged, and (for category 1) counted. The log message verb identifies the site: `'mapping condition failed'`, `'source factory failed'`, `'transformer init failed'`, `'on callback failed'`, etc. Operators grep for the verb. No `kind` field is required except in `on.ts`, where seven sites share one verb and disambiguate via a typed `kind` field on the structured payload. ## Testing[​](#testing "Direct link to Testing") **REQUIRED SKILL:** Use `testing-strategy` for detailed testing patterns. Quick reference: * Use `env` pattern for mocking (not Jest mocks) * Import from `dev.ts` for examples * Test first, watch it fail * Verify before claiming complete ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-testing-strategy](/skills/walkeros-testing-strategy.md) - Testing patterns and env mocking **Source Files:** * [packages/core/](https://github.com/elbwalker/walkerOS/blob/main/packages/core) - Core utilities * [packages/config/](https://github.com/elbwalker/walkerOS/blob/main/packages/config) - Shared configuration * [apps/quickstart/](https://github.com/elbwalker/walkerOS/blob/main/apps/quickstart) - Validated examples --- # Understanding walkerOS Events ## Overview[​](#overview "Direct link to Overview") walkerOS events are self-describing, stateless, vendor-agnostic data structures. They capture user interactions in a standardized format that can be transformed for any destination. **Core principle:** Events describe WHAT happened, not WHERE it goes. Stateless. Self-describing. Industry-agnostic. ## Entity-Action Naming (Critical)[​](#entity-action-naming-critical "Direct link to Entity-Action Naming (Critical)") **STRICT REQUIREMENT:** All events use "entity action" format with space separation. ``` // Correct 'page view'; 'product add'; 'order complete'; 'button click'; // Wrong 'page_view'; // underscore 'pageview'; // no separator 'purchase'; // no entity 'add_to_cart'; // wrong format ``` **Parsing:** `const [entity, action] = event.split(' ')` * **Entity:** Noun (page, product, user, order, button) * **Action:** Verb (view, add, complete, click, login) ## Event Properties[​](#event-properties "Direct link to Event Properties") See [packages/core/src/types/walkeros.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/walkeros.ts) for canonical types (Event interface, plus event helpers in `event.ts`). | Property | Type | Purpose | Example | | ----------------- | ------ | ------------------------------------------------------------ | -------------------------------------- | | `name` | string | "entity action" format | `"product view"` | | `data` | object | Entity-specific properties | `{ id: "P123", price: 99 }` | | `context` | object | State/environment info (optional) | `{ stage: ["checkout", 1] }` | | `globals` | object | Global properties | `{ language: "en" }` | | `user` | object | User identification | `{ id: "user123" }` | | `nested` | array | Related entities (optional) | `[{ entity: "product", data: {...} }]` | | `consent` | object | Consent states | `{ marketing: true }` | | `id` | string | W3C span\_id, 16 lowercase hex chars, generated by collector | `"0123456789abcdef"` | | `timestamp` | number | Auto-generated Unix ms | `1647261462000` | | `entity` | string | Parsed from name | `"product"` | | `action` | string | Parsed from name | `"view"` | | `source.type` | string | Source kind (`browser`, `dataLayer`, `cookiefirst`, ...) | `"browser"` | | `source.platform` | string | Runtime platform (`web`, `server`) | `"web"` | | `source.schema` | string | Source-emitted schema/version (optional) | `"datalayer-v2"` | | `source.trace` | string | Run-scoped W3C trace\_id, shared by every event of a run | `"0123...cdef"` (32 hex) | | `source.count` | number | Per-run emission sequence (1, 2, 3, ...) | `1` | | `source.release` | object | Per-flow config release map, accumulates across crossings | `{ web: "3", server: "5" }` | ### data Property[​](#data-property "Direct link to data Property") Entity-specific properties. Schema-free but consistent within entity type. ``` // product entity data: { id: "P123", name: "Laptop", price: 999, currency: "USD" } // page entity data: { title: "Home", path: "/", referrer: "https://..." } ``` ### context Property[​](#context-property "Direct link to context Property") Hierarchical state information. Format: `{ name: [value, order] }`. Optional. ``` context: { stage: ["checkout", 1], // checkout stage, first step test: ["variant-A", 0], // A/B test variant group: ["premium", 2] // user segment } ``` ### globals Property[​](#globals-property "Direct link to globals Property") Properties that apply to ALL events in the session. ``` globals: { language: "en", currency: "USD", environment: "production" } ``` ### nested Property[​](#nested-property "Direct link to nested Property") Related entities captured together. Optional. ``` // Order with line items nested: [ { entity: 'product', data: { id: 'P1', quantity: 2 } }, { entity: 'product', data: { id: 'P2', quantity: 1 } }, ]; ``` ### user Property[​](#user-property "Direct link to user Property") User identification across sessions. ``` user: { id: "user123", // Your user ID device: "device456", // Device fingerprint session: "sess789" // Session ID } ``` ### source Property[​](#source-property "Direct link to source Property") Where the event came from. The collector enriches the event with `source` data based on which source emitted it. ``` source: { type: 'browser', // source kind (browser, dataLayer, cookiefirst, ...) platform: 'web', // runtime: 'web' or 'server' version: '4.0.0', // external source-emitter version (collector no longer sets this) schema: 'datalayer-v2', // optional schema/version emitted by the source trace: '0123...cdef', // run-scoped W3C trace_id, groups all events of a run count: 1, // per-run emission sequence (1, 2, 3, ...) release: { web: '3' }, // per-flow config release map, accumulates web to server url: 'https://...', // page URL (web only, set by web-context transformer) referrer: '...', // page referrer (web only) tool: 'cli', // tool that produced the event (optional) command: 'simulate', // command name (optional) } ``` CMP and other non-page sources do NOT set `source.url`/`source.referrer` - that's the responsibility of a web-context transformer. `source.trace` is the run grouping key: the collector mints a fresh run-scoped trace\_id on each run and stamps it (plus a per-run `source.count`) on every event when absent. It is preserved unchanged when an event is forwarded from web to server, so the whole pipeline shares one trace. `source.release` records which config handled the event, keyed by flow name. Each flow the event passes through adds its own entry (that flow's config release), so an event captured on the web and processed on the server carries both flow releases, and a warehouse row shows exactly which config version touched it. This is distinct from `source.version`, the source package version set only by external source emitters (the collector no longer stamps it). ## Migration from v3[​](#migration-from-v3 "Direct link to Migration from v3") If you have existing v3 events or configs, here is the v4 mapping: | v3 | v4 | | --------------------------------- | ------------------------------------------------------------------- | | `event.id` = `"--"` | `event.id` = W3C span\_id (16 lowercase hex chars) | | `event.version` | removed - see `source.version` and `source.schema` | | `event.group` | removed - use `source.trace` (run-scoped trace\_id) for correlation | | `event.count` | removed - use `source.count` (per-run emission sequence) | | `event.source.id` | `event.source.url` (when it meant page URL) | | `nested: [{ type, data }]` | `nested: [{ entity, data }]` (`Entity.entity` replaces `.type`) | ## Design Principles[​](#design-principles "Direct link to Design Principles") ### Statelessness[​](#statelessness "Direct link to Statelessness") Events are immutable snapshots. They don't reference previous events or maintain state. ### Self-Describing[​](#self-describing "Direct link to Self-Describing") Events contain all context needed to understand them. No external lookups required. ### Vendor-Agnostic[​](#vendor-agnostic "Direct link to Vendor-Agnostic") Events use generic concepts (product, order) not vendor-specific (GA4 item, FB content). Transformation to vendor formats happens in **mapping**, not in event creation. ## Creating Events[​](#creating-events "Direct link to Creating Events") ``` import { elb } from '@walkeros/collector'; // Basic event await elb('page view', { title: 'Home', path: '/' }); // With all properties await elb( 'product add', { id: 'P123', price: 99 }, // data { stage: ['cart', 1] }, // context (optional) { currency: 'USD' }, // globals (optional) ); ``` ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-mapping](/skills/walkeros-understanding-mapping.md) - Transform events for destinations * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Data flow architecture **Source Files:** * [packages/core/src/types/walkeros.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/walkeros.ts) - Event types * [packages/core/src/schemas/](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/schemas) - Event schemas **Documentation:** * [Website: Event Model](/docs/getting-started/event-model.md) - User-facing docs * [walkeros.io/docs](https://www.walkeros.io/docs/) - Public documentation --- # Understanding walkerOS Flow ## Overview[​](#overview "Direct link to Overview") walkerOS follows a **Source → Collector → Destination(s)** architecture for composable, modular event processing. **Core principle:** Separation of concerns. Each component has one job. Components are composable and replaceable. ## The Flow Pattern[​](#the-flow-pattern "Direct link to The Flow Pattern") ``` [Source.before] → Sources → [Source.next] → Collector → [Dest.before] → Destinations → [Dest.next] (Preprocessing) (Capture) (Pre-chain) (Processing) (Post-chain) (Delivery) (Post-push) Consent-exempt: Post-consent: - Decode - Validation - Event creation - Validation - Push - Audit logging - Validate format - Enrichment - Enrichment - Enrichment - Send - Response parsing - Authenticate - Redaction - Consent check - Routing - Store - Webhooks ``` ## Key Concepts[​](#key-concepts "Direct link to Key Concepts") ### Composability[​](#composability "Direct link to Composability") A Flow combines components. You can: * Use multiple sources feeding one collector * Route events to multiple destinations * Swap components without changing others ### The Flow Type[​](#the-flow-type "Direct link to The Flow Type") See [packages/core/src/types/flow.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/flow.ts) for the canonical interface. ``` // Conceptual structure (see source for full type) interface Flow { stores?: Record; sources?: Record; transformers?: Record; destinations?: Record; collector?: Collector.InitConfig; } ``` ### Universal Push Interface[​](#universal-push-interface "Direct link to Universal Push Interface") **All components communicate via `push` functions:** | Component | Push Signature | Purpose | | ----------- | ----------------------------- | --------------------- | | Source | `push(input) → events` | Capture external data | | Collector | `push(event) → void` | Process and route | | Destination | `push(event, context) → void` | Transform and deliver | The `elb()` function is an alias for `collector.push` - used for component wiring. ### startFlow Helper[​](#startflow-helper "Direct link to startFlow Helper") See [packages/collector/src/flow.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/flow.ts) for the `startFlow` function. ``` import { startFlow } from '@walkeros/collector'; const { collector, elb } = await startFlow({ stores: { /* key-value storage, init first, destroy last */ }, sources: { /* ... */ }, transformers: { /* ... */ }, destinations: { /* ... */ }, }); ``` ## Ingest: Mutable Pipeline Context[​](#ingest-mutable-pipeline-context "Direct link to Ingest: Mutable Pipeline Context") walkerOS uses a two-layer data model: * **Event** - strict schema, structured analytics data (name, data, context, etc.) * **Ingest** - free-form mutable context that flows alongside events Any step can read and write arbitrary keys on ingest. The runtime manages `_meta`: * `_meta.hops` - increments per step (safety valve at 256) * `_meta.path` - ordered list of step IDs visited (path\[0] = source ID) ``` // In a transformer push: async (event, context) => { context.ingest.botScore = 0.95; // Write freely context.ingest.geo = { country: 'DE' }; console.log(context.ingest._meta.path); // ['express', 'validate', 'enrich'] return { event }; }; ``` Ingest is cloned per destination to prevent cross-contamination in parallel processing. After a destination push, the response is available at `ingest._response`. ## Stores[​](#stores "Direct link to Stores") Three pipeline components (Source / Transformer / Destination) plus Stores as **passive infrastructure**. Stores are not a fourth pipeline stage - they're key-value storage that other components consume via `env`. They have no `push`, no `next`, no `before`; they sit alongside the pipeline rather than inside it. * Referenced via `$store.storeId` in `env` values (bundled mode) or passed directly as store instances (integrated mode) * **Init first, destroy last** - stores are available before any source, transformer, or destination starts, and outlive them on shutdown * **No chains** - stores don't participate in the event pipeline. Components access them through their `env`. * Implementations: `@walkeros/server-store-fs` (async, filesystem), `@walkeros/server-store-s3` (async, S3-compatible), `@walkeros/server-store-gcs`, `@walkeros/server-store-sheets`. The collector ships a built-in in-memory cache tier — enable it on any store via `Flow.Store.cache` instead of declaring a separate memory store. ``` { "stores": { "data": { "package": "@walkeros/server-store-fs" } }, "transformers": { "fingerprint": { "package": "@walkeros/server-transformer-fingerprint", "env": { "store": "$store.data" } } } } ``` See [walkeros-understanding-stores](/skills/walkeros-understanding-stores.md) for the full store interface and lifecycle. ## Separation of Concerns[​](#separation-of-concerns "Direct link to Separation of Concerns") | Concern | Handled By | NOT Handled By | | ---------------- | -------------- | ----------------------- | | Event capture | Sources | Collector, Destinations | | Event structure | Event model | Components | | Consent checking | Collector | Sources, Destinations | | Transformation | Mapping system | Raw push calls | | Delivery | Destinations | Sources, Collector | ## Step-Level Primitives[​](#step-level-primitives "Direct link to Step-Level Primitives") Every step (source, transformer, destination) supports a small set of inline primitives alongside its package wiring: `cache`, `mapping`, and `consent`. Event shapes are not a step-level primitive. They live in the top-level `contract` block (a sibling of `flows`) as named JSON Schemas, and enforcement is an explicit `@walkeros/transformer-validate` step that references a contract via `$contract.`. See [Website: Contract](/docs/getting-started/flow/contract.md) for the contract shape and [Website: Validate](/docs/getting-started/flow/validate.md) for runtime enforcement. ### How a step references its implementation[​](#how-a-step-references-its-implementation "Direct link to How a step references its implementation") Every step picks exactly one of three forms to point at its implementation: * `package: ""` alone loads the package's default export. The common case for sources and destinations. * `package: ""` plus `import: ""` loads a specific named export from that package. Use when a package ships multiple named exports or has no default export. * `code: { push, type?, init? }` is inline implementation, no package wiring. Useful for one-off custom logic in TypeScript flows. There is no string form of `code`. A bare step with no `package`, `import`, or `code` is a valid no-op for all four step kinds (handy as a chain-only or mapping-only transformer). ## Transformer Chains[​](#transformer-chains "Direct link to Transformer Chains") Transformers run at two points in the pipeline, configured via `next` and `before`: ### Pre-Collector Chain[​](#pre-collector-chain "Direct link to Pre-Collector Chain") Runs after source captures event, before collector processing: **Bundled mode (flow\.json):** ``` { "sources": { "browser": { "package": "@walkeros/web-source-browser", "next": "enrich" } }, "transformers": { "enrich": { "package": "@walkeros/transformer-enricher", "next": "redact" }, "redact": { "package": "@walkeros/transformer-redact" } } } ``` **Integrated mode (TypeScript):** ``` sources: { browser: { code: sourceBrowser, next: 'enrich' } }, transformers: { enrich: { code: transformerEnrich, config: { next: 'redact' } }, redact: { code: transformerRedact } } ``` Note: In flow\.json, `next` is at the reference level. The CLI bundler automatically moves it into `config.next` for runtime - you don't need to handle this yourself. ### Transformer before chain[​](#transformer-before-chain "Direct link to Transformer before chain") Each transformer can have its own `before` chain that runs before its push function: ``` { "transformers": { "enrich": { "before": "lookup", "package": "@walkeros/transformer-enricher" }, "lookup": { "package": "@walkeros/transformer-lookup" } } } ``` ### Post-Collector Chain[​](#post-collector-chain "Direct link to Post-Collector Chain") Runs after collector enrichment, before destination receives event: **Bundled mode (flow\.json):** ``` { "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "before": "redact" } }, "transformers": { "redact": { "package": "@walkeros/transformer-redact" } } } ``` **Integrated mode (TypeScript):** ``` destinations: { gtag: { code: destinationGtag, before: 'redact' } }, transformers: { redact: { code: transformerRedact } } ``` ### Post-push chain (`destination.next`)[​](#post-push-chain-destinationnext "Direct link to post-push-chain-destinationnext") Runs after destination push completes. The push response is available at `context.ingest._response`: **Bundled mode (flow\.json):** ``` { "destinations": { "api": { "package": "@walkeros/server-destination-api", "next": "auditLog" } }, "transformers": { "auditLog": { "package": "@walkeros/transformer-audit" } } } ``` **Integrated mode (TypeScript):** ``` destinations: { api: { code: destinationApi, next: 'auditLog' } }, transformers: { auditLog: { code: transformerAudit } } ``` ### Chain Resolution[​](#chain-resolution "Direct link to Chain Resolution") * `source.before` → consent-exempt preprocessing chain * `source.next` → starts pre-collector chain * `transformer.before` → pre-transform enrichment chain * `transformer.next` (flow\.json) or `transformer.config.next` (runtime) → links transformers * `destination.before` → starts post-collector chain per destination * `destination.next` → post-push processing chain ## Cross-Flow References (`$flow`)[​](#cross-flow-references-flow "Direct link to cross-flow-references-flow") When a single flow\.json defines multiple flows, any flow can pull values from another flow's `config` block via `$flow.(.)?`. The most common case is wiring a web flow's API destination to a server flow's deployed URL, so the two stay in sync without duplication. ``` { "version": 4, "flows": { "server": { "config": { "platform": "server", "url": "https://collect.example.com" }, "sources": { "http": { "package": "@walkeros/server-source-express" } } }, "web": { "config": { "platform": "web" }, "destinations": { "api": { "package": "@walkeros/web-destination-api", "config": { "settings": { "url": "$flow.server.url" } } } } } } } ``` `validate` warns on unresolved `$flow` references (lenient), `bundle` errors out (strict), so production builds never ship with an empty cross-flow value. ## Step Examples[​](#step-examples "Direct link to Step Examples") Each step in a flow (source, transformer, destination) can ship **step examples** -- structured `{ in, out }` pairs that define expected input/output behavior. ### The Three Type Zones[​](#the-three-type-zones "Direct link to The Three Type Zones") Steps sit at boundaries between arbitrary formats and walkerOS events: * **Source:** arbitrary `in` (HTTP request, DOM event) → walkerOS event `out` * **Transformer:** walkerOS event `in` → walkerOS event `out` (or `false`) * **Destination:** walkerOS event `in` → arbitrary `out` (vendor API call) See [using-step-examples](/skills/walkeros-using-step-examples.md) for the full ASCII diagram and detailed explanation. ### Example: Step with Examples in Flow Config[​](#example-step-with-examples-in-flow-config "Direct link to Example: Step with Examples in Flow Config") ``` { "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "config": { "measurementId": "G-XXXXXX" }, "examples": { "purchase": { "in": { "name": "order complete", "data": { "id": "ORD-123", "total": 149.97 } }, "out": [ "event", "purchase", { "transaction_id": "ORD-123", "value": 149.97 } ] } } } } } ``` Step examples enable `it.each` testing, CLI simulation with `--example`, and deep validation with `--deep`. See [using-step-examples](/skills/walkeros-using-step-examples.md) for the complete lifecycle. ## Flow Graph Connection Rules[​](#flow-graph-connection-rules "Direct link to Flow Graph Connection Rules") This section defines which components can connect to which, and how chains are resolved at runtime. Use it as the canonical reference for building flow graphs, validating configurations, and rendering UI visualizations. ### Valid connection matrix[​](#valid-connection-matrix "Direct link to Valid connection matrix") | From | To | Via Field | Valid? | | ----------- | ----------- | --------------------------- | --------------------- | | Source | Transformer | `source.before` | Yes (consent-exempt) | | Source | Transformer | `source.next` | Yes (pre-collector) | | Source | Collector | (implicit, no next) | Yes | | Source | Source | - | No | | Source | Destination | - | No | | Transformer | Transformer | `transformer.before` | Yes (pre-transform) | | Transformer | Transformer | `transformer.next` | Yes (chain continues) | | Transformer | Collector | (implicit, pre-chain ends) | Yes | | Transformer | Destination | (implicit, post-chain ends) | Yes | | Collector | Destination | (implicit, no before) | Yes | | Collector | Transformer | `destination.before` | Yes (post-chain) | | Destination | Transformer | `destination.next` | Yes (post-push) | | Collector | Source | - | No | ### Pre-transformer chains (`source.next`)[​](#pre-transformer-chains-sourcenext "Direct link to pre-transformer-chains-sourcenext") * Entry: `source.next: "transformerId"` or `source.next: ["t1", "t2"]` * Chaining: `transformer.next: "nextId"` walks forward; array stops walking * Exit: chain ends, event reaches collector * Multiple sources can reference the same transformer (fan-in) * No `next` = source connects directly to collector ### Post-transformer chains (`destination.before`)[​](#post-transformer-chains-destinationbefore "Direct link to post-transformer-chains-destinationbefore") * Entry: `destination.before: "transformerId"` or `destination.before: ["t1", "t2"]` * Same `transformer.next` chain-walking logic as pre-chains * Exit: chain ends, event reaches destination * Multiple destinations can share the same transformer * No `before` = collector connects directly to destination ### Chain resolution algorithm (`getNextSteps`)[​](#chain-resolution-algorithm-getnextsteps "Direct link to chain-resolution-algorithm-getnextsteps") See [packages/collector/src/transformer.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/transformer.ts) for the implementation. `getNextSteps` is the public dispatch helper that replaces the previous `walkChain` entry point. * **String start:** walks `transformer.next` links until chain ends * **Array start:** uses array as-is (explicit chain, no walking) * **Array `next` inside chain:** appends array elements and stops walking * **Circular references:** detected via visited set, silently breaks loop * **Non-existent transformer ID:** chain ends (no error, event proceeds without transformation) Note: `getNextSteps` is deterministic for the supplied event context. Static analyzers without a real event can only enumerate reachability under "match may pass or fail" speculation. ### Conditional routing (`one` operator)[​](#conditional-routing-one-operator "Direct link to conditional-routing-one-operator") The `next` and `before` properties accept a `Route` (`string | Route[] | RouteConfig`). A `RouteConfig` is a **disjoint union**: each config sets at most one of `next` (gated link), `one` (first-match dispatch), or `many` (all-match dispatch), never more than one. The `one` operator enables conditional routing evaluated against ingest data and picks the first entry whose `match` succeeds: ``` "next": { "one": [ { "match": { "key": "ingest.path", "operator": "prefix", "value": "/api" }, "next": "api-handler" }, { "next": "default" } ] } ``` * `one` entries are evaluated in order, first match wins * An entry without `match` always matches, use it as the fallback * No matching entry means the event passes through unchanged * Works on all chain positions: `source.before`, `source.next`, `transformer.before`, `transformer.next`, `destination.before`, and `destination.next` * Routes are compiled to closures at init time for fast per-event evaluation * See [packages/core/src/route.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/route.ts) for `compileNext()` and `resolveNext()` ### All-match dispatch (`many` operator)[​](#all-match-dispatch-many-operator "Direct link to all-match-dispatch-many-operator") Use `many` when every matching entry should produce an independent parallel flow (audit-while-process, multi-decoder fan-out). `many` terminates the main chain, each branch runs to its own exit. Available only pre-collector. Post-collector fan-out uses the destinations map. ``` "next": { "many": [ { "match": { "key": "event.consent.analytics", "operator": "eq", "value": "granted" }, "next": "ga4-pipeline" }, { "next": "audit-log" } ] } ``` ### Paths and pass-through steps (code-less transformer entries)[​](#paths-and-pass-through-steps-code-less-transformer-entries "Direct link to Paths and pass-through steps (code-less transformer entries)") A **path** is the multi-step chain through a flow's `transformers` section. A **pass-through step** (short: **pass**) is a single step inside a path that declares no `code` and no `package`; the runtime synthesizes its push from the operative fields the step does declare. Pass-through steps come in three variants: * **Chain-only:** only `before` and/or `next` set. A named hop that shares a chain across multiple call sites (avoids duplicating arrays). * **Cache-only:** only `cache` set. A dedup or short-circuit step. `cache.stop: true` at a pre-collector position halts the pipeline. * **Mapping-only:** only `mapping: Mapping.Config` set. A declarative event-to-event transform that mutates the event in-flight. ``` { "transformers": { "validateThenEnrich": { "before": ["validate", "enrich"] } }, "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "before": "validateThenEnrich" }, "meta": { "package": "@walkeros/web-destination-meta", "before": "validateThenEnrich" } } } ``` Transformer step entries follow a **closed schema**: unknown top-level keys are validation errors, and at least one operative field (`code` / `package` / `before` / `next` / `cache` / `mapping`) must be set. See [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) for full depth on the three variants, the closed-schema rule, and the dual semantic of `mapping` at the transformer position versus the destination position. ### Transformer sharing[​](#transformer-sharing "Direct link to Transformer sharing") A single transformer can appear in both pre-chains (`source.next`) and post-chains (`destination.before`). The same transformer pool is shared; role depends on which chain references it. ### Deferred activation (`require`)[​](#deferred-activation-require "Direct link to deferred-activation-require") * `source.config.require: ["consent"]` - source deferred until "consent" event fires * `destination.config.require: ["user"]` - destination deferred until "user" event fires * Multiple requirements: all must be fulfilled (each fires independently) ### Mapping and consent gating[​](#mapping-and-consent-gating "Direct link to Mapping and consent gating") * **Source-level:** `source.config.mapping` and `source.config.consent` - applied before pre-chain; blocks event entirely * **Destination-level:** `destination.config.mapping` and `destination.config.consent` - applied after post-chain; skips only that destination, queues denied events ### Canvas rendering rules (for UI graph visualization)[​](#canvas-rendering-rules-for-ui-graph-visualization "Direct link to Canvas rendering rules (for UI graph visualization)") * **Shared transformers (pre+post):** duplicate visually with a link indicator; editing one updates the other * **Router fan-out:** keep graph planar; trace edges individually * **Orphan transformers (not in any chain):** render grey/muted; gain color when connected * **Diamond patterns (fan-in + fan-out):** expected and valid * **Overlapping `destination.before` chains:** intentional (e.g., shared validator for monitoring) ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-events](/skills/walkeros-understanding-events.md) - Event model * [walkeros-understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface * [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination interface **Package READMEs:** * [packages/collector/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/README.md) - Collector details **Source Files:** * [packages/collector/src/flow.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/flow.ts) - startFlow implementation **Documentation:** * [Website: Flow](/docs/getting-started/modes/bundled.md) - Flow concept * [Website: Collector](/docs/collector/.md) - Collector docs --- # Understanding walkerOS Mapping ## Overview[​](#overview "Direct link to Overview") Mapping transforms data at multiple points in the walkerOS flow: 1. **Source → Collector**: Transform raw input (HTTP requests, dataLayer pushes) into walkerOS events 2. **Collector → Destination**: Transform walkerOS events into vendor-specific formats **Core principle:** Mapping is the universal transformation layer. Same strategies work everywhere in the flow. ## Core Functions[​](#core-functions "Direct link to Core Functions") See [packages/core/src/mapping.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/mapping.ts) for implementation. | Function | Purpose | | ----------------------------------------------- | ------------------------------------------- | | `getMappingEvent(event, rules)` | Find mapping rule for an event | | `getMappingValue(value, data, options)` | Transform a value using mapping config | | `processEventMapping(event, config, collector)` | Unified processing for sources/destinations | ### processEventMapping Flow[​](#processeventmapping-flow "Direct link to processEventMapping Flow") ``` 1. Apply config.policy (modifies event) 2. Find matching rule via getMappingEvent() 3. Apply rule.policy (modifies event) 4. Transform config.data (global) 5. Check rule.ignore (short-circuits if true) 5b. Read rule.silent (informational - destination honors it) 6. Override event.name if rule.name 7. Transform rule.data (event-specific) ``` *** ## Configuration Hierarchy[​](#configuration-hierarchy "Direct link to Configuration Hierarchy") ### Mapping.Config (Top Level)[​](#mappingconfig-top-level "Direct link to Mapping.Config (Top Level)") ``` interface Config { consent?: Consent; // Required consent for ALL events data?: Value; // Global data transformation include?: string[]; // Event sections to flatten into context.data policy?: Policy; // Pre-processing for ALL events mapping?: Rules; // Event-specific rules } ``` #### Mapping.Config.include[​](#mappingconfiginclude "Direct link to Mapping.Config.include") `include` lists event sections to **flatten** into the destination's `PushContext.data` before the rule runs. Sections are top-level keys of the event (`globals`, `user`, `consent`, `data`, `context`, `nested`, `source`, `custom`). The runtime helper `flattenIncludeSections` (in `packages/core/src/include.ts`) handles the merge. Use this when your mapping needs values from multiple event sections without plumbing each path through a `key:`/`map:` config. `include` is also available on `Mapping.Rule` (rule-level overrides config-level). ``` { include: ['globals', 'user'], mapping: { page: { view: { // user.id and globals.* are now reachable via top-level paths in // data/key/map values, e.g. `key: 'id'` for user.id. data: { map: { user_id: 'id', site: 'language' } }, }, }, }, } ``` ### Mapping.Rule (Per Event)[​](#mappingrule-per-event "Direct link to Mapping.Rule (Per Event)") ``` interface Rule { name?: string; // Override event name data?: Value; // Event-specific data transformation include?: string[]; // Event sections to flatten into context.data (rule-level) ignore?: boolean; // Skip this event entirely (no processing, no push) silent?: boolean; // Process settings side effects, skip destination default push policy?: Policy; // Event-specific pre-processing condition?: Function; // Match condition (for arrays) consent?: Consent; // Required consent for this rule settings?: unknown; // Custom event configuration batch?: number; // Batch size for grouping extend?: RulePatch; // Config-layer merge onto a package-shipped default rule remove?: string[]; // Output-layer: dotted paths stripped from the final payload } ``` #### `extend` and `remove` (patching package-shipped rules)[​](#extend-and-remove-patching-package-shipped-rules "Direct link to extend-and-remove-patching-package-shipped-rules") Some packages ship their own default mapping rules. A user rule at the same key normally replaces the default in full. Two keywords change that: * **`extend`** (config layer): a partial rule deep-merged onto the package-shipped default at init, before any event is evaluated. A `null` value clears an inherited field. Lets you add or override one field while keeping the rest. * **`remove`** (output layer): dotted paths stripped from the produced payload after evaluation, applied last. Useful for dropping PII or vendor-reserved fields without rewriting the full rule. A rule with **neither** keyword keeps the standard replace behavior. See the [website mapping docs](/docs/mapping/rule.md#patching-package-shipped-rules) for the authoritative reference and the GA4 example. ``` { "purchase": { "extend": { "data": { "map": { "affiliation": "params.ep.affiliation" } } }, "remove": ["currency"] } } ``` ### Silent vs Ignore[​](#silent-vs-ignore "Direct link to Silent vs Ignore") Both flags control rule behavior but have different semantics: * `ignore: true` - the rule matched but **nothing happens**. No data transform, no destination call, no side effects. Use for suppression. * `silent: true` - the rule matched and the destination `push()` **is** called. `settings.identify`, `settings.revenue`, `settings.group`, etc. still run. Only the destination's default forwarding call (e.g. `track()`, `capture()`, `event()`) is suppressed. Use for "identify without an event" style flows. If both flags are set on the same rule, `ignore` wins. Common use case: a `user login` event that should call `amplitude.identify()` but should not create a separate `track("user login")` event in Amplitude. ### Mapping.ValueConfig (Value Extraction)[​](#mappingvalueconfig-value-extraction "Direct link to Mapping.ValueConfig (Value Extraction)") ``` interface ValueConfig { key?: string; // Extract from path value?: Primitive; // Static fallback value fn?: Function; // Custom transformation map?: Record; // Object transformation loop?: [path, config]; // Array transformation set?: Value[]; // Create array from values condition?: Function; // Conditional extraction consent?: Consent; // Consent-gated extraction validate?: Function; // Value validation } ``` *** ## Event Matching[​](#event-matching "Direct link to Event Matching") Match events to transformation rules by entity and action. ``` const mapping = { // Exact match: "product view" → view_item product: { view: { name: 'view_item' }, add: { name: 'add_to_cart' }, }, // Wildcard action: "foo *" → foo_interaction foo: { '*': { name: 'foo_interaction' }, }, // Wildcard entity: "* click" → generic_click '*': { click: { name: 'generic_click' }, }, }; ``` ### Conditional Mapping (Array)[​](#conditional-mapping-array "Direct link to Conditional Mapping (Array)") Array of rules - first matching condition wins: ``` order: { complete: [ { condition: (event) => event.data?.value > 100, name: 'high_value_purchase', }, { name: 'purchase' }, // Fallback (no condition) ], } ``` **JSON with $code:** ``` { "order": { "complete": [ { "condition": "$code:(event) => event.data?.value > 100", "name": "high_value_purchase" }, { "name": "purchase" } ] } } ``` *** ## Value Mapping Strategies[​](#value-mapping-strategies "Direct link to Value Mapping Strategies") Common patterns shown below. For detailed examples of all 12 strategies, see [value-strategies.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-understanding-mapping/value-strategies.md). ### Essential Patterns[​](#essential-patterns "Direct link to Essential Patterns") ``` // Key extraction (string shorthand) 'data.price' // → event.data.price // Key with fallback { key: 'data.currency', value: 'USD' } // Use USD if missing // Static value { value: 'USD' } // Function transform { fn: (event) => event.data.price * 100 } // Convert to cents // Object map { map: { item_id: 'data.id', item_name: 'data.name' } } // Array loop { loop: ['nested', { map: { item_id: 'data.id' } }] } // Loop with "this" (single item as array) { loop: ['this', { map: { item_id: 'data.id' } }] } // Set (create array) { set: ['data.id'] } // → ["SKU-123"] // Fallback chain: Value[] at any value position (first defined value wins) [{ key: 'data.sku' }, { key: 'data.id' }, { value: 'unknown' }] // Consent-gated { key: 'user.email', consent: { marketing: true } } // Validate { key: 'data.email', validate: (v) => v.includes('@') } ``` *** ## Policy (Pre-Processing)[​](#policy-pre-processing "Direct link to Policy (Pre-Processing)") Policy modifies the event BEFORE mapping rules are applied. Use for: * Adding computed fields * Normalizing data structure * Consent-gated field injection ### Config-Level Policy[​](#config-level-policy "Direct link to Config-Level Policy") Applied to ALL events: ``` config: { policy: { 'user_data.external_id': 'user.id', 'custom_data.server_processed': { value: true }, }, mapping: { /* ... */ } } ``` ### Event-Level Policy[​](#event-level-policy "Direct link to Event-Level Policy") Applied after config policy, only for specific event: ``` mapping: { order: { complete: { policy: { 'enriched.total_cents': { fn: (event) => Math.round(event.data.total * 100) } }, name: 'purchase', data: { /* ... */ } } } } ``` ### Policy with Consent[​](#policy-with-consent "Direct link to Policy with Consent") ``` { "policy": { "user_data.em": { "key": "user.email", "consent": { "marketing": true } } } } ``` *** ## Rule Features[​](#rule-features "Direct link to Rule Features") ### Ignore Events[​](#ignore-events "Direct link to Ignore Events") ``` mapping: { test: { '*': { ignore: true } }, // Ignore all test events } ``` ### Batch Processing[​](#batch-processing "Direct link to Batch Processing") To batch **all** of a destination's events, set `config.batch` (no mapping rule needed): ``` config: { batch: { size: 5 }, // Flush after every 5 events (a bare number is the wait window in ms) } ``` A rule-level `batch` batches only that entity-action into its own buffer and overrides `config.batch` per field: ``` mapping: { order: { complete: { batch: { wait: 1000 }, // Only order complete batches, with a 1s debounce } } } ``` ### Custom Settings[​](#custom-settings "Direct link to Custom Settings") ``` mapping: { order: { complete: { name: 'purchase', settings: { priority: 'high', retryCount: 3 } } } } ``` *** ## $code: Prefix (JSON Configs)[​](#code-prefix-json-configs "Direct link to $code: Prefix (JSON Configs)") The `$code:` prefix enables JavaScript functions in JSON configurations: ``` { "fn": "$code:(event) => event.data.price * 100", "condition": "$code:(event) => event.data?.value > 100", "validate": "$code:(value) => value > 0" } ``` **Important:** The `$code:` prefix is processed by the CLI bundler. It converts JSON strings to actual JavaScript functions during build. ### Function Signatures[​](#function-signatures "Direct link to Function Signatures") All mapping callbacks share `(value, context)`. The second argument is a `Mapping.Context` object (see "Context object" below). | Context | Signature | | ------------------- | ----------------------------- | | `fn` | `(value, context) => result` | | `condition` (value) | `(value, context) => boolean` | | `condition` (rule) | `(event, context) => boolean` | | `validate` | `(value, context) => boolean` | | `loop` condition | `(value, context) => boolean` | One-arg signatures like `(value) => value.toUpperCase()` continue to work, TypeScript ignores the unused second arg. ### Context object[​](#context-object "Direct link to Context object") | Field | Type | Required | Description | | ----------- | ------------------------------- | -------- | ------------------------------------------ | | `event` | `WalkerOS.DeepPartialEvent` | yes | The root event being mapped | | `mapping` | `Mapping.Value \| Mapping.Rule` | yes | The surrounding mapping config (or rule) | | `collector` | `Collector.Instance` | yes | Active collector, use for `push` and queue | | `logger` | `Logger.Instance` | yes | Use for `info`/`warn`/`error`/`debug` | | `consent` | `WalkerOS.Consent` (optional) | no | Resolved consent at this evaluation point | *** ## Quick Reference[​](#quick-reference "Direct link to Quick Reference") ### Value Extraction Cheatsheet[​](#value-extraction-cheatsheet "Direct link to Value Extraction Cheatsheet") | Pattern | Result | | ----------------------------- | ------------------------------ | | `"data.id"` | Extract `event.data.id` | | `{ value: "USD" }` | Static `"USD"` | | `{ key: "x", value: "y" }` | Extract `x`, fallback to `"y"` | | `{ fn: (e) => ... }` | Custom function | | `{ map: {...} }` | Object transformation | | `{ loop: ["nested", {...}] }` | Array transformation | | `{ loop: ["this", {...}] }` | Single-item as array | | `{ set: ["a", "b"] }` | Create array `[valA, valB]` | | `[m1, m2, m3]` | Fallback chain | | `{ consent: {...} }` | Consent-gated | | `{ condition: fn }` | Conditional | | `{ validate: fn }` | Validated | ### Rule Features Cheatsheet[​](#rule-features-cheatsheet "Direct link to Rule Features Cheatsheet") | Feature | Purpose | | ----------- | ----------------------------------------------------------------------- | | `name` | Override event name | | `data` | Transform event data | | `ignore` | Skip event entirely (no processing, no push) | | `silent` | Run settings side effects, skip default forwarding | | `policy` | Pre-process event | | `condition` | Match condition (arrays) | | `consent` | Required consent | | `settings` | Custom configuration | | `batch` | Batch size | | `extend` | Config-layer merge onto a package-shipped default (null clears a field) | | `remove` | Output-layer: dotted paths stripped from the final payload | ### Config Features Cheatsheet[​](#config-features-cheatsheet "Direct link to Config Features Cheatsheet") | Feature | Purpose | | --------- | ----------------------------- | | `consent` | Required consent (all events) | | `data` | Global data transformation | | `policy` | Global pre-processing | | `mapping` | Event-specific rules | *** ## Complete Examples[​](#complete-examples "Direct link to Complete Examples") For full destination configuration examples (TypeScript + JSON), see [complete-examples.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-understanding-mapping/complete-examples.md). *** ## Mapping at the Transformer Position[​](#mapping-at-the-transformer-position "Direct link to Mapping at the Transformer Position") `Mapping.Config` accepts the same shape at three positions in the flow, but the semantic differs by position: | Position | What the mapping produces | | ----------- | --------------------------------------------------------- | | Source | A walkerOS event from raw input | | Transformer | A mutated walkerOS event that continues through the chain | | Destination | A vendor-shaped payload that the destination consumes | When a transformer step declares only a `mapping` (no `code`, no `package`), the collector synthesizes a push that runs `processEventMapping` against each event. Same keyword as the destination field, different semantic at this step position. See [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) for the pass-through-step model. ### Which fields apply at the transformer position[​](#which-fields-apply-at-the-transformer-position "Direct link to Which fields apply at the transformer position") Only **event-mutating** fields run; vendor-payload fields are no-ops with a one-time init warning: | Field | Transformer position | | ------------------------ | --------------------------------------------------------------------------------- | | `policy` | Applies, pre-processes the event before rule matching | | `include` | Applies, flattens event sections into mapping context | | `mapping[].policy` | Applies, per-event policy | | `mapping[].name` | Applies, **renames** the event (mutation is observable downstream) | | `mapping[].ignore` | Applies, drops the event **from the chain entirely** (no downstream step sees it) | | `mapping[].consent` | Applies, consent gate | | `data`, `mapping[].data` | Ignored at this position (event mutation does not produce a vendor payload) | | `mapping[].silent` | Ignored at this position (destination-only concept) | Note the `ignore: true` semantic shift: at a **destination** it skips delivery to that destination only; at a **transformer step** it drops the event from the chain so no downstream step (transformer or destination) sees it. ``` { "transformers": { "redactPII": { "mapping": { "policy": { "user.email": { "value": "[redacted]" } }, "mapping": { "test": { "*": { "ignore": true } }, "order": { "complete": { "name": "purchase" } } } } } } } ``` ## Where Mapping Lives[​](#where-mapping-lives "Direct link to Where Mapping Lives") | Location | Purpose | | ------------------------------------------ | ----------------------------------------- | | Source config | Transform raw input → walkerOS events | | Transformer step config | Mutate walkerOS events in-flight | | Destination config | Transform walkerOS events → vendor format | | `packages/core/src/mapping.ts` | Core mapping functions | | `packages/core/src/types/mapping.ts` | Type definitions | | `packages/cli/examples/flow-complete.json` | Comprehensive example (53 features) | *** ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-events](/skills/walkeros-understanding-events.md) - Event structure * [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination-side mapping * [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer-position mapping and pass-through-step variants * [walkeros-mapping-configuration](/skills/walkeros-mapping-configuration.md) - Recipes and patterns **Source Files:** * [packages/core/src/mapping.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/mapping.ts) - Implementation * [packages/core/src/types/mapping.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/mapping.ts) - Types **Detailed References:** * [value-strategies.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-understanding-mapping/value-strategies.md) - All 12 value extraction strategies with examples * [complete-examples.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-understanding-mapping/complete-examples.md) - Full destination config examples **Examples:** * [packages/cli/examples/flow-complete.json](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/examples/flow-complete.json) - Comprehensive example * [packages/cli/examples/flow-complete.md](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/examples/flow-complete.md) - Feature inventory --- # Understanding walkerOS Sources ## Overview[​](#overview "Direct link to Overview") Sources capture events from the external world (browser DOM, dataLayer, HTTP requests, cloud functions) and feed them to the collector. **Core principle:** Sources capture. They don't process or deliver—that's collector and destinations. ## Source Interface[​](#source-interface "Direct link to Source Interface") See [packages/core/src/types/source.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/source.ts) for canonical interface. ### Init Function (Context Pattern)[​](#init-function-context-pattern "Direct link to Init Function (Context Pattern)") Sources use a context-based initialization pattern: ``` import type { Source } from '@walkeros/core'; export const sourceMySource: Source.Init = async (context) => { const { config = {}, env, logger, id } = context; // ... }; ``` **Context contains:** | Property | Type | Purpose | | ----------- | ------------------------------------ | -------------------------------------------------------- | | `config` | `Source.Config` | Settings, mapping, options | | `env` | `Types['env']` | Environment (push, logger) | | `logger` | `Logger` | Logging functions | | `id` | `string` | Source identifier | | `collector` | `Collector.Instance` | Reference to collector | | `withScope` | `(raw, respond, body) => Promise` | Bind ingest + respond to a single scope (server sources) | ### Push Method[​](#push-method "Direct link to Push Method") | Method | Purpose | | ------------- | ----------------------------------- | | `push(input)` | Receive external input, emit events | ### Init Method[​](#init-method "Direct link to Init Method") `init?: () => void | Promise` — Optional eager-startup hook on the returned `Source.Instance`. The factory must be **side-effect-free**: build the instance and return it. The collector calls `init()` on every source eagerly after all factories register, regardless of `config.require`. Use `init` for work that previously sat in the factory body: adopting a pre-init window queue (e.g., `window.elbLayer`), attaching DOM listeners, opening sockets, intercepting `window.dataLayer`. After `init` runs the collector flips `Source.Config.init` to `true`. ### queueOn Buffer[​](#queueon-buffer "Direct link to queueOn Buffer") `queueOn?: Array<{ type: On.Types; data: unknown }>` — Optional buffer on the `Source.Instance` for lifecycle events delivered before the source is **started** (started ≡ `config.init === true && !config.require?.length`). The collector pushes `{ type, data }` here when it would otherwise call `source.on(type, data)`. Once the source becomes started, the collector replays each entry via `source.on(...)` and clears the queue. ### Destroy Method[​](#destroy-method "Direct link to Destroy Method") `destroy?: DestroyFn` — Optional cleanup method. Called during `command('shutdown')`. Use to close HTTP servers, timers, or connections. Receives `{ id, config, env, logger }`. ## Source Lifecycle[​](#source-lifecycle "Direct link to Source Lifecycle") The collector and every source agree on three lifecycle markers, all on `Source.Config`: | Field | Purpose | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `init?: boolean` | Set by the collector to `true` after `Instance.init()` resolves. Authors do not write to it. Reflects "init has run", not "is started". | | `require?: string[]` | Author-supplied **timing hint**. Lists collector state a source needs before its first `on()` delivery. Satisfied by the collector's **current recorded state** (level), not only by a future event — so order does not matter: the gate clears whether the required state was recorded before or after this source registered. It is not a correctness dependency: a source reacts to state correctly whether or not it declares `require`. | | `disabled?: boolean` | Hard skip — no factory invocation, no init, no event capture. | The flow is: 1. **Register** — collector invokes the factory. Factory returns a fresh `Source.Instance` with no side effects. 2. **Init pass** — collector calls `Instance.init()` on every registered source, then sets `config.init = true`. 3. **Lifecycle delivery** — for each collector event (`consent`, `user`, `session`, `run`, …) the collector decrements every source's `require` in place AND reconciles every still-pending source/destination against current state, so a gate also clears from state that was already recorded. If a source is started, it calls `source.on(type, data)` directly. Otherwise it pushes `{ type, data }` into `Instance.queueOn`. 4. **Replay** — when a source becomes started (require empties), the collector replays its `queueOn` via `source.on(...)` and clears the queue. `require` therefore gates the **timing** of `on()` delivery, not code execution and not correctness. `Instance.init()` always runs eagerly. There is no `collector.pending.sources` map: per-source state lives entirely on `Source.Instance` and `Source.Config`. ### Exactly-once state delivery is a collector guarantee[​](#exactly-once-state-delivery-is-a-collector-guarantee "Direct link to Exactly-once state delivery is a collector guarantee") State commands (`consent`, `user`, `globals`, `custom`) are recorded by the collector immediately, even before `run`, and delivered to each source's `on` handler exactly once per change. The collector tracks what each subscriber has already received, so re-running or re-registering never double-fires a state reaction. Sources should **not** hand-roll their own deduplication for state deliveries: the collector enforces exactly-once, and delivery is order-independent (it does not depend on source init order or on whether the state arrived before or after `run`). This mirrors the destination model: `Destination.Instance.init` handles one-time bootstrap, `Destination.Config.init` is the collector-managed "init has run" flag, and `Destination.Config.require` gates event delivery the same way. See [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md). ## Push Signatures by Type[​](#push-signatures-by-type "Direct link to Push Signatures by Type") | Source Type | Signature | Example | | -------------- | ----------------------------------- | ------------ | | Cloud Function | `push(req, res) → Promise` | HTTP handler | | Browser | `push(event, data) → Promise` | DOM events | | DataLayer | `push(event, data) → Promise` | GTM-style | **Key insight:** Source `push` IS the handler. No wrappers needed. ``` // Direct deployment http('handler', source.push); ``` When a test or integration code needs to invoke a source's `push` through the collector bag, `collector.sources` erases the per-source generic on read. Use `Source.getSource(collector, id)` to recover the narrow signature without a cast. See the testing-strategy skill for the full pattern (symmetric helpers exist for destinations, transformers, stores). ## Source Paths[​](#source-paths "Direct link to Source Paths") | Type | Path | Examples | | ------ | -------------------------- | ------------------ | | Web | `packages/web/sources/` | browser, dataLayer | | Server | `packages/server/sources/` | gcp | ## Browser Source[​](#browser-source "Direct link to Browser Source") The browser source captures events from DOM using data attributes. ``` ``` See [packages/web/sources/browser/](https://github.com/elbwalker/walkerOS/blob/main/packages/web/sources/browser) for implementation. ## DataLayer Source[​](#datalayer-source "Direct link to DataLayer Source") Captures events from a GTM-style dataLayer array. ``` window.dataLayer.push({ event: 'product view', product: { id: 'P123', name: 'Laptop' }, }); ``` See [packages/web/sources/dataLayer/](https://github.com/elbwalker/walkerOS/blob/main/packages/web/sources/dataLayer) for implementation. ## Server Sources[​](#server-sources "Direct link to Server Sources") Handle HTTP requests in cloud functions. Server sources use the context pattern: ``` import type { Source } from '@walkeros/core'; export const sourceCloudFunction: Source.Init = async (context) => { const { config = {}, env } = context; const { push: envPush } = env; // Apply defaults inline — flow.json is developer-controlled, so no // runtime validation. Shape checks live in ./schemas and are used by // `walkeros validate` and dev tooling, never at runtime. const userSettings = config.settings || {}; const settings = { ...userSettings, // example default: port: userSettings.port ?? 3000, }; const push = async (req: Request, res: Response): Promise => { // Transform HTTP request → walkerOS event const event = transformRequest(req); await envPush(event); res.json({ success: true }); }; return { type: 'cloudfunction', config: { ...config, settings }, push }; }; // Direct deployment export const handler = source.push; ``` See [packages/server/sources/gcp/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/sources/gcp) for implementation. ## Env Pattern (Dependency Injection)[​](#env-pattern-dependency-injection "Direct link to Env Pattern (Dependency Injection)") Platform dependencies go through `env` with fallback to globals or direct imports. This enables testing and simulation without touching globals. ``` // Express source: env.express ?? express (import fallback) const expressLib = env.express ?? express; const app = expressLib(); // Web sources: env.window ?? window (global fallback) const win = env.window ?? window; const doc = env.document ?? document; ``` Every source's `Env` interface extends `Source.BaseEnv` with optional platform deps: ``` export interface Env extends Source.BaseEnv { window?: Window & typeof globalThis; // web sources document?: Document; // web sources express?: typeof express; // express source cors?: typeof cors; // express source } ``` Tests inject mocks via `env` instead of mocking globals. See [testing-strategy](/skills/walkeros-testing-strategy.md). ## Transformer Wiring[​](#transformer-wiring "Direct link to Transformer Wiring") Sources can wire to pre-collector transformer chains via the `next` property: ``` sources: { browser: { code: sourceBrowser, next: 'validate' // First transformer to run after this source } } ``` The transformer chain runs before events reach the collector. See [understanding-transformers](/skills/walkeros-understanding-transformers.md) for chain details. ## Per-Scope Context (server sources)[​](#per-scope-context-server-sources "Direct link to Per-Scope Context (server sources)") A single source factory instance handles many concurrent invocations: Express processes overlapping requests, Lambda reuses one handler across calls, queue consumers loop over messages. Each logical unit of work is a **scope**. Server sources MUST wrap each invocation with `context.withScope(rawScope, respond, body)`: ``` const push = async (req, res) => { const respond = createRespond((options) => { /* wire options into res */ }); await context.withScope(req, respond, async (env) => { await env.push(parsedData); }); }; ``` Inside `body`, `env.push` carries that scope's `ingest` (extracted from `rawScope` via `config.ingest` mapping) and `respond` end to end through the pipeline. Concurrent scopes never share ingest or respond. **Browser sources skip `withScope`.** A browser tab is a single logical scope for its lifetime; calling `env.push` directly is correct. ## Response Delegation (env.respond)[​](#response-delegation-envrespond "Direct link to Response Delegation (env.respond)") When a server source passes a `respond` to `withScope`, every transformer and destination in the pipeline can call `env.respond?.({ body, status?, headers? })` to customize the HTTP response. First call wins (`createRespond` is idempotent), so the source's default response is a no-op if a step already responded. See `@walkeros/server-source-express` for the reference implementation. ## createTrigger Pattern[​](#createtrigger-pattern "Direct link to createTrigger Pattern") Every source exports a `createTrigger` factory from its examples (`dev` entry) that follows the unified `Trigger.CreateFn` interface: ``` type CreateFn = ( config: Collector.InitConfig, options?: unknown, ) => Promise>; ``` `createTrigger` simulates real-world invocations from the outside — full blackbox, no source instance access. Each package implements it differently: | Source | Content | Trigger type | Mechanism | | ------- | -------------- | ------------------ | ----------------------- | | Browser | HTML string | `click`, `load`... | DOM injection + events | | Express | HTTP req shape | `POST`, `GET` | Real `fetch()` requests | The trigger lazily calls `startFlow(config)` on first invocation. Tests capture events via spy destinations. See [using-step-examples](/skills/walkeros-using-step-examples.md) for testing patterns. ## Setup (optional)[​](#setup-optional "Direct link to Setup (optional)") Sources can implement an optional `setup()` lifecycle to provision external resources, for example registering a webhook with a third-party provider, creating a Pub/Sub subscription, or pre-allocating queue resources. Setup is **never** invoked by the runtime, push, init, or deploy. It runs only when an operator explicitly types `walkeros setup source.`. The signature is `(ctx: LifecycleContext, Env>) => Promise`, where `LifecycleContext` carries `{ id, config, env, logger }`. Idempotency is the package's responsibility: the framework adds no opinion. Use `resolveSetup(ctx.config.setup, DEFAULTS)` from `@walkeros/core` to normalize the `boolean | object` shape into a concrete options object. See [walkeros-create-source](/skills/walkeros-create-source.md), [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md), [walkeros-understanding-stores](/skills/walkeros-understanding-stores.md), and the `walkeros setup` CLI documentation for the authoring template and operator workflow. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - How sources fit in architecture * [walkeros-understanding-events](/skills/walkeros-understanding-events.md) - Events that sources emit * [walkeros-create-source](/skills/walkeros-create-source.md) - Create new source **Source Files:** * [packages/core/src/types/source.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/source.ts) - Interface **Package READMEs:** * [packages/web/sources/browser/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/sources/browser/README.md) - Browser source * [packages/web/sources/dataLayer/README.md](https://github.com/elbwalker/walkerOS/blob/main/packages/web/sources/dataLayer/README.md) - DataLayer source **Documentation:** * [Website: Sources](/docs/sources/.md) - Overview * [Website: Browser Source](/docs/sources/web/browser/.md) - Browser docs * [Website: Create Your Own](/docs/sources/create-your-own.md) - Guide --- # Understanding walkerOS Stores ## Overview[​](#overview "Direct link to Overview") Stores provide key-value storage that other components (sources, transformers, destinations) consume via environment injection. They are the 4th component type in Flow\.Json alongside sources, transformers, and destinations. **Core principle:** Stores are passive infrastructure. They don't process events or participate in chains — they provide state that other components read and write. ## Store interface[​](#store-interface "Direct link to Store interface") See [packages/core/src/types/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/store.ts) for the canonical interface. ### Instance[​](#instance "Direct link to Instance") | Property | Type | Purpose | Required | | --------- | ---------------------------------- | ---------------------------- | ------------ | | `type` | `string` | Store type identifier | **Required** | | `config` | `Store.Config` | Settings and env | **Required** | | `get` | `(key) => StoreValue \| undefined` | Read a value | **Required** | | `set` | `(key, value, ttl?) => void` | Write a value (optional TTL) | **Required** | | `delete` | `(key) => void` | Remove a value | **Required** | | `destroy` | `DestroyFn` | Cleanup on shutdown | Optional | All methods can be sync or async (return `Promise`). ### Value type and `file` mode[​](#value-type-and-file-mode "Direct link to value-type-and-file-mode") Stores hold one canonical value type: structured data (`StoreValue`), with binary (`Uint8Array`) as a first-class leaf. `StoreValue` is `string | number | boolean | null | Uint8Array | StoreValue[] | { [key: string]: StoreValue }` (`undefined` is reserved as the "miss" sentinel and is never a stored value). A shared core codec (`serializeStoreValue` / `deserializeStoreValue`) round-trips that value to and from each backing. `Store.Config.file?: boolean` (default `false`) picks the mode, decided once at init: * **Structured (default):** values are `StoreValue` data, serialized by the shared codec. * **File (`file: true`):** a byte-native backend (fs, S3, GCS) persists raw bytes byte-exact. `set()` accepts a `Uint8Array` or `string` and stores it untouched; `get()` hands the exact bytes back. Use for serving assets such as walker.js. The Sheets store is structured-only and rejects `file: true` at init. TTL is owned by the cache layer, not the store. The store persists values; a `cache` wrapper manages expiry. `flow_validate` warns when a store sets both `file: true` and `cache`, and when a `@walkeros/server-transformer-file` is wired to a byte-native store that does not set `file: true`. ### Init function (context pattern)[​](#init-function-context-pattern "Direct link to Init function (context pattern)") Stores use the same context-based init pattern as other components: ``` import type { Store } from '@walkeros/core'; export const storeMyStore: Store.Init = (context) => { const { config, env, logger, id } = context; const settings = config.settings || {}; return { type: 'my-store', config: context.config as Store.Config, get(key) { /* ... */ }, set(key, value, ttl) { /* ... */ }, delete(key) { /* ... */ }, destroy() { /* cleanup */ }, }; }; ``` **Context contains:** | Property | Type | Purpose | | ----------- | -------------------- | ------------------------- | | `config` | `Store.Config` | Settings from flow config | | `env` | `Store.Env` | Environment dependencies | | `logger` | `Logger.Instance` | Scoped logger | | `id` | `string` | Store identifier | | `collector` | `Collector.Instance` | Reference to collector | ## Lifecycle[​](#lifecycle "Direct link to Lifecycle") Stores have the simplest lifecycle of all component types: ``` Startup: Stores → Destinations → Transformers → Sources Shutdown: Sources → Destinations → Transformers → Stores ``` * **Init first:** Stores init before all other components so they're available when sources/transformers/destinations start * **Destroy last:** Stores destroy after all other components so transformers can flush during their own destroy * **No lazy init:** Unlike destinations, stores don't support `require` or deferred activation — they are always eager ## Wiring stores via `$store.`[​](#wiring-stores-via-store "Direct link to wiring-stores-via-store") ### Bundled mode (flow\.json)[​](#bundled-mode-flowjson "Direct link to Bundled mode (flow.json)") Use `$store.storeId` in a component's `env` to inject a store instance: ``` { "stores": { "data": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./data" } }, "cache": { "rules": [{ "ttl": 60 }] } } }, "transformers": { "fingerprint": { "package": "@walkeros/server-transformer-fingerprint", "env": { "store": "$store.data" } } } } ``` The bundler resolves `$store.data` to a runtime reference. Invalid references are caught at build time. `walkeros validate` also catches typos at validation time, including unknown store names and the colon-instead-of-dot mistake (e.g. `$store:data` is flagged with the suggested form `$store.data`). The `cache` field enables the built-in in-memory cache tier (`__cache`) on top of any backing store. No separate memory store declaration is needed. ### Integrated mode (TypeScript)[​](#integrated-mode-typescript "Direct link to Integrated mode (TypeScript)") Pass store instances directly — no `$store.` prefix needed: ``` import { startFlow } from '@walkeros/collector'; import { storeFsInit } from '@walkeros/server-store-fs'; import { transformerFingerprint } from '@walkeros/server-transformer-fingerprint'; const { collector } = await startFlow({ stores: { data: { code: storeFsInit, config: { settings: { basePath: './data' } }, cache: { rules: [{ ttl: 60 }] }, }, }, transformers: { fingerprint: { code: transformerFingerprint, env: { store: collector.stores.data }, // Direct reference }, }, }); ``` Note: In integrated mode, you wire the store instance directly in `env` rather than using the `$store.` string prefix (that's a bundler feature). ## Available stores[​](#available-stores "Direct link to Available stores") ### Built-in cache tier (`__cache`)[​](#built-in-cache-tier-__cache "Direct link to built-in-cache-tier-__cache") The collector ships a built-in in-memory cache with LRU eviction, TTL, and entry/byte caps. Enable it on any store by setting `Flow.Store.cache`. No separate package import needed: ``` { "stores": { "files": { "package": "@walkeros/server-store-fs", "cache": { "rules": [{ "ttl": 60 }] } } } } ``` Use this for the common "cache in front of a slow backing store" pattern (API, GCS, Sheets, etc.). ### `@walkeros/server-store-fs` (filesystem)[​](#walkerosserver-store-fs-filesystem "Direct link to walkerosserver-store-fs-filesystem") File-based store for serving static assets. Server-only. ``` import { storeFsInit } from '@walkeros/server-store-fs'; ``` ### `@walkeros/server-store-s3` (S3-compatible object storage)[​](#walkerosserver-store-s3-s3-compatible-object-storage "Direct link to walkerosserver-store-s3-s3-compatible-object-storage") S3-compatible store using `s3mini` (\~20 KB, zero dependencies). Works with AWS S3, Cloudflare R2, Scaleway, DigitalOcean Spaces, Backblaze B2, MinIO, and any S3-compatible provider. Structured by default (stored as `application/json`); set `file: true` to serve raw bytes byte-exact with the real mime. Server-only. ``` import { storeS3Init } from '@walkeros/server-store-s3'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | ----------------- | -------- | -------- | -------- | -------------------------- | | `bucket` | `string` | Yes | — | S3 bucket name | | `endpoint` | `string` | Yes | — | S3-compatible endpoint URL | | `accessKeyId` | `string` | Yes | — | S3 access key ID | | `secretAccessKey` | `string` | Yes | — | S3 secret access key | | `region` | `string` | No | `"auto"` | AWS region (SigV4 signing) | | `prefix` | `string` | No | — | Key prefix for scoping | **Primary use case:** Serving static files in managed deployments (Mode D) where files live in a bucket rather than being baked into a Docker image. ### `@walkeros/server-store-gcs` (Google Cloud Storage)[​](#walkerosserver-store-gcs-google-cloud-storage "Direct link to walkerosserver-store-gcs-google-cloud-storage") Zero-dependency GCS store using raw `fetch` + GCS JSON API. Built-in auth: ADC on Cloud Run / GKE, or explicit service account JWT. Server-only. ``` import { storeGcsInit } from '@walkeros/server-store-gcs'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | -------- | -------- | -------- | ------- | ---------------------- | | `bucket` | `string` | Yes | — | GCS bucket name | | `prefix` | `string` | No | — | Key prefix for scoping | Credentials live at `config.credentials` (sibling of `settings`): `string | object` SA JSON for non-GCP envs, `$env`-resolvable; omit for ADC. The deprecated `settings.credentials` still works. **Primary use case:** Serving static files on GCP infrastructure (Cloud Run, GKE) where ADC provides seamless authentication. ### `@walkeros/server-store-sheets` (Google Sheets)[​](#walkerosserver-store-sheets-google-sheets "Direct link to walkerosserver-store-sheets-google-sheets") Zero-dependency Google Sheets store using raw `fetch` + Sheets v4 REST API. One row per key, one cell per value (JSON-serialized). Built-in auth shared with the GCS store. Server-only. Structured-only: cells hold structured JSON, so it rejects `file: true` at init and rejects values carrying a binary (`Uint8Array`) leaf. Use fs, S3, or GCS for byte-exact serving. ``` import { storeSheetsInit } from '@walkeros/server-store-sheets'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | ------------ | -------- | -------- | ---------- | -------------------------------- | | `id` | `string` | Yes | — | Spreadsheet ID (segment in URL) | | `sheet` | `string` | No | `'Sheet1'` | Sheet (tab) name | | `key` | `string` | No | `'A'` | Column letter for keys | | `value` | `string` | No | `'B'` | Column letter for JSON values | | `headerRows` | `number` | No | `1` | Header rows to skip when reading | Credentials live at `config.credentials` (sibling of `settings`): `string | object` SA JSON for non-GCP envs, `$env`-resolvable; omit for ADC. The deprecated `settings.credentials` still works. **Primary use case:** Demos and small prototypes where the spreadsheet is the operator-facing UI for tweaking lookup data. Quota: 60 reads/min and 60 writes/min per project. Enable the built-in cache via `Flow.Store.cache` on the store declaration to absorb the quota, otherwise quota burns in seconds. **Not a production CRM substitute.** See [Website: Sheets Store](/docs/stores/server/sheets.md) for the cache-wiring example. ## Stores in flow config[​](#stores-in-flow-config "Direct link to Stores in flow config") Declare any store consumed by one or more components in the `stores` section of the flow config and wire it via `$store.` in component `env`. The built-in cache tier (`Flow.Store.cache`) covers the "fast in-memory cache in front of a slow backing store" pattern without a separate memory store. ## Accessing stores at runtime[​](#accessing-stores-at-runtime "Direct link to Accessing stores at runtime") After `startFlow()`, stores are available on the collector instance: ``` const { collector } = await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './data' } } }, }, }); // Read/write await collector.stores.files.set('key', 'value', 60000); // 60s TTL const value = await collector.stores.files.get('key'); await collector.stores.files.delete('key'); ``` ## Hooks[​](#hooks "Direct link to Hooks") Store operations (`get`, `set`, `delete`) are wrapped with `useHooks` during initialization, enabling pre/post interception via the collector's hooks system. **Available hook names:** | Hook name | Pre hook | Post hook | | ------------- | ---------------- | ----------------- | | `StoreGet` | `preStoreGet` | `postStoreGet` | | `StoreSet` | `preStoreSet` | `postStoreSet` | | `StoreDelete` | `preStoreDelete` | `postStoreDelete` | Hooks fire on every store operation regardless of which component triggered it (cache system, transformer via env, destination via env, direct access on `collector.stores`). ``` const { collector } = await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './data' } } }, }, }); // Intercept all store reads collector.hooks.preStoreGet = ({ fn }, key) => { console.log('Reading key:', key); return fn(key); }; ``` ## Key differences from other components[​](#key-differences-from-other-components "Direct link to Key differences from other components") | Aspect | Sources/Transformers/Destinations | Stores | | ------------- | --------------------------------- | ------------------------ | | Event flow | Participate in push chain | No push, no chain | | `next/before` | Chain connection fields | None (passive) | | Lifecycle | Init after stores | Init first, destroy last | | `require` | Deferred activation supported | Always eager | | Interface | `push(event, context)` | `get/set/delete(key)` | ## Dev exports (`dev.ts`)[​](#dev-exports-devts "Direct link to dev-exports-devts") Every store package must export schemas and examples via `dev.ts`, matching the convention used by sources and destinations: ### Required files[​](#required-files "Direct link to Required files") ``` src/ ├── schemas/ │ ├── settings.ts # Zod schema for store settings │ └── index.ts # Re-export + zodToSchema() conversion ├── examples/ │ └── index.ts # Example Store.Config objects └── dev.ts # export * as schemas; export * as examples ``` ### Schema pattern[​](#schema-pattern "Direct link to Schema pattern") ``` // src/schemas/settings.ts import { z } from '@walkeros/core/dev'; export const SettingsSchema = z.object({ mySetting: z.string().describe('Setting description for docs and MCP'), }); export type Settings = z.infer; // src/schemas/index.ts import { zodToSchema } from '@walkeros/core/dev'; import { SettingsSchema } from './settings'; export { SettingsSchema, type Settings } from './settings'; export const settings = zodToSchema(SettingsSchema); ``` ### tsup + package.json[​](#tsup--packagejson "Direct link to tsup + package.json") ``` // tsup.config.ts import { defineConfig, buildModules, buildDev } from '@walkeros/config/tsup'; export default defineConfig([buildModules(), buildDev()]); ``` Add `./dev` export to package.json: ``` "./dev": { "types": "./dist/dev.d.ts", "import": "./dist/dev.mjs", "require": "./dist/dev.js" } ``` ### Why this matters[​](#why-this-matters "Direct link to Why this matters") * `buildDev()` generates `dist/walkerOS.json` at build time * MCP tools fetch `walkerOS.json` from CDN for package discovery and schema validation * Website docs use `` instead of hardcoded markdown tables * Without `dev.ts`, a store is invisible to MCP and docs tables rot ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") Stores can optionally export hints — lightweight, actionable context for AI agents beyond what schemas and examples already provide. Create `src/hints.ts`: ``` import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'persistence-behavior': { text: 'Describes persistence guarantees. See settings schema for options.', }, }; ``` Export from `src/dev.ts` alongside schemas and examples: ``` export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Most stores don't need hints — only add them for non-obvious behaviors, prerequisites, or troubleshooting patterns. ## Setup (optional)[​](#setup-optional "Direct link to Setup (optional)") Stores can implement an optional `setup()` lifecycle to provision external resources, for example creating a SQLite table, initializing an S3 bucket, or running a one-off schema migration. Setup is **never** invoked by the runtime, push, init, or deploy. It runs only when an operator explicitly types `walkeros setup store.`. The signature is `(ctx: LifecycleContext, Env>) => Promise`, where `LifecycleContext` carries `{ id, config, env, logger }`. Idempotency is the package's responsibility: the framework adds no opinion. Use `resolveSetup(ctx.config.setup, DEFAULTS)` from `@walkeros/core` to normalize the `boolean | object` shape into a concrete options object. See [walkeros-create-destination](/skills/walkeros-create-destination.md), [walkeros-create-source](/skills/walkeros-create-source.md), [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md), [walkeros-understanding-sources](/skills/walkeros-understanding-sources.md), and the `walkeros setup` CLI documentation for the authoring template and operator workflow. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Architecture and Flow structure * [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer interface and env pattern * [walkeros-using-store-cache](/skills/walkeros-using-store-cache.md) - Store-level cache and the declarative `state` block for fetch/stash without `$code:` * [walkeros-using-cli](/skills/walkeros-using-cli.md) - Bundling flows with stores **Source files:** * [packages/core/src/types/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/store.ts) - Store types * [packages/collector/src/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/store.ts) - initStores implementation * [packages/server/stores/fs/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/fs/src) - Filesystem store package * [packages/server/stores/s3/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/s3/src) - S3 store package * [packages/server/stores/gcs/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/gcs/src) - GCS store package * [packages/server/stores/sheets/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/sheets/src) - Google Sheets store package **Documentation:** * [Website: Flow](/docs/getting-started/modes/bundled.md) - Flow configuration with stores section * [Website: Stores](/docs/stores/.md) - Stores overview * [Website: S3 Store](/docs/stores/server/s3.md) - S3 store documentation * [Website: FS Store](/docs/stores/server/fs.md) - Filesystem store documentation * [Website: GCS Store](/docs/stores/server/gcs.md) - GCS store documentation * [Website: Sheets Store](/docs/stores/server/sheets.md) - Google Sheets store documentation --- # Understanding walkerOS Transformers ## Overview[​](#overview "Direct link to Overview") Transformers are middleware for **validating**, **enriching**, and **redacting** events in the walkerOS pipeline. They run in chains at configurable points between sources, collector, and destinations. **Core principle:** Transformers transform events. They don't capture (sources) or deliver (destinations)—they modify events in-flight. ## Use Cases[​](#use-cases "Direct link to Use Cases") | Use Case | Purpose | Example | | ------------ | ----------------------------------------- | ------------------------ | | **Validate** | Ensure events match schema contracts | JSON Schema validation | | **Enrich** | Add server-side data to events | User segments, geo data | | **Redact** | Remove sensitive data before destinations | Strip PII, anonymize IPs | ## Available Packages[​](#available-packages "Direct link to Available Packages") | Package | Env | Purpose | | ------------------------------------------ | ------ | -------------------------------------------------- | | `@walkeros/transformer-validate` | both | Enforce JSON Schema contracts on events at runtime | | `@walkeros/transformer-ga4` | server | Decode GA4 Measurement Protocol hits into events | | `@walkeros/server-transformer-bot` | server | Annotate bot / AI-agent scores | | `@walkeros/server-transformer-fingerprint` | server | Derive a stable visitor fingerprint | ### Contract validation[​](#contract-validation "Direct link to Contract validation") `@walkeros/transformer-validate` is the runtime arm of a [contract](/docs/getting-started/flow/contract.md). Event shapes live in the top-level `contract` block; the transformer references one via `$contract.` in its `contract` setting and validates the canonical event. `mode: "strict"` drops invalid events (chain-stop); `mode: "pass"` (default) annotates `event.source.valid` and continues so a downstream step can route on the verdict. `format: true` additionally checks the canonical `WalkerOS.Event` structure. Filtering is the same mechanism: an inline schema that rejects the unwanted events plus `mode: "strict"`, there is no separate `ignore` field. See [Website: Validate transformer](/docs/transformers/validate.md). ## Transformer Interface[​](#transformer-interface "Direct link to Transformer Interface") See [packages/core/src/types/transformer.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/transformer.ts) for canonical interface. ### Init Function (Context Pattern)[​](#init-function-context-pattern "Direct link to Init Function (Context Pattern)") Transformers use a context-based initialization pattern: ``` import type { Transformer } from '@walkeros/core'; export const transformerMyTransformer: Transformer.Init = (context) => { const { config = {}, env, logger, id } = context; // Apply defaults inline — flow.json is developer-controlled, so no // runtime validation. Shape checks live in ./schemas and are used by // `walkeros validate` and dev tooling, never at runtime. const userSettings = config.settings || {}; const settings = { ...userSettings, // example default: threshold: userSettings.threshold ?? 100, }; return { push(event, pushContext) { // Process event return { event }; }, }; }; ``` **Init Context contains:** | Property | Type | Purpose | | ----------- | -------------------- | --------------------------------------- | | `config` | `Transformer.Config` | Settings, mapping, next chain | | `env` | `Types['env']` | Environment deps (stores via `$store.`) | | `logger` | `Logger` | Logging functions | | `id` | `string` | Transformer identifier | | `collector` | `Collector.Instance` | Reference to collector | | `ingest` | `Ingest` (optional) | Request metadata from source | ### Instance Methods[​](#instance-methods "Direct link to Instance Methods") | Method | Purpose | Required | | --------- | ------------------------------------ | ------------ | | `push` | Process event, return modified/false | **Required** | | `init` | One-time initialization | Optional | | `destroy` | Cleanup resources | Optional | ## Return Values[​](#return-values "Direct link to Return Values") The `push` function controls event flow: | Return | Behavior | | -------------------- | --------------------------------------------------- | | `{ event }` | Continue chain with modified event | | `void` | Continue chain, event unchanged | | `false` | Stop chain, event dropped | | `{ event, next }` | Redirect chain to a different transformer (fan-out) | | `{ event, respond }` | Continue chain with wrapped respond function | ``` push(event, context) { if (!event.data?.id) { context.logger.error('Missing required id'); return false; // Stop chain } event.data.enrichedAt = Date.now(); return { event }; // Continue with modified event } ``` ## Inline Code Transformers[​](#inline-code-transformers "Direct link to Inline Code Transformers") For simple transformations without external packages, use inline code with the `$code:` string prefix in JSON configs. The `$code:` prefix tells the CLI bundler to parse the following string as executable JavaScript: ``` { "transformers": { "enrich": { "code": { "push": "$code:(event) => { event.data.enrichedAt = Date.now(); return { event }; }" }, "next": "validate" } } } ``` **Inline code structure:** | Property | Purpose | | ----------- | ----------------------------------- | | `code.init` | Code run once during initialization | | `code.push` | Code run for each event | **Push code has access to:** * `event` - The event being processed * `context` - Push context with logger, config, etc. **Return values in push code:** * Return `{ event }` to continue chain with modified event * Return `undefined` to pass event unchanged * Return `false` to drop event from chain **Example: Filtering internal events** ``` { "transformers": { "filter": { "code": { "push": "$code:(event) => { if (event.name.startsWith('internal_')) return false; return { event }; }" } } } } ``` **Mixing inline and package transformers:** ``` { "transformers": { "addTimestamp": { "code": { "push": "$code:(event) => { event.data.processedAt = new Date().toISOString(); return { event }; }" }, "next": "enrich" }, "enrich": { "package": "@walkeros/transformer-enricher" } } } ``` ## Pipeline Integration[​](#pipeline-integration "Direct link to Pipeline Integration") Transformers run at two points in the pipeline: ``` Source → [Pre-Transformers] → Collector → [Post-Transformers] → Destination (source.next) (destination.before) ``` ### Pre-Collector Chain[​](#pre-collector-chain "Direct link to Pre-Collector Chain") Runs after source captures event, before collector enrichment: ``` sources: { browser: { code: sourceBrowser, next: 'validate' // First transformer in pre-chain } } ``` ### Post-Collector Chain[​](#post-collector-chain "Direct link to Post-Collector Chain") Runs after collector enrichment, before destination receives event: ``` destinations: { gtag: { code: destinationGtag, before: 'redact' // First transformer in post-chain } } ``` ### Chain Linking[​](#chain-linking "Direct link to Chain Linking") Transformers link together via `next`: ``` transformers: { fingerprint: { code: transformerFingerprint, config: { next: 'enrich' } // Chain to next transformer }, enrich: { code: transformerEnrich, config: { next: 'redact' } }, redact: { code: transformerRedact // No next = end of chain } } ``` ### Branching and fan-out[​](#branching-and-fan-out "Direct link to Branching and fan-out") Transformers can redirect events to different chains using the `branch()` factory from `@walkeros/core`: ``` import { branch } from '@walkeros/core'; push(event, context) { return branch(event, 'parser'); // Single target return branch(event, ['a', 'b']); // Fan-out to multiple } ``` Conditional routing is built into `next`/`before` properties using the `one` operator, no separate router transformer needed: ``` "next": { "one": [ { "match": { "key": "ingest.path", "operator": "prefix", "value": "/api" }, "next": "api-handler" }, { "next": "default" } ] } ``` `one` entries are evaluated in order, first match wins. A `RouteConfig` is a disjoint union: each config sets at most one of `next` (gated link), `one` (first-match dispatch), or `many` (all-match dispatch), never more than one. An entry with no `match` always matches (use it as a fallback). If no entry matches, the event passes through unchanged. Use `many` (pre-collector only) when every matching branch should run in parallel, terminating the main chain: ``` "next": { "many": [ { "match": { "key": "event.consent.analytics", "operator": "eq", "value": "granted" }, "next": "ga4-pipeline" }, { "next": "audit-log" } ] } ``` ### Paths and pass-through steps[​](#paths-and-pass-through-steps "Direct link to Paths and pass-through steps") walkerOS uses two vocabulary terms for chain composition: * **Path:** the multi-step chain that an event walks through a flow's `transformers` section. * **Pass-through step** (short: **pass**): a single step inside a path that declares no `code` and no `package`. The runtime synthesizes the push, so the step contributes structure without shipping executable code. A pass-through step ships in three variants. Each variant uses a different operative field; combine them on the same step when it helps. #### Variant 1: chain-only (before / next)[​](#variant-1-chain-only-before--next "Direct link to Variant 1: chain-only (before / next)") A named hop that shares a chain across multiple call sites. Use it to avoid duplicating arrays in `before` / `next` references: ``` { "transformers": { "validateThenEnrich": { "before": ["validate", "enrich"] } }, "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "before": "validateThenEnrich" }, "meta": { "package": "@walkeros/web-destination-meta", "before": "validateThenEnrich" } } } ``` #### Variant 2: cache-only[​](#variant-2-cache-only "Direct link to Variant 2: cache-only") A step that declares only a `cache` block. Useful for deduplication or short-circuit halts. `cache.stop: true` at a pre-collector position halts the pipeline (not just the local chain): ``` { "transformers": { "dedup": { "cache": { "stop": true, "rules": [{ "key": ["event.id"], "ttl": 60 }] } } } } ``` #### Variant 3: mapping-only[​](#variant-3-mapping-only "Direct link to Variant 3: mapping-only") A step that declares only a `mapping: Mapping.Config`. The runtime synthesizes a push that calls `processEventMapping` and mutates the event in-flight: ``` { "transformers": { "redactPII": { "mapping": { "policy": { "user.email": { "value": "[redacted]" } } } } } } ``` See [walkeros-understanding-mapping](/skills/walkeros-understanding-mapping.md) for the mapping primitives (`policy`, `data`, `mapping[].name`, etc.) and the "Mapping at the transformer position" section for the dual semantic. ### Mapping at transformer position vs destination position[​](#mapping-at-transformer-position-vs-destination-position "Direct link to Mapping at transformer position vs destination position") `mapping` is the same field shape (`Mapping.Config`) in both positions, but the semantic is disambiguated by where the step sits: | Position | What `mapping` produces | | ----------- | --------------------------------------------------------- | | Destination | A vendor-shaped payload (the destination consumes `data`) | | Transformer | A mutated event that continues through the chain | At the transformer position, only event-mutating fields apply: `policy`, `mapping[].policy`, `mapping[].name`, `mapping[].ignore`, `mapping[].consent`, and `include`. Vendor-payload fields (`data`, `mapping[].data`, `silent`) are ignored with a one-time init warning. `mapping[].ignore: true` drops the event from the chain (not "skip this destination", which is the destination-position semantic). ### Closed schema (unknown keys are errors)[​](#closed-schema-unknown-keys-are-errors "Direct link to Closed schema (unknown keys are errors)") Transformer step entries follow a **closed schema**. Known keys only: `code`, `package`, `config`, `before`, `next`, `cache`, `mapping`. Unknown keys at the top of a step are validation errors. This catches misrouted keys (e.g. `{ rules: [], stop: true }` placed at the top of a step instead of nested under `cache:`) at validate time instead of letting them silently pass through at runtime. A step must declare at least one operative field. An empty `{}` is rejected with `EMPTY_TRANSFORMER`. Declaring both `code` and `package` on the same step is rejected with `CONFLICT`. ### Chain resolution safety[​](#chain-resolution-safety "Direct link to Chain resolution safety") `getNextSteps()` (the public dispatch helper, previously `walkChain`) uses a visited set to detect circular references. If a cycle is found, the loop is silently broken and the chain ends. If `next` points to a non-existent transformer, the chain also ends without error. Note: `getNextSteps` is deterministic for the supplied event context. Static analyzers without a real event can only enumerate reachability under "match may pass or fail" speculation. ### Composition principle[​](#composition-principle "Direct link to Composition principle") A transformer owns its own chain. When a chain references a transformer by name, that transformer's own `before` chain runs before its push, and its `next` chain after, both are walked recursively, with cycle detection. Cache halt signals (`cache.stop: true`) at pre-collector positions propagate pipeline-wide; destinations do not see the dropped event. The grammar's recursive `Route` shape (`string | Route[] | RouteConfig`) compiles element-by-element, so sequences can mix transformer IDs and inline `one` / `many` / `next` routes (`next: ["dedup", { one: [...] }]` is valid). This is the model to default to when adding new chain primitives. See [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) for the full connection rules between sources, transformers, and destinations. ## Push Context[​](#push-context "Direct link to Push Context") The `push` function receives a context with event metadata: | Property | Purpose | | ----------- | ---------------------------- | | `config` | Transformer configuration | | `env` | Environment dependencies | | `logger` | Scoped logger for output | | `id` | Transformer identifier | | `collector` | Access to collector instance | | `ingest` | Request metadata from source | ``` push(event, context) { const { logger, id, ingest } = context; logger.debug('Processing', { transformer: id, event: event.name }); // Access request metadata if available if (ingest?.ip) { event.data = { ...event.data, clientIp: ingest.ip }; } return { event }; } ``` ## Response Delegation (env.respond)[​](#response-delegation-envrespond "Direct link to Response Delegation (env.respond)") Transformers can customize HTTP responses by calling `context.env.respond?.({ body, status?, headers? })`. This is useful for validation transformers that reject events with custom error responses, or transformers that short-circuit the pipeline. First call wins (idempotent). The respond function is optional — only present when the source provides one. ## Transformer Paths[​](#transformer-paths "Direct link to Transformer Paths") | Path | Description | | ------------------------ | -------------------- | | `packages/transformers/` | Transformer packages | ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - How transformers fit in architecture * [walkeros-create-transformer](/skills/walkeros-create-transformer.md) - Create new transformer * [walkeros-using-store-cache](/skills/walkeros-using-store-cache.md) - Declarative `state` block for fetch/stash without `$code:` **Source Files:** * [packages/core/src/types/transformer.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/transformer.ts) - Interface **Documentation:** * [Website: Transformers](/docs/transformers/.md) - Overview * [Website: Create Your Own](/docs/transformers/create-your-own.md) - Guide --- # Using the walkerOS CLI ## Overview[​](#overview "Direct link to Overview") The walkerOS CLI (`walkeros`) bundles, tests, and runs event collection flows. **Core workflow:** 1. **Configure** - Write Flow\.Json JSON config 2. **Bundle** - Generate optimized JS bundle 3. **Test** - Simulate events (mocked) or push (real) 4. **Deploy** - Run locally or deploy to production ## Quick Start[​](#quick-start "Direct link to Quick Start") ``` # Install npm install -g @walkeros/cli # Bundle a flow walkeros bundle flow.json # Test with simulated event walkeros push flow.json -e '{"entity":"page","action":"view"}' --simulate destination.demo # Push real event walkeros push flow.json -e '{"entity":"page","action":"view"}' ``` ## Commands Overview[​](#commands-overview "Direct link to Commands Overview") | Command | Purpose | Safe? | | ---------- | -------------------------------------------------------- | ----- | | `bundle` | Generate JS bundle from config | ✅ | | `push` | Execute with real API calls (or `--simulate` for mocked) | ⚠️ | | `run` | Local HTTP event collection | ✅ | | `deploy` | Deploy flows to cloud | ⚠️ | | `previews` | Manage preview bundles for testing on live sites | ⚠️ | | `validate` | Validate configs/events | ✅ | | `cache` | Manage caching | ✅ | For detailed command reference, see [commands-reference.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-using-cli/commands-reference.md). *** ## Common Workflows[​](#common-workflows "Direct link to Common Workflows") ### Development Workflow[​](#development-workflow "Direct link to Development Workflow") ``` 1. Write flow.json config 2. Bundle: walkeros bundle flow.json 3. Simulate: walkeros push flow.json -e event.json --simulate destination.demo 4. Fix issues, repeat 2-3 5. Push test: walkeros push flow.json -e event.json 6. Deploy: walkeros deploy start ``` ### Multi-Flow Development[​](#multi-flow-development "Direct link to Multi-Flow Development") ``` # Bundle specific flow walkeros bundle flow.json --flow myFlow # Bundle all flows walkeros bundle flow.json --all # Test specific flow walkeros push flow.json --flow myFlow -e event.json --simulate destination.demo ``` ### Local Development Server[​](#local-development-server "Direct link to Local Development Server") ``` # HTTP event collection server walkeros run flow.json --port 3000 ``` **Server port note:** The `--port` flag (or `PORT` env var) is forwarded at runtime to all source configs that have a `port` setting. You don't need to hardcode ports in the flow config — set `port: 8080` as a default and let the runtime override it. *** ## Flow\.Json Configuration[​](#flowjson-configuration "Direct link to Flow.Json Configuration") ### Minimal Config[​](#minimal-config "Direct link to Minimal Config") ``` { "version": 4, "flows": { "default": { "config": { "platform": "web", "bundle": { "packages": { "@walkeros/web-destination-gtag": {} } } }, "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "config": { "measurementId": "G-XXXXXX" } } } } } } ``` ### Config Structure[​](#config-structure "Direct link to Config Structure") ``` { "version": 4, "flows": { "": { "config": { "platform": "web" | "server", // Platform (required) "settings": {}, // Platform-specific settings (optional) "bundle": { "packages": {}, // NPM packages pacote will install "overrides": {}, // Transitive dep version pins (npm-style) "traceInclude": [] // Optional: nft escape hatch (paths/globs) } }, "sources": {}, // Event sources "destinations": {}, // Event destinations "transformers": {}, // Transformer chain (optional) "mappings": {}, // Event transformation rules "collector": {} // Collector configuration } } } ``` **You do NOT need `npm install` for step packages.** flow\.json's `config.bundle.packages` is the single source of truth. Pacote installs them transparently when you run `walkeros bundle`. Only `@walkeros/cli` belongs in your project's `package.json` (as a devDependency). **`config.bundle.overrides`** pins transitive dependency versions. Use it when a vendor SDK's declared peer/dep range conflicts with another required version in the same tree. Example: `{"@amplitude/analytics-types": "2.11.1"}` forces that exact version everywhere in the install graph. Direct `packages` specs always win over overrides; overrides only substitute transitive resolution. **Schema version stays at 4.** Build-time fields live under `flow..config.bundle.{packages, overrides, traceInclude}`. The `flow..config.bundle.external` sub-field is no longer supported in @walkeros/cli\@4.x. For detailed configuration options, see [flow-configuration.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-using-cli/flow-configuration.md). *** ## Testing with Step Examples[​](#testing-with-step-examples "Direct link to Testing with Step Examples") ### Simulate with `--step`[​](#simulate-with---step "Direct link to simulate-with---step") Target a specific step and provide input as `SourceInput` (`{ content, trigger? }`): ``` # Simulate a source step with trigger metadata walkeros push flow.json --simulate source.browser --event '{"content":"...","trigger":{"type":"click"}}' # Simulate a destination step with an event walkeros push flow.json --simulate destination.gtag -e '{"entity":"order","action":"complete","data":{"total":149.97}}' ``` Example output: ``` Step: destinations.gtag in: { name: "order complete", data: { id: "ORD-123", total: 149.97 } } out: ["event", "purchase", { transaction_id: "ORD-123", value: 149.97 }] Status: PASS ``` ### Same flow via MCP (`flow_simulate`)[​](#same-flow-via-mcp-flow_simulate "Direct link to same-flow-via-mcp-flow_simulate") From an AI assistant the equivalent tool is `flow_simulate`. A few specifics: * **`step` is required.** Pass the target as `"type.name"`, e.g. `"source.browser"`, `"collector.default"`, `"destination.gtag"`, or `"transformer.router"`. There is no all-steps mode. * **Four step types: `source`, `collector`, `transformer`, `destination`.** Source-step `event` shape is `{ content, trigger? }`, where `content` is the walkerOS event `{ name, data }` and `trigger` is optional `{ type?, options? }`. There is no `env` field. Destination and transformer steps take a plain walkerOS event `{ name, data, consent? }`. * **`collector` is the enrichment step.** It takes a post-`next` partial event plus an optional state snapshot `{ consent?, user?, globals?, timing? }`, applies the collector's `createEvent`, and returns the fully enriched event. * **`transformer` steps accept an optional `ingest`** (a raw ingest without `_meta`). Supply it to test a request decoder standalone, for example a GA4 decoder reading `ctx.ingest.url`: pass `ingest: { url: "..." }` with the event. * **Sources are simulatable as a step**, including the `@walkeros/source-demo` demo source. * **`configPath` accepts a cloud flow id** (`flow_...` / `cfg_...`), resolved the same way `flow_load` does, so you can simulate a saved flow without a file round-trip. Repeated simulations of the same configuration reuse a prebuilt bundle for faster runs; local file paths always rebuild. `flow_bundle` accepts a cloud flow id the same way. `flow_load` loads a flow from a local path, URL, inline JSON, or a cloud flow/config ID (`flow_...` / `cfg_...`). Configs returned by `flow_load` and `flow_manage` are round-trip safe: structural values (package names, platform, IDs) come back literally, so a returned config can be edited and sent back to `flow_manage({ action: "update" })` unchanged. When an MCP request fails, the `diagnostics` tool (read-only, no parameters, works logged out) reports the MCP and CLI versions, the resolved app URL, app `/api/health` reachability, the bundled OpenAPI contract version, and which source served the last package lookup. ### Validate flow config[​](#validate-flow-config "Direct link to Validate flow config") Validate schema, references, and cross-step example compatibility: ``` walkeros validate flow.json ``` All checks run automatically — schema validation, reference checking, and cross-step example compatibility. No flags needed for full validation. For full details on writing and testing with step examples, see [using-step-examples](/skills/walkeros-using-step-examples.md). *** ## $code: Prefix (Inline JavaScript)[​](#code-prefix-inline-javascript "Direct link to $code: Prefix (Inline JavaScript)") Embed JavaScript functions in JSON configs: ``` { "fn": "$code:(event) => event.data.price * 100", "condition": "$code:(event) => event.data?.value > 100" } ``` **Important:** The CLI bundler converts `$code:` strings to actual JavaScript functions during build. This is essential for mappings in JSON configs. For mapping patterns, see [understanding-mapping](/skills/walkeros-understanding-mapping.md). *** ## Quick Reference[​](#quick-reference "Direct link to Quick Reference") ### Bundle Command[​](#bundle-command "Direct link to Bundle Command") ``` walkeros bundle [options] Options: --flow Bundle specific flow (default: "default") --all Bundle all flows --stats Show bundle statistics --json JSON output --no-cache Skip build cache -v, --verbose Verbose output -s, --silent Silent mode ``` Output: * Web: `dist/walker.js` (single self-contained IIFE) * Server: `dist/{flow.mjs, package.json, node_modules/}` (always a directory; nft-traced) * Server archive: `flow.tar.gz` / `flow.tgz` (the server bundle directory packed into a single gzip file) Use `-o ./dist/walker.js` for web, `-o ./dist/` for a server directory, or `-o ./flow.tar.gz` for a server archive. Web single-file bundles do not support archive output. ### Push Command[​](#push-command "Direct link to Push Command") ``` walkeros push [options] Options: -e, --event Event to process (required) --flow Flow to use -p, --platform Platform override --simulate Simulate a step (repeatable for destination.*). Format: source.NAME | destination.NAME | transformer.NAME --mock Mock a step with a specific return value (repeatable) --snapshot JS file to eval before execution (sets global state) ``` ### Validate Command[​](#validate-command "Direct link to Validate Command") ``` walkeros validate [options] Options: --type Validation type (default: flow). Also: event, mapping, contract --path Validate entry against package schema (e.g. destinations.snowplow) --flow Flow name for multi-flow configs --strict Treat warnings as errors --json JSON output Exit codes: 0 = Valid 1 = Errors found 2 = Warnings (with --strict) 3 = Input error ``` ### Run Command[​](#run-command "Direct link to Run Command") ``` # HTTP event collection server walkeros run [options] Options: -p, --port Port (default: 8080) -h, --host Host (default: 0.0.0.0) ``` `run` accepts a flow config, a pre-built bundle, or a `.tar.gz`/`.tgz` flow archive (URL or local file). For an archive, the CLI fetches or reads the gzip, extracts the bundle and its sibling `node_modules/`, and runs the entry. This lets server flows whose step packages are external resolve those packages at runtime from the extracted `node_modules/`. ``` # Run a packed server bundle from a local archive walkeros run flow.tar.gz --port 8080 # Run a packed server bundle from a URL walkeros run https://example.com/flow.tar.gz ``` *** ## Bundler Gotchas[​](#bundler-gotchas "Direct link to Bundler Gotchas") * **Circular copies:** Never include the output directory itself (e.g., `include: ["./dist"]` when output is `dist/`). The CLI detects this and errors. * **Runtime paths:** The runner sets CWD to the bundle directory. File paths in `settings` resolve relative to the bundle, not the project root. * **Component names:** Source, transformer, destination, and store names must be valid JavaScript identifiers (camelCase). Hyphens like `gtag-wrapper` cause syntax errors — use `gtagWrapper` instead. * **Range conflicts:** When two transitive consumers declare incompatible ranges for the same dep (e.g., `arrify@^3.0.0` vs `arrify@^2.0.0`), the bundler resolves the chosen range to a concrete version and nests non-satisfying specs under their consumer. If a post-install warning surfaces declared-vs-installed mismatches, pin the dep in `config.bundle.overrides`. Set `BUNDLER_STRICT_RANGES=0` to bypass strict range validation when the npm registry is unreachable. *** ## Server bundles use nft tracing[​](#server-bundles-use-nft-tracing "Direct link to Server bundles use nft tracing") Server flows are bundled with [`@vercel/nft`](https://github.com/vercel/nft). The CLI: 1. Pacote installs every package declared in `flow..config.bundle.packages` into a per-build install root. Users do **not** run `npm install` for step packages; only `@walkeros/cli` lives in their `package.json`. 2. esbuild stage 1 externalizes all step packages. 3. esbuild stage 2 emits a small ESM `flow.mjs` that imports from those externalized packages. 4. nft traces `flow.mjs`, finds every file actually reachable at runtime (including `__dirname`-loaded `.proto` files and other assets), and copies only those files into `dist/node_modules/`. There is no `walkerOS.bundle.external` annotation. nft figures it out. **Bundle directory (the server flow's unpacked artifact):** ``` dist/ ├── flow.mjs # ESM entry, expects to be at /app/flow/flow.mjs in prod ├── package.json # informational sidecar (not used by the runner) └── node_modules/ # only the files nft traced ``` The same directory can be packed into a `.tar.gz`/`.tgz` archive (see the Bundle Command section), and `walkeros run` accepts either form. Web flows are unchanged: a single `dist/walker.js`. ### Canonical Dockerfile[​](#canonical-dockerfile "Direct link to Canonical Dockerfile") ``` FROM node:22.23.0-alpine AS builder WORKDIR /build RUN npm init -y && npm install --save-dev @walkeros/cli COPY flow.json ./ RUN npx walkeros bundle flow.json -o dist/ FROM walkeros/flow:4 WORKDIR /app/flow COPY --from=builder /build/dist/ ./ ENV PORT=8080 EXPOSE 8080 ``` Notes: * The build stage only needs `@walkeros/cli`. flow\.json drives every step package install; pacote handles it. * `COPY --from=builder /build/dist/ ./` copies the whole directory (flow\.mjs + package.json + node\_modules/) into `/app/flow/`. * The runner image's defaults match `/app/flow/flow.mjs`. No `BUNDLE` env var needed. ### Escape hatch: `traceInclude`[​](#escape-hatch-traceinclude "Direct link to escape-hatch-traceinclude") If nft cannot statically reach a runtime asset (rare: `require()` of a path constructed from a runtime variable), declare it explicitly under `flow..config.bundle.traceInclude`. Paths and globs both work; both resolve against the install root, not the project directory: ``` "flows": { "default": { "config": { "platform": "server", "bundle": { "packages": { "@walkeros/server-destination-gcp": {} }, "traceInclude": [ "node_modules/some-pkg/data/*.json", "node_modules/another-pkg/lib/runtime-loaded.js" ] } } } } ``` ### Cache (CI)[​](#cache-ci "Direct link to Cache (CI)") The bundler caches pacote downloads under `process.env.NPM_CACHE_DIR` (default `/cache/npm`). On CI, persist that path with `actions/cache`: ``` - uses: actions/cache@v4 with: path: .walkeros-cache/npm key: walkeros-${{ hashFiles('**/flow.json') }} - run: WALKEROS_TMP_DIR=.walkeros-cache npx walkeros bundle flow.json -o dist/ ``` **CI smoke check:** `cd dist && node -e "import('./flow.mjs').then(()=>console.log('ok'))"` plus `du -sh node_modules` (typical: 30-50MB for GCP destination, 10k+ files; use `.dockerignore`). *** ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Bundle Fails[​](#bundle-fails "Direct link to Bundle Fails") 1. **Check JSON syntax**: `walkeros validate flow.json --flow` 2. **Check package names**: Ensure packages exist on npm 3. **Clear cache**: `walkeros cache clear` ### Events Not Processing[​](#events-not-processing "Direct link to Events Not Processing") 1. **Validate event**: `walkeros validate event.json` 2. **Check mapping**: Event must match entity/action in mapping 3. **Use simulate first**: `walkeros push flow.json -e event.json --simulate destination.demo -v` ### Destination Not Found in Simulation[​](#destination-not-found-in-simulation "Direct link to Destination Not Found in Simulation") If `--simulate destination.NAME` errors with "not found in collector", the destination likely has `require: ["consent"]` in its config. This delays initialization until a `walker consent` event fires — which doesn't happen during simulation. **Fix:** Remove or comment out the `require` field for simulation testing: ``` { "destinations": { "gtag": { "package": "@walkeros/web-destination-gtag", "config": { "settings": { "measurementId": "G-XXXXXX" } } } } } ``` ### Destination Silent (0 Events Received)[​](#destination-silent-0-events-received "Direct link to Destination Silent (0 Events Received)") If the destination is found but receives 0 events: 1. **Check consent**: If destination has `consent: { marketing: true }`, the event must include matching consent. Add to event JSON: `{ "name": "page view", "data": {...}, "consent": { "marketing": true } }` 2. **Check mapping**: The event name must match a mapping rule (entity/action keys). Unmapped events pass through unmodified. 3. **Check policy**: Policy runs BEFORE mapping — verify policy isn't redacting fields needed by mapping rules. ### Web Simulation Transport[​](#web-simulation-transport "Direct link to Web Simulation Transport") Web simulations run in JSDOM. `fetch` and `navigator.sendBeacon` are polyfilled as tracked no-ops -- no real HTTP requests are made. Captured network calls are included in `PushResult.networkCalls` when present. ### Local Packages Not Found[​](#local-packages-not-found "Direct link to Local Packages Not Found") Use absolute or relative paths in `flow..config.bundle.packages`: ``` { "config": { "platform": "web", "bundle": { "packages": { "my-destination": { "path": "./local/my-destination" } } } } } ``` ### CLI Prints "Upgrade Required" and Exits With Code 2[​](#cli-prints-upgrade-required-and-exits-with-code-2 "Direct link to CLI Prints \"Upgrade Required\" and Exits With Code 2") The walkerOS app requires a newer `@walkeros/cli` version for the endpoint you just called. Upgrade and retry: ``` npm install -g @walkeros/cli@latest ``` See [Upgrading](https://walkeros.io/docs/upgrading) for the full version-negotiation rules. *** ## Where CLI Lives[​](#where-cli-lives "Direct link to Where CLI Lives") | Location | Purpose | | ---------------------------- | ----------------------- | | `packages/cli/` | CLI source code | | `packages/cli/src/commands/` | Command implementations | | `packages/cli/examples/` | Example flow configs | | `packages/cli/README.md` | Full CLI documentation | *** ## Networking[​](#networking "Direct link to Networking") Outbound requests to a configured `WALKEROS_APP_URL` carry an `X-Walkeros-Client: walkeros-cli/{version}` header so the host can attribute usage. No PII; the header is the only client identifier. ## Telemetry[​](#telemetry "Direct link to Telemetry") The CLI supports anonymous usage telemetry (installation id, command name, outcome, duration, environment) to help improve the tool. Telemetry is **off by default** and requires explicit opt-in. No persistent identifier is written before consent. * **Opt in:** `walkeros telemetry enable`. * **Opt out:** `walkeros telemetry disable`. * **Current state:** `walkeros telemetry status`. * **Forced off (override):** set `DO_NOT_TRACK=1` or `WALKEROS_TELEMETRY_DISABLED=1`. * **Debug:** set `WALKEROS_TELEMETRY_DEBUG=1` to print the payload to stderr (requires opt-in; no network traffic). * **Docs:** see [apps/telemetry](/docs/apps/telemetry.md) for the full event list, privacy details, and legal basis. * **Contract (source of truth):** [`packages/cli/src/telemetry/flow.json`](https://github.com/elbwalker/walkerOS/blob/main/packages/cli/src/telemetry/flow.json). *** ## Related Skills[​](#related-skills "Direct link to Related Skills") * [understanding-mapping](/skills/walkeros-understanding-mapping.md) - Mapping configuration * [understanding-flow](/skills/walkeros-understanding-flow.md) - Data flow architecture * [create-destination](/skills/walkeros-create-destination.md) - Creating destinations * [create-source](/skills/walkeros-create-source.md) - Creating sources * [debugging](/skills/walkeros-debugging.md) - Troubleshooting event flow **Detailed References:** * [commands-reference.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-using-cli/commands-reference.md) - All commands with full options * [flow-configuration.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-using-cli/flow-configuration.md) - Complete Flow\.Json reference * [server-deployment.md](https://github.com/elbwalker/walkerOS/blob/main/skills/walkeros-using-cli/server-deployment.md) - Server flow deployment guide --- # Using the walkerOS Logger ## Overview[​](#overview "Direct link to Overview") The logger is walkerOS's standard logging system, available in all sources and destinations via `env.logger` or `logger` parameter. It provides scoped, level-aware logging that replaces console.log. **Core principle:** Don't log what the collector already logs. Only log meaningful operations like external API calls, transformations, and validation errors. ## Logger Access[​](#logger-access "Direct link to Logger Access") ### In Sources[​](#in-sources "Direct link to In Sources") ``` export const sourceFetch = async ( config: PartialConfig, env: Types['env'], // env.logger is available here ): Promise => { // Logger is scoped automatically by collector: [type:sourceId] env.logger.info('Server listening on port 3000'); }; ``` ### In Destinations[​](#in-destinations "Direct link to In Destinations") ``` export const destinationDataManager: DestinationInterface = { async init({ config, env, logger }) { // logger parameter is scoped automatically: [datamanager] logger.debug('Auth client created'); }, async push(event, { config, data, env, logger }) { // logger parameter is scoped: [datamanager] logger.debug('API response', { status: 200 }); }, }; ``` **Note:** You don't need to create or configure the logger—it's provided automatically with proper scoping. ## Logger Methods[​](#logger-methods "Direct link to Logger Methods") ``` interface Logger.Instance { error(message: string | Error, context?: unknown | Error): void; warn(message: string | Error, context?: unknown | Error): void; info(message: string | Error, context?: unknown | Error): void; debug(message: string | Error, context?: unknown | Error): void; throw(message: string | Error, context?: unknown): never; json(data: unknown): void; scope(name: string): Logger.Instance; } ``` ### Log Levels[​](#log-levels "Direct link to Log Levels") * **ERROR (0)**: Always visible—use for fatal/critical errors only * **WARN (1)**: Degraded state, config issues, transient failures * **INFO (2)**: High-level operations (server startup, event processed) * **DEBUG (3)**: Low-level details (API calls, transformations) **Default**: ERROR only (must configure to see WARN/INFO/DEBUG) ### Context Parameter[​](#context-parameter "Direct link to Context Parameter") All methods accept optional structured context: ``` logger.debug('Sending to API', { endpoint: '/events', method: 'POST', eventCount: 5, }); // Output: DEBUG [datamanager] Sending to API { endpoint: '/events', method: 'POST', eventCount: 5 } ``` ## When to Log (and When NOT to)[​](#when-to-log-and-when-not-to "Direct link to When to Log (and When NOT to)") ### ❌ DON'T Log These (Collector Handles)[​](#-dont-log-these-collector-handles "Direct link to ❌ DON'T Log These (Collector Handles)") * **Init status**: "Initializing...", "Init started", "Init complete" * **Push status**: "Processing event...", "Event received" * **Generic status**: "Settings validated", "Config loaded" * **Duplicate scoping**: Don't add source/dest name (already in scope) **Why:** Collector can log these automatically since it calls init/push and has scoped logger. ### ✅ DO Log These (Meaningful Operations)[​](#-do-log-these-meaningful-operations "Direct link to ✅ DO Log These (Meaningful Operations)") * **External API calls**: Before/after with request/response details * **Auth operations**: Token refresh, client creation/failures * **Transformations**: Complex mappings or data processing * **Validation errors**: Always use `logger.throw` for fatal errors ## Usage Patterns[​](#usage-patterns "Direct link to Usage Patterns") ### Pattern 1: Validation Errors (Always Use logger.throw)[​](#pattern-1-validation-errors-always-use-loggerthrow "Direct link to Pattern 1: Validation Errors (Always Use logger.throw)") ``` // ✅ GOOD - Fatal configuration error async init({ config, logger }) { const { apiKey, projectId } = config.settings || {}; if (!apiKey) { logger.throw('Config settings apiKey missing'); } if (!projectId) { logger.throw('Config settings projectId missing'); } } ``` **Why logger.throw:** * Logs the error at ERROR level (always visible) * Throws Error automatically (no separate throw needed) * Collector catches and handles gracefully * Never returns (TypeScript type: `never`) ### Pattern 2: External API Calls[​](#pattern-2-external-api-calls "Direct link to Pattern 2: External API Calls") ``` // ✅ GOOD - Log external calls with context async push(event, { config, logger }) { const endpoint = 'https://api.vendor.com/events'; // Log before call logger.debug('Calling API', { endpoint, method: 'POST', eventId: event.id, }); const response = await fetch(endpoint, { method: 'POST', body: JSON.stringify(event), }); // Log after call logger.debug('API response', { status: response.status, ok: response.ok, }); if (!response.ok) { const errorText = await response.text(); logger.throw(`API error (${response.status}): ${errorText}`); } } ``` ### Pattern 3: Auth Operations[​](#pattern-3-auth-operations "Direct link to Pattern 3: Auth Operations") ``` // ✅ GOOD - Log auth client creation async init({ config, logger }) { try { const authClient = await createAuthClient(config.settings); logger.debug('Auth client created'); return { env: { authClient }, }; } catch (error) { logger.throw( `Authentication failed: ${error instanceof Error ? error.message : 'Unknown error'}`, ); } } ``` ### Pattern 4: Server Startup (Sources)[​](#pattern-4-server-startup-sources "Direct link to Pattern 4: Server Startup (Sources)") ``` // ✅ GOOD - Log server listening (high-level info) if (settings.port !== undefined) { server = app.listen(settings.port, () => { env.logger.info( `Express source listening on port ${settings.port}\n` + ` POST ${settings.path} - Event collection (JSON body)\n` + ` GET ${settings.path} - Pixel tracking (query params)\n` + ` OPTIONS ${settings.path} - CORS preflight`, ); }); } ``` ## Anti-Patterns (What NOT to Do)[​](#anti-patterns-what-not-to-do "Direct link to Anti-Patterns (What NOT to Do)") ### ❌ BAD: Verbose Init Logging[​](#-bad-verbose-init-logging "Direct link to ❌ BAD: Verbose Init Logging") ``` async init({ logger }) { logger.debug('Data Manager init started'); // Redundant logger.info('Data Manager initializing...'); // Redundant logger.debug('Settings validated'); // Redundant const authClient = await createAuthClient(); logger.debug('Auth client created'); // OK logger.info('Data Manager ready'); // Redundant } ``` **Problem:** Collector knows when init is called. Only log meaningful operations (auth client creation). ### ❌ BAD: Redundant Push Logging[​](#-bad-redundant-push-logging "Direct link to ❌ BAD: Redundant Push Logging") ``` async push(event, { logger }) { logger.debug('Processing event', { // Redundant name: event.name, id: event.id, }); // Do work... logger.info('Event processed'); // Redundant } ``` **Problem:** Collector knows when push is called and can log automatically. ### ❌ BAD: Using console.log[​](#-bad-using-consolelog "Direct link to ❌ BAD: Using console.log") ``` // ❌ NEVER use console.log in sources/destinations console.log('Processing event:', event.name); // ✅ Use logger instead logger.debug('API call', { endpoint }); ``` ## Migration Checklist[​](#migration-checklist "Direct link to Migration Checklist") When updating a source/destination to use the logger: * Remove all `console.log`, `console.warn`, `console.error` statements * Remove verbose init/push status logging (let collector handle) * Add `logger.throw` for all validation errors (apiKey missing, etc.) * Add `logger.debug` before external API calls (with endpoint, method) * Add `logger.debug` after external API calls (with response status) * Add `logger.debug` for auth operations (client creation, token refresh) * Use structured context (objects) instead of string concatenation * Verify tests still pass with logger mocked ## Testing with Logger[​](#testing-with-logger "Direct link to Testing with Logger") Use `createMockLogger` from `@walkeros/core` in tests: ``` import { createMockLogger } from '@walkeros/core'; test('throws on missing apiKey', () => { const logger = createMockLogger(); expect(() => { destination.init({ config: {}, logger }); }).toThrow('Config settings apiKey missing'); expect(logger.throw).toHaveBeenCalledWith('Config settings apiKey missing'); }); ``` ## Log Level Configuration[​](#log-level-configuration "Direct link to Log Level Configuration") Default log level is ERROR. To see INFO/DEBUG logs: ``` import { startFlow } from '@walkeros/collector'; const { elb } = await startFlow({ logger: { level: 'DEBUG', // Show all logs }, destinations: { /* ... */ }, }); ``` **Levels:** * `'ERROR'`: Only errors (default) * `'WARN'`: Errors + warnings * `'INFO'`: Errors + warnings + info * `'DEBUG'`: Everything ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-destinations](/skills/walkeros-understanding-destinations.md) - Destination interface * [walkeros-understanding-sources](/skills/walkeros-understanding-sources.md) - Source interface **Key Files:** * [packages/core/src/logger.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/logger.ts) - Logger implementation * [packages/core/src/types/logger.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/logger.ts) - Logger types * [packages/core/src/mockLogger.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/mockLogger.ts) - Testing utilities **Best Practice Examples:** * [packages/server/destinations/datamanager/src/push.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/server/destinations/datamanager/src/push.ts) - API logging patterns * [packages/server/sources/express/src/index.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/server/sources/express/src/index.ts) - Server startup logging **Needs Improvement:** * [packages/server/sources/fetch/src/index.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/server/sources/fetch/src/index.ts) - Missing all logging * [packages/server/destinations/meta/src/push.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/server/destinations/meta/src/push.ts) - Missing push logging * [packages/server/destinations/aws/src/firehose/push.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/server/destinations/aws/src/firehose/push.ts) - Missing push logging --- # Using Step Examples ## Overview[​](#overview "Direct link to Overview") Step examples are structured `{ in, out }` pairs (with optional `mapping` for destinations) that define the expected input/output behavior of each step in a flow. They serve as: * **Test fixtures** for automated `it.each` testing * **Simulation data** for `walkeros push --simulate` * **MCP context** for AI-assisted development * **Documentation** showing real-world usage Every source, transformer, and destination can ship step examples alongside its code. ## The Three Type Zones[​](#the-three-type-zones "Direct link to The Three Type Zones") Each step in a flow sits at a boundary between arbitrary external formats and the walkerOS event model: ``` Source Transformer Destination ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ in: arbitrary │ │ in: walkerOS │ │ in: walkerOS │ │ (HTTP req, DOM) │ │ Event │ │ Event │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ ▼ │ │ ▼ │ │ out: walkerOS │ │ out: walkerOS │ │ out: arbitrary │ │ Event │ │ Event | false │ │ (gtag, API) │ └─────────────────┘ └─────────────────┘ └─────────────────┘ arbitrary ──────────── walkerOS.Event ──────────── arbitrary ``` * **Source** converts platform-specific input into a walkerOS event * **Transformer** receives and returns walkerOS events (or `false` to filter) * **Destination** converts a walkerOS event into vendor-specific output ## The `in`/`out`/`trigger` Format[​](#the-inouttrigger-format "Direct link to the-inouttrigger-format") Every step example is an object with `in` (input), `out` (array of observable effects), and optional `trigger` (how to invoke): ``` type StepEffect = readonly [callable: string, ...args: unknown[]]; type StepOut = readonly StepEffect[]; interface StepExample { in: unknown; // Platform-specific input out: StepOut; // Array of effect tuples - see shape rules below trigger?: { type?: string; options?: unknown }; // How to invoke mapping?: unknown; // Destination mapping rule command?: 'config' | 'consent' | 'user' | 'run'; // Route to walker command } ``` `out` is **always an array of effect tuples**, even for a single effect. Each tuple is `[callable, ...args]`. The first element is the callable's public name (the SDK function users would write). Remaining elements are the arguments. ### Callable conventions[​](#callable-conventions "Direct link to Callable conventions") | Component type | Callable | Notes | | ------------------------------ | --------------------------------------- | --------------------------------------------------------------------------------------- | | Destination (SDK function) | `'gtag'`, `'fbq'`, `'ttq.track'` | Public SDK name. Dotted paths render literally. | | Destination (method on global) | `'analytics.track'`, `'dataLayer.push'` | Method notation. | | Destination (HTTP) | `'fetch'`, `'sendServer'` | The actual call users would make. NOT `env.*` prop names - those are internal plumbing. | | Source | `'elb'` | The walker public push API. | | Transformer | `'return'` | Reserved keyword. Renders as `return ` (no parens). | `'return'` is reserved. Don't use it for anything else. Empty `out: []` means the step produced no observable effect (filtered input, transformer passthrough, validator rejection). Reserved only for cases where the destination/source deliberately emits nothing. ### Source Step Example - Server (Express)[​](#source-step-example---server-express "Direct link to Source Step Example - Server (Express)") `in` is an HTTP request shape, `out` is a tuple of the `elb()` call the source makes: ``` export const checkoutPost: Flow.StepExample = { trigger: { type: 'POST' }, in: { method: 'POST', path: '/collect', body: { name: 'order complete', data: { id: 'ORD-123', total: 149.97 } }, }, out: [ ['elb', { name: 'order complete', data: { id: 'ORD-123', total: 149.97 } }], ], }; ``` ### Source Step Example - Browser[​](#source-step-example---browser "Direct link to Source Step Example - Browser") `in` is an HTML string, `out` is a tuple of the `elb()` call: ``` export const clickEvent: Flow.StepExample = { trigger: { type: 'click', options: 'button' }, in: '', out: [ [ 'elb', { name: 'cta click', data: { label: 'Sign Up' }, trigger: 'click', entity: 'cta', action: 'click', }, ], ], }; ``` ### Transformer Step Example[​](#transformer-step-example "Direct link to Transformer Step Example") `in` is a walkerOS event, `out` is a `['return', value]` tuple. An empty array means the transformer passed the event through unchanged: ``` export const step = { orderPasses: { in: { name: 'order complete', data: { id: 'ORD-123' } }, out: [['return', { name: 'order complete', data: { id: 'ORD-123' } }]], }, debugFiltered: { in: { name: 'debug test', data: { message: 'noise' } }, out: [['return', false]], // Transformer rejects }, passthrough: { in: { name: 'page view' }, out: [], // No modification }, }; ``` ### Destination Step Example[​](#destination-step-example "Direct link to Destination Step Example") `in` is a walkerOS event, `mapping` is the mapping rule that transforms it, `out` is an array of call tuples - one per observable effect the destination produces. Multi-call events (e.g., GA4 + Ads + GTM for a single walker event) flatten into a single array in execution order: ``` import type { Flow } from '@walkeros/core'; import { getEvent } from '@walkeros/core'; export const purchase: Flow.StepExample = { in: getEvent('order complete', { timestamp: 1700000000 }), mapping: { name: 'Purchase', data: { map: { value: 'data.total', currency: { value: 'EUR' } } }, }, out: [ [ 'fbq', 'track', 'Purchase', { value: 555, currency: 'EUR' }, { eventID: '1700000000-gr0up-1' }, ], ], }; ``` Multi-tool example (one walker event produces GA4, Ads, GTM in order): ``` out: [ ['gtag', 'event', 'purchase', { transaction_id: 'o1', value: 555 }], ['gtag', 'event', 'conversion', { send_to: 'AW-123', value: 555 }], ['dataLayer.push', { event: 'purchase', ecommerce: { ... } }], ], ``` Each export is a self-contained `Flow.StepExample` - no intermediate variables, no `all` aggregation. The `mapping` field ties the mapping rule to the example so tests can register it dynamically: `{ [event.entity]: { [event.action]: example.mapping } }`. Consumers iterate all examples via `Object.entries(examples.step)` - `export * as step` exposes every named export directly. ### Command Step Example[​](#command-step-example "Direct link to Command Step Example") Some destinations need to respond to walker commands (`walker consent`, `walker user`, `walker run`), not events. Set the `command` field to route `in` through `elb('walker ', in)` instead of pushing it as an event: ``` export const consentGranted: Flow.StepExample = { command: 'consent', in: { marketing: true, functional: true }, out: [ 'consent', 'update', { ad_storage: 'granted', analytics_storage: 'granted' }, ], }; ``` Supported commands: `config`, `consent`, `user`, `run`. * When `command` is absent (default), `in` is pushed as a regular event via `elb(in)`. * When `command` is set, `mapping` is **not** applied - commands don't flow through event mapping. * The `out` format is destination-specific. For gtag it's `[action, subAction, params]` matching `gtag(...)` calls. ## Writing Examples[​](#writing-examples "Direct link to Writing Examples") ### Best Practices[​](#best-practices "Direct link to Best Practices") 1. **Use camelCase names** -- descriptive of the scenario (`checkoutPost`, `debugFiltered`, `pageViewBasic`) 2. **Cover happy path and edge cases** -- include at least one positive and one negative case for transformers 3. **Use realistic data** -- real-looking event names, IDs, and values 4. **Keep examples minimal** -- only include fields relevant to the step's logic; omit empty/default-value fields (`context: {}`, `nested: []`, `user: {}`, etc.) 5. **Export from `examples/step.ts`** -- follow the existing `dev.ts` structure 6. **Title and describe public examples** -- add `title` (2-5 words) and `description` (one short sentence, 10-25 words) for every example that should appear in docs. Mark test-only fixtures with `public: false`. ### File Structure[​](#file-structure "Direct link to File Structure") ``` // src/examples/step.ts - only Flow.StepExample exports, nothing else import type { Flow } from '@walkeros/core'; import { getEvent } from '@walkeros/core'; export const purchase: Flow.StepExample = { in: getEvent('order complete', { timestamp: 1700000000 }), mapping: { name: 'Purchase', data: { map: { value: 'data.total' } } }, out: ['track', 'Purchase', { value: 555 }, { eventID: '1700000000-gr0up-1' }], }; // src/examples/index.ts export * as env from './env'; export * as step from './step'; ``` The file exports **only** `Flow.StepExample` objects. No intermediate variables, no `all`, no `config`. Consumers iterate via `Object.entries(examples.step)`. ### Rule: examples live only on the `./dev` subpath[​](#rule-examples-live-only-on-the-dev-subpath "Direct link to rule-examples-live-only-on-the-dev-subpath") Examples (and schemas) must be exported **only** from `src/dev.ts` (the `./dev` package subpath), never from a production entry (`src/index.ts` / the package main). Production bundles tree-shake on the import path: if `examples` is reachable from the main entry, the bundler inlines every example (and its `getEvent` fixtures) into the shipped JS, bloating consumer bundles with test-only data. Importing them from `./dev` keeps them available for simulation and testing while leaving production output clean. A `no-restricted-syntax` ESLint rule enforces this, so a `export ... examples` from a production entry fails lint. ## Metadata: title, description, public[​](#metadata-title-description-public "Direct link to Metadata: title, description, public") Every `Flow.StepExample` accepts three optional metadata fields that control how it surfaces in docs and MCP output: * **`title?: string`** - overrides the default camelCase-to-spaced heading in website docs. Keep it short (2-5 words), human-readable. * **`description?: string`** - one short sentence (10-25 words) rendered above each example in docs and surfaced in MCP `flow_examples` output. Explains what the example demonstrates, not how. * **`public?: boolean`** - defaults to `true`. When `false`, the example is excluded from the website docs render and from default MCP `flow_examples` output. It still runs in tests and remains callable via CLI/MCP `flow_simulate`. ``` export const purchase: Flow.StepExample = { title: 'Purchase', description: 'A completed order mapped to the Meta Pixel Purchase standard event.', in: getEvent('order complete', { timestamp: 1700000000 }), mapping: { name: 'Purchase', data: { map: { value: 'data.total' } } }, out: [['fbq', 'track', 'Purchase', { value: 555, currency: 'EUR' }]], }; // Internal fixture - runs in tests, hidden from docs and default MCP output. export const debugFiltered: Flow.StepExample = { public: false, in: { name: 'debug test', data: { message: 'noise' } }, out: [['return', false]], }; ``` ### When to mark `public: false`[​](#when-to-mark-public-false "Direct link to when-to-mark-public-false") **Mark `public: false` when:** * The example is a transformer rejection (`out: [['return', false]]`) that only proves filtering works. * It's an error-path or malformed-input case. * It's a redundant variant kept only for regression coverage. * It's an internal fixture used by cross-package tests. **Keep public (omit the field) when:** * It's the happy path. * It teaches a distinct mapping or usage pattern. * It's the only example for some entity/action the package handles. ## Source Trigger Metadata[​](#source-trigger-metadata "Direct link to Source Trigger Metadata") Source step examples can include a `trigger` field for simulation: ``` { in: '', trigger: { type: 'click' }, out: { name: 'cta click', data: { label: 'Sign Up' } } } ``` When simulating via CLI or MCP, the step example maps to `SourceInput`: * `in` -> `content` (the actual source input) * `trigger` -> `trigger` (which mechanism to fire) Destination and transformer examples don't use `trigger`. ## Simulating with Step Examples[​](#simulating-with-step-examples "Direct link to Simulating with Step Examples") Use the `--step` flag to target a specific step, then provide the event as `SourceInput` (`{ content, trigger? }`): ``` # Simulate a source step with trigger metadata walkeros push flow.json --simulate source.browser --event '{"content":"...","trigger":{"type":"click"}}' ``` The MCP `flow_examples` tool returns `trigger` metadata alongside `in`/`out`, and `mapping` for destination examples, giving full visibility into how input events are transformed to vendor-specific output. ## Validating Examples[​](#validating-examples "Direct link to Validating Examples") Cross-step example validation is included automatically when validating a flow: ``` # Validate flow config including step example compatibility walkeros validate flow.json ``` Flow validation checks that: * Source `out` types match transformer `in` types * Transformer `out` types match destination `in` types * Connected steps have compatible examples ## Testing with Examples[​](#testing-with-examples "Direct link to Testing with Examples") The primary use of step examples is automated testing with `it.each`. ### Source Tests (createTrigger)[​](#source-tests-createtrigger "Direct link to Source Tests (createTrigger)") Sources export `createTrigger` from their examples. It follows the unified `Trigger.CreateFn` interface: receives `Collector.InitConfig`, lazily starts the flow, and returns a trigger function that simulates real-world invocations. Each package implements createTrigger differently: * **Browser:** Injects HTML into DOM, starts flow, dispatches native events * **Express:** Boots real HTTP server, sends `fetch()` requests Use a spy destination to capture events: ``` import type { Destination, WalkerOS } from '@walkeros/core'; import { sourceExpress } from '@walkeros/server-source-express'; import { examples } from '@walkeros/server-source-express/dev'; describe('Step Examples', () => { it.each(Object.entries(examples.step))('%s', async (name, example) => { const events: WalkerOS.Event[] = []; const spy: Destination.Instance = { type: 'spy', config: { init: true }, push: jest.fn((event) => { events.push(JSON.parse(JSON.stringify(event))); }), }; const instance = await examples.createTrigger({ consent: { functional: true }, sources: { express: { code: sourceExpress, config: { settings: { port: 0 } } }, }, destinations: { spy: { code: spy } }, }); await instance.trigger(example.trigger?.type)(example.in); const found = events.find((e) => e.name === example.out.name); expect(found).toBeDefined(); }); }); ``` **Browser source note:** The browser source's elbLayer processes events via a detached promise chain (fire-and-forget). For interactive triggers (click, submit), poll until events arrive: ``` while (!events.length) await Promise.resolve(); ``` ### CMP Source Tests (Consent Assertion)[​](#cmp-source-tests-consent-assertion "Direct link to CMP Source Tests (Consent Assertion)") CMP sources push `walker consent` commands, not regular events. Assert on collector consent state instead of using a spy destination: ``` const instance = await examples.createTrigger({ consent: {}, sources: { usercentrics: { code: sourceUsercentrics, config: { settings: {} }, }, }, }); await instance.trigger( example.trigger?.type, example.trigger?.options, )(example.in); // Yield for detached elb('walker consent') chain while (!Object.keys(instance.flow!.collector.config.consent || {}).length) await Promise.resolve(); expect(instance.flow!.collector.config.consent).toEqual( expect.objectContaining(expected), ); ``` ### Server Function Handler Tests (collector.sources)[​](#server-function-handler-tests-collectorsources "Direct link to Server Function Handler Tests (collector.sources)") Server function handlers (fetch, AWS Lambda, GCP CloudFunction) don't own servers. Their `createTrigger` accesses the source instance from `collector.sources` after `startFlow` and calls `source.push()` with platform-native types: ``` function findSource(collector) { for (const source of Object.values(collector.sources || {})) { if (source.type === 'fetch') return source; } } // In trigger: const source = findSource(flow.collector); const response = await source.push(request); ``` ### Destination Functional Tests[​](#destination-functional-tests "Direct link to Destination Functional Tests") For destinations, use `startFlow` + `elb()` to run events through the real collector pipeline. This verifies the full flow including mapping: ``` import { startFlow } from '@walkeros/collector'; import { clone } from '@walkeros/core'; import { examples } from './dev'; describe('Step Examples', () => { it.each(Object.entries(examples.step))('%s', async (name, example) => { const event = example.in as WalkerOS.Event; const mapping = example.mapping as Rule | undefined; const mockFn = jest.fn(); const env = clone(examples.env.push); env.window.fbq = mockFn; const dest = jest.requireActual('.').default; const { elb } = await startFlow({}); // Build mapping config from event entity/action const mappingConfig = mapping ? { [event.entity]: { [event.action]: mapping } } : undefined; elb('walker destination', { code: { ...dest, env }, config: { settings: { pixelId: '1234567890' }, mapping: mappingConfig, }, }); await elb(event); expect(mockFn).toHaveBeenLastCalledWith(...(example.out as unknown[])); }); }); ``` ### Transformer Tests[​](#transformer-tests "Direct link to Transformer Tests") For transformers that can return `false`: ``` import { examples } from '../dev'; describe('transformer', () => { it.each(Object.entries(examples.step))( '%s', async (name, { in: input, out: expected }) => { const result = await transformer.push(input, context); if (expected === false) { expect(result).toBe(false); } else { expect(result).toEqual(expected); } }, ); }); ``` ## Checklist[​](#checklist "Direct link to Checklist") When adding step examples to a package or flow: * Create `src/examples/step.ts` with `Flow.StepExample` exports * Export from `src/examples/index.ts` * Use camelCase, descriptive example names * Include at least one happy-path example * For transformers, include a `false` (filtered) case * Add `it.each` test using step examples * Verify examples compile: `npm run build` * Run tests: `npm run verify:touched -- ` (L1) * Add `title` and `description` to every public example. * Mark internal/test-only examples with `public: false`. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [testing-strategy](/skills/walkeros-testing-strategy.md) - Testing patterns and env mocking * [understanding-flow](/skills/walkeros-understanding-flow.md) - Architecture and data flow * [using-cli](/skills/walkeros-using-cli.md) - CLI simulate and validate commands * [create-destination](/skills/walkeros-create-destination.md) - Destination creation workflow * [create-source](/skills/walkeros-create-source.md) - Source creation workflow * [create-transformer](/skills/walkeros-create-transformer.md) - Transformer creation workflow --- # Using store-level cache ## Overview[​](#overview "Direct link to Overview") `Flow.Store.cache` wraps any store with a read-through, write-through cache tier. Reads check the cache first, fall through to the backing on miss, and populate every tier on the unwind. Writes go to the backing first, then to the cache best-effort. The wrapping is transparent: a transformer wired to `$store.crm` does not know whether reads hit a memory cache, a Redis tier, or the underlying API. **Core principle:** the cache is advisory. Backing is the source of truth. Failed cache operations degrade performance, never correctness. **TTL is owned by the cache layer, not the store.** The store persists structured values; the cache wrapper manages expiry via the per-rule `ttl`. An expired entry is treated as a miss and re-fetched from the backing. Caching works over any structured backing. A store's byte-native `file` mode (`config.file: true`) is for serving raw assets byte-exact, not for caching. The cache wraps structured stores, so `flow_validate` warns when a store sets both `file: true` and `cache`. Drop one: serve bytes with `file: true` and no cache, or cache a structured store with no `file`. ## When to use this skill[​](#when-to-use-this-skill "Direct link to When to use this skill") * A transformer reads the same key many times across events (sessions, user lookups, product catalog enrichment) * A store has rate limits or slow HTTP round-trips (Sheets, custom API, S3 metadata) * You need to deduplicate concurrent reads on a cold key (thundering herd) * You want to compose multi-tier caches (memory → Redis → API) * You're migrating off `@walkeros/store-memory` (removed in favor of the built-in tier) ## Minimal recipe: cache a Sheets store[​](#minimal-recipe-cache-a-sheets-store "Direct link to Minimal recipe: cache a Sheets store") The talk-demo use case. A `sessions` lookup runs on every event but most events in a session share the same key. ``` { "stores": { "sessions": { "package": "@walkeros/server-store-sheets", "config": { "credentials": "$var.sheetsCredentials", "settings": { "id": "1AbC...", "sheet": "Sessions" } }, "cache": { "rules": [{ "ttl": 300 }] } } }, "transformers": { "enrich": { "package": "@walkeros/transformer-enrich", "env": { "store": "$store.sessions" } } } } ``` The first lookup hits Sheets and populates the built-in in-memory tier with a 300-second TTL. The next 300 seconds of identical reads hit memory and skip the Sheets API. Without the cache: 60 events in 60 seconds = 60 Sheets reads = quota tripped (60 req/min limit) in one minute. With the cache: 60 events in 60 seconds = 1 Sheets read. ## Recipe: memoize a slow API store[​](#recipe-memoize-a-slow-api-store "Direct link to Recipe: memoize a slow API store") Same shape, longer TTL because the API is the cold backing: ``` { "stores": { "users": { "package": "@walkeros/server-store-api", "config": { "settings": { "endpoint": "$env.USER_API_URL" } }, "cache": { "rules": [{ "ttl": 3600 }] } } } } ``` One-hour TTL is reasonable when user records change rarely. Use `flow_validate` to verify the config; use `flow_simulate` with a representative event to confirm the cache hit rate. ## Recipe: TTL by key prefix[​](#recipe-ttl-by-key-prefix "Direct link to Recipe: TTL by key prefix") Different keys can have different TTLs. Rules evaluate top-down, first match wins. The match context is `{ key, value? }`, not event data: ``` "cache": { "rules": [ { "match": { "key": "key", "operator": "prefix", "value": "session:" }, "ttl": 300 }, { "match": { "key": "key", "operator": "prefix", "value": "user:" }, "ttl": 3600 }, { "ttl": 60 } ] } ``` * `session:*` keys cache for 5 minutes * `user:*` keys cache for 1 hour * Everything else caches for 1 minute A rule without `match` always matches. Place it last as a fallback. ## Recipe: multi-tier composition (memory → Redis → API)[​](#recipe-multi-tier-composition-memory--redis--api "Direct link to Recipe: multi-tier composition (memory → Redis → API)") When the working set exceeds the memory tier's capacity, add a Redis layer between memory and the cold backing. The consumer still wires to `$store.api`; the tiers resolve automatically. ``` { "stores": { "redis": { "package": "@walkeros/server-store-redis", "config": { "settings": { "url": "$env.REDIS_URL" } }, "cache": { "rules": [{ "ttl": 300 }] } }, "api": { "package": "@walkeros/server-store-api", "config": { "settings": { "endpoint": "$env.API_URL" } }, "cache": { "store": "redis", "rules": [{ "ttl": 86400 }] } } }, "transformers": { "enrich": { "env": { "store": "$store.api" } } } } ``` Lookup chain on `api.get(K)`: 1. `api`'s tier (Redis) — HIT, return. 2. On Redis MISS, the Redis wrapper checks its own tier (memory `__cache`). If memory HIT, return up and Redis populates. 3. On all MISS, call the underlying API. Each traversed tier populates on the unwind. TTL ordering: shortest at the top (memory 300s), longest at the cold end (API 86400s). The bound on staleness is the longest TTL in the chain. **Async-safe by design.** Whether your cache store's `get` is synchronous (the built-in `__cache`, an in-memory store) or asynchronous (`@walkeros/server-store-fs`, Redis, the cache wrapper itself), the collector reads through with an `await` internally. You can mix sync and async stores freely in a multi-tier chain without any extra configuration — the same HIT/MISS semantics apply. ## Recipe: prevent thundering herd on a cold key[​](#recipe-prevent-thundering-herd-on-a-cold-key "Direct link to Recipe: prevent thundering herd on a cold key") Single-flight deduplication is on by default. 50 concurrent `store.get('session:abc')` calls on a cold cache produce **one** backing call, not 50. All callers receive the same promise. This is what makes store-level cache useful on a slow backing under high concurrency. No configuration needed; just set `cache` on the store. Verify it works with the `inflight_dedups` counter (see Observability below). ## What `cache` rules cannot do (compared to event cache)[​](#what-cache-rules-cannot-do-compared-to-event-cache "Direct link to what-cache-rules-cannot-do-compared-to-event-cache") Store rules are a stricter subset: * **No `key` field.** The cache key comes from the caller (`store.get(K)`); there is no event path to compose. * **No `update` field.** Stores have no event to mutate on hit. * **No `stop` field.** Stores always fall through on miss; halting the pipeline is an event-cache concept. * **Empty `namespace: ""` is rejected** by the schema (re-introduces the collision footgun across stores sharing `__cache`). Use the [event-level Cache](/docs/collector/cache.md) on sources, transformers, or destinations when you need `key`, `update`, or `stop`. ## Default tier: built-in `__cache`[​](#default-tier-built-in-__cache "Direct link to default-tier-built-in-__cache") Omitting `cache.store` falls back to the collector's built-in `__cache`. It is an in-memory LRU map with: * `maxEntries: 10000` (fixed in v1) * LRU access ordering on reads * Batched eviction down to 80% on overflow * Active TTL sweep every 60 seconds Each wrapped store gets an automatic namespace prefix (the store id) so multiple stores sharing `__cache` do not collide. Override with `cache.namespace: "myns"` if you want explicit control. The collector logs one line per wrapped store at startup: ``` store "sessions" caches with namespace "sessions:" via __cache ``` ## Write-through error policy[​](#write-through-error-policy "Direct link to Write-through error policy") `wrapped.set(K, V)` runs two steps: 1. **Backing first.** Await `backing.set(K, V)`. If this throws, the wrapper throws. The cache is not touched. 2. **Cache best-effort.** If the backing succeeded, attempt to populate the cache. If this throws, log a warning and return success. `wrapped.delete(K)` follows the same shape. A failed cache delete leaves a poisoned entry that serves stale data until TTL; the warning lets operators react. Backing is the source of truth. Code that wraps `set` / `delete` should assume the cache may be lagging. ## Coherence model[​](#coherence-model "Direct link to Coherence model") Read this before relying on the cache for anything correctness-sensitive: * **Read-your-writes (in-process):** yes. After `wrapped.set(K, V)`, a subsequent `wrapped.get(K)` in the same process returns `V`. * **Cross-process consistency:** eventual, bounded by the longest TTL in the chain. No invalidation channel. * **Tier-skipping repopulation:** a MISS in tier N that HITs in tier N+1 repopulates tier N. Subsequent reads hit tier N. Pick TTLs accordingly. Short TTLs (1-60s) for mostly-static lookups behind a fast backing; long TTLs (minutes-hours) for cold, expensive lookups where staleness is tolerable. ## Observability[​](#observability "Direct link to Observability") Each wrapped store exposes counters. Per-store telemetry keys: `walkeros.store_cache..`. | Counter | Use this to detect | | ------------------- | ----------------------------------- | | `hits` | Cache is actually working | | `misses` | Working-set size, cold start | | `populates` | New keys being added to cache | | `writes` | Set call volume | | `deletes` | Delete call volume | | `evictions_entries` | `maxEntries` cap being hit | | `evictions_ttl` | TTL sweeper finding expired entries | | `inflight_dedups` | Concurrent reads on a cold key | For interactive debugging at runtime: ``` const { collector } = await startFlow({ /* ... */ }); const snapshot = collector.stores.sessions.counters; console.log(snapshot); // { hits: 412, misses: 18, populates: 18, writes: 0, deletes: 0, // evictions_entries: 0, evictions_ttl: 0, inflight_dedups: 7 } ``` Healthy cache: `hits / (hits + misses)` rises over time. `inflight_dedups` proves the herd prevention worked. ## Known limitations[​](#known-limitations "Direct link to Known limitations") * **No negative caching.** A `get(K)` that returns `undefined` from the backing is not populated. Every subsequent call for a missing key re-hits the backing until the value exists. Workaround: write a sentinel value on the first miss and treat it as "not present" in transformer logic. * **No cross-process invalidation.** Writes from one process do not invalidate caches in other processes. TTL is the only mechanism. * **Cycles are rejected at startup.** `A.cache.store = B` and `B.cache.store = A` throws during init. The collector logs the cycle path before exiting. * **Renaming a store is a breaking change** to anything caching through it (`cache.store: "X"` references break). Migrate explicitly. ## Fetch & stash without `$code:`[​](#fetch--stash-without-code "Direct link to fetch--stash-without-code") When a step only needs to read a value out of a store or write one into it, you do not need to wire `$store` into the step's `env` and hand-write a `$code:` push. The declarative `state` block on a source, transformer, or destination does both directions through the mapping engine. ``` "transformers": { "stashGclid": { "state": { "mode": "set", "store": "sessions", "key": "user.session", "value": "data.gclid" } }, "restoreGclid": { "state": { "mode": "get", "store": "sessions", "key": "user.session", "value": "data.gclid" } } } ``` `key` is always the store side; `value` is always the event side. `mode` sets the direction: * **`set`** resolves `value` against the event (a path, constant, `fn`, or `map`) and writes that payload to the store under `key`. * **`get`** reads `key` from the store and writes the fetched value onto the event at the `value` path. For a `get`, `value` must be a bare string path (or a `ValueConfig` with `key`), not a constant or operator. Omit `store` to use the built-in `__cache` tier; state keys there are prefixed with `state:` so they never collide with cache entries. State is **fail-open**: a store error is logged and the event passes through unchanged. Use `state` for simple fetch/stash; reach for `$code:` only when the logic is genuinely non-declarative. Full reference: [Website: State](/docs/collector/state.md). ## Migration from `@walkeros/store-memory`[​](#migration-from-walkerosstore-memory "Direct link to migration-from-walkerosstore-memory") The dedicated `@walkeros/store-memory` package was deleted once the built-in `__cache` reached feature parity. One-line migration per occurrence: * If used **only** as a cache target (`cache.store: "memory"`): drop the store declaration and omit `cache.store`. The wrapper falls back to `__cache` automatically. * If wired into a component's `env` for non-cache use: replace with a small inline `Map` inside the component, or use one of the persistent stores (`@walkeros/server-store-fs`, `-s3`, `-gcs`, `-sheets`). `flow_validate` rejects `package: "@walkeros/store-memory"` and points at the replacement. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-stores](/skills/walkeros-understanding-stores.md) - Store interface, `$store.` wiring, lifecycle * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Architecture, stores as passive infrastructure * [walkeros-using-cli](/skills/walkeros-using-cli.md) - Bundling and simulating flows with stores **Documentation:** * [Website: Store-level cache](/docs/stores/cache.md) - Full reference for `Flow.Store.cache` * [Website: Stores](/docs/stores/.md) - Stores overview * [Website: Collector Cache](/docs/collector/cache.md) - Event-level cache integrated into the collector (separate concept) **Source files:** * [packages/core/src/types/cache.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/cache.ts) - `Cache`, `StoreCacheRule`, `EventCacheRule` types * [packages/collector/src/store-cache-wrapper.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/store-cache-wrapper.ts) - Wrapper implementation * [packages/collector/src/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/store.ts) - Two-phase store init --- # Using `@walkeros/transformer-ga4` ## Overview[​](#overview "Direct link to Overview") `@walkeros/transformer-ga4` decodes Google Analytics 4 Measurement Protocol v2 hits (`/g/collect`, `/mp/collect`) into walkerOS events. It sits in a server source's `before` chain, reads the raw HTTP request via `ctx.ingest`, and returns one walkerOS event per GA4 event in the hit (one hit can carry many events). Boundaries: server-side decoding via `source-express`, GA4 v2 only, `G-` tids only by default. Per-field patching via `extend`/`remove` is supported. ## When to use this skill[​](#when-to-use-this-skill "Direct link to When to use this skill") * You are wiring `transformer-ga4` into a server flow for the first time * You want to override one of the 33 default GA4 mappings * You need to drop a GA4 event (auto-fired noise like `user_engagement`) * You want to add a custom event key for a `gtag('event', '', ...)` call * You need to widen `tidPattern` to accept Ads (`AW-`) or DC (`DC-`) traffic * Events are not arriving and you suspect the decoder is silently dropping them ## Quickstart wiring[​](#quickstart-wiring "Direct link to Quickstart wiring") ``` { "version": 4, "flows": { "default": { "config": { "platform": "server" }, "sources": { "http": { "package": "@walkeros/server-source-express", "config": { "ingest": { "map": { "url": { "key": "url" }, "path": { "key": "path" }, "method": { "key": "method" }, "body": { "key": "body" } } } }, "before": "ga4" } }, "transformers": { "ga4": { "package": "@walkeros/transformer-ga4" } }, "destinations": { "log": { "package": "@walkeros/destination-demo" } } } } } ``` **Wiring rules:** * `transformer-ga4` belongs in `source.before`, not `destination.before`. Decoding is a pre-collector concern: GA4 hits are not yet walkerOS events. * The source must populate `ctx.ingest.url` (required, string) and `ctx.ingest.body` (optional, string). Raw text body only — pre-parsed JSON will not decode. * One HTTP request can fan out to N walkerOS events. This requires the collector fan-out fix (shipped in this release). **Common mistake:** `config.ingest` must be the `map` operator with direct `req` field paths (no `req.` prefix), e.g. `{ "map": { "url": { "key": "url" } } }`. The bare `{ "url": "req.url" }` form is silently inert: without a `map` operator the source passes the whole `req` through `getMappingValue`, and a real Express `req` (which carries functions and circular references) fails the property check, so `ctx.ingest` stays empty, the decoder reads no `url`, and the `/g/collect` `before` match never fires. This wiring contract (before-chain placement plus the `config.ingest` keys the source must populate) is also surfaced as a package hint, so `package_get` on `@walkeros/transformer-ga4` returns it without this skill. ## Configuration recipes[​](#configuration-recipes "Direct link to Configuration recipes") ### Patch one field in a default mapping (extend + remove)[​](#patch-one-field-in-a-default-mapping-extend--remove "Direct link to Patch one field in a default mapping (extend + remove)") Use `extend` to add or override individual fields of a shipped default rule without replacing it in full. Use `remove` to strip fields from the output. The `purchase` default ships `id`, `currency`, `total`, `tax`, `shipping`, and `coupon`. To add `affiliation` from a GA4 event parameter and drop `currency`: ``` "transformers": { "ga4": { "package": "@walkeros/transformer-ga4", "config": { "settings": { "mapping": { "purchase": { "extend": { "data": { "map": { "affiliation": "params.ep.affiliation" } } }, "remove": ["currency"] } } } } } } ``` `extend.data.map` is deep-merged onto the default `data.map`, leaving `id`, `total`, `tax`, `shipping`, and `coupon` intact. `remove: ["currency"]` strips that field from the final payload. A `null` value in `extend` clears an inherited field entirely (e.g. `"extend": { "name": null }`). **Full replace:** if you need to rewrite a rule from scratch (no merge), omit `extend` and `remove` and supply the complete rule directly. A rule with neither keyword keeps the existing replace behavior. ``` "settings": { "mapping": { "purchase": { "name": "order complete", "data": { "map": { "id": "params.ep.transaction_id", "total": "params.epn.value", "currency": "params.ep.currency", "coupon": "params.ep.promo_code" } } } } } ``` ### Drop an event[​](#drop-an-event "Direct link to Drop an event") Set `ignore: true` on any key. This is how `user_engagement`, `session_start`, and `first_visit` are silenced by default: ``` "settings": { "mapping": { "click": { "ignore": true } } } ``` ### Add a custom event mapping[​](#add-a-custom-event-mapping "Direct link to Add a custom event mapping") For events fired via `gtag('event', '', ...)`: ``` "settings": { "mapping": { "newsletter_subscribe": { "name": "newsletter signup", "data": { "map": { "source": "params.ep.source" } } } } } ``` Unknown event names fall through to the `'*'` key, which by default emits `ga4 track` with `data.event_name` set to the original `en`. Override `'*'` to change the global fallback rule. ### Set a custom `tidPattern`[​](#set-a-custom-tidpattern "Direct link to set-a-custom-tidpattern") By default only `G-` tids are accepted. Widen via a string regex (compiled at init): ``` "settings": { "tidPattern": "^(G|AW|DC)-" } ``` ### Custom destination after decode[​](#custom-destination-after-decode "Direct link to Custom destination after decode") The transformer returns walkerOS events with `entity action` names. Anything downstream of the collector treats them as native events — no GA4 awareness needed: ``` "destinations": { "bq": { "package": "@walkeros/server-destination-bigquery", "config": { "settings": { "project": "my-eu-project", "dataset": "events", "location": "EU" } } } } ``` ## Mapping path syntax[​](#mapping-path-syntax "Direct link to Mapping path syntax") Mappings reference fields on the decoded `GA4Hit` shape via dotted paths: | Path | Meaning | | -------------- | --------------------------------------------------------- | | `params.ep.X` | Event parameter (string), e.g. `params.ep.transaction_id` | | `params.epn.X` | Event parameter (numeric), e.g. `params.epn.value` | | `params.up.X` | User property (string) | | `params.upn.X` | User property (numeric) | | `items[0].X` | First item in the items array | | `items` | Full items array (mapped to `nested` via item rules) | Use `params.ep.X` not `ep.X`. The decoder materializes the prefixed keys under the `params` namespace. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### No events arriving[​](#no-events-arriving "Direct link to No events arriving") 1. Check that `source.config.ingest` is the `map` operator and populates `url` (required, string) and `body` (optional, string), e.g. `{ "map": { "url": { "key": "url" }, "body": { "key": "body" } } }`. A bare `{ "url": "req.url" }` object is silently inert: the source passes the whole `req` through, the real Express `req` fails the property check, and `ctx.ingest` is left empty. 2. Confirm `before: "ga4"` is set on the source, not on a destination. 3. Check the `tidPattern`: by default `^G-` blocks `AW-` / `DC-` traffic silently. 4. Confirm the request really is Measurement Protocol v2. v1 hits will not decode. ### Wrong field in the output event[​](#wrong-field-in-the-output-event "Direct link to Wrong field in the output event") 1. Check the path syntax: `params.ep.X`, not `ep.X`. 2. Verify the GA4 hit actually carries the parameter. Use a debug destination to log the decoded `params` object. 3. If you used a full-replace rule (no `extend`), your `data.map` is the entire rule. Use `extend` to inherit the defaults and add only the fields you need. ### Batched POST drops events after the first[​](#batched-post-drops-events-after-the-first "Direct link to Batched POST drops events after the first") This was a pre-`0.1.0` bug in `@walkeros/collector`. The fix preserves fan-out in `source.before` chains. Confirm both packages are on the same release wave (transformer-ga4 `0.1.0` + collector minor bump). ### `gcs` consent not applied[​](#gcs-consent-not-applied "Direct link to gcs-consent-not-applied") v1 decodes basic `gcs` (`G1XX` → `marketing` / `analytics` booleans) only. Functional/preferences flags and the newer `gcd` parameter are not decoded. Override the consent path manually if you need richer mapping. ## Limitations[​](#limitations "Direct link to Limitations") * **GA4 v2 only.** v1 Measurement Protocol is out of scope. * **`G-` tids only by default.** Override `tidPattern` for Ads/DC. * **Basic `gcs` only.** No `gcd`, no functional/preferences flags. * **Raw text body required.** Pre-parsed JSON bodies will not decode. * **Server-side only.** Web ingest via interception sources is not supported. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-transformers](/skills/walkeros-understanding-transformers.md) - Transformer interface, return values, and before/next chaining this skill relies on * [walkeros-understanding-mapping](/skills/walkeros-understanding-mapping.md) - The `data`/`map` path syntax used in the GA4 mapping recipes * [walkeros-mapping-configuration](/skills/walkeros-mapping-configuration.md) - Mapping recipes for GA4 and other vendors * [walkeros-using-cli](/skills/walkeros-using-cli.md) - Validate and simulate a flow that wires the GA4 transformer See the [website docs](https://www.walkeros.io/docs/transformers/ga4) for the authoritative reference and the [README](https://github.com/elbwalker/walkerOS/blob/main/packages/transformers/ga4/README.md) for the in-repo summary. --- # Writing Documentation ## When to Use This Skill[​](#when-to-use-this-skill "Direct link to When to Use This Skill") * Creating a new package README * Writing website documentation (MDX) * Creating or updating skills * Reviewing documentation for quality * Documenting Phase 7 of create-destination or create-source ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [understanding-flow](/skills/walkeros-understanding-flow.md) - Architecture context * [understanding-events](/skills/walkeros-understanding-events.md) - Event naming rules *** ## Documentation Types[​](#documentation-types "Direct link to Documentation Types") ### Where Content Belongs[​](#where-content-belongs "Direct link to Where Content Belongs") | Type | Purpose | Audience | | ------------------ | --------------------------------------------- | --------------------------- | | **Package README** | Installation, basic usage, API reference | Package users | | **Website docs** | Guides, integration examples, detailed config | Integrators | | **Skills** | Process knowledge, workflows | AI assistants, contributors | ### Divio Documentation Types[​](#divio-documentation-types "Direct link to Divio Documentation Types") Keep these separate - don't mix tutorials with reference: | Type | Purpose | User State | | ---------------- | ------------------ | ----------------------------- | | **Tutorial** | Learning | Studying, beginner | | **How-To Guide** | Problem-solving | Working, knows what they need | | **Reference** | Information lookup | Working, needs facts | | **Explanation** | Understanding | Studying, needs context | *** ## Example Validation (CRITICAL)[​](#example-validation-critical "Direct link to Example Validation (CRITICAL)") ### The Problem[​](#the-problem "Direct link to The Problem") AI-generated examples can be: * Syntactically correct but use non-existent APIs * Plausible-looking but don't match actual exports * Outdated, referencing deprecated patterns ### Source of Truth Hierarchy[​](#source-of-truth-hierarchy "Direct link to Source of Truth Hierarchy") ``` TIER 1: apps/quickstart/ ✓ Tested ✓ Compiled ✓ CI-validated → USE FOR: All code examples TIER 2: packages/core/src/eventGenerator.ts ✓ Canonical events ✓ Real data structures → USE FOR: Event examples TIER 3: packages/*/src/index.ts exports ✓ Actual public API → USE FOR: Verifying API names exist TIER 4: Package READMEs & Website docs ⚠ May contain errors → VERIFY against Tier 1-3 before trusting ``` ### Pinned onboarding examples[​](#pinned-onboarding-examples "Direct link to Pinned onboarding examples") Getting-started and onboarding code examples are sourced from Tier-1 `apps/quickstart/src/*.ts` (CI-validated) first, then the docs source them. The current canonical examples: * `first-event.ts`, the zero-to-first-event console example used by the quickstart and integrated-mode pages and the README. * `ga4-ecommerce.ts`, the `product add` to GA4 `add_to_cart` mapping used by the GA4 reference page and the GA4 ecommerce tutorial. These are pinned against drift by the doc-lint at `apps/scripts/validate-docs.ts` (run `npx tsx apps/scripts/validate-docs.ts`). It extracts canonical tokens from the Tier-1 source and asserts the docs snippets contain them, tolerating intentional doc simplifications (dropping the `seen` capture, arrow `push` instead of a method). `first-event.ts` is pinned on the quickstart, integrated-mode, and README pages, plus the GA4 ecommerce page; `react.mdx` is pinned shape-only (it has no `elb('page view')` push). The doc-lint also runs these structural and coherence checks: * **Bundled-snippet platform guard**: every JSON flow with `bundle` carries `config.platform` (getting-started, README, flow reference). * **Docker env guard**: quickstart Docker snippets use `BUNDLE`, not `FLOW`. * **Forbidden CLI verbs**: no `walkeros serve` or standalone `walkeros simulate` in getting-started, guides, or the gtag page. * **Same-page simulate-key resolution** (getting-started): every `push --simulate .` must resolve to a step key declared in a flow\.json on the SAME page; a simulate command on a page with no flow\.json fails. This is what catches an integrated page showing a bundled push loop. * **Flow\.json schema validation** (all docs): every full `version: 4` flow snippet is run through `schemas.validateFlowConfig` from `@walkeros/core/dev`; fragments and `version: 3` migration "before" snippets are out of scope. * **Boundary register + `:::cloud` placement** (getting-started + MCP page): no prices, tier names, or upgrade language; `:::cloud` is forbidden on golden-path pages and required where an app-only capability is documented. The rule: when you edit a pinned example, edit the Tier-1 `.ts` source and re-run the doc-lint. If the docs snippet must change too, update both so the lint passes. ### Getting-started editorial rules[​](#getting-started-editorial-rules "Direct link to Getting-started editorial rules") Three rules govern the getting-started surface (decided 2026-07-06, encoded in the doc-lint): **See your event, per mode.** Integrated-mode pages (quickstart, react, nextjs, modes/integrated, ga4-ecommerce) verify via their own console output; they must never show `walkeros push --simulate`. Only bundled-mode pages may show it, and only targeting a `.` that a flow\.json on the SAME page defines. **Open/paid boundary (Self-Host Sufficiency Rule).** Every getting-started task must reach a working result with MIT packages, the CLI, and reader-controlled infrastructure only. The hosted app appears at most once per page, only as a marked optional alternative, only via the `:::cloud` admonition, and only on the Deploy page and the MCP page. The `:::cloud` template: first sentence names the capability, second names the open-source path to the same result, optional third links `https://app.walkeros.io`. Never prices, tier names, or upgrade language anywhere in open docs. Self-host is always listed first. **Examples recipes.** Pages under `getting-started/examples/` are lean proof recipes with a fixed shape: exactly one flow\.json, one run loop, one expected-output block, plus a short intro and next steps. The run loop uses `walkeros push … --simulate .` where `` is a step key declared in the page's own flow\.json; recipes are bundled flows, so the per-mode rule applies and the same-page simulate gate enforces resolution. The expected-output block shows what actually arrives (the captured destination call or the logged event), derived from the package's step examples or the real CLI output; never invent config, mappings, or output. Every snippet is derived from validated material (package examples, tested sources, existing validated docs snippets). No `:::cloud` on recipe pages. The shape is enforced by `checkRecipeProof` in `apps/scripts/validate-docs.ts`. ### Validation Checklist[​](#validation-checklist "Direct link to Validation Checklist") Before publishing ANY code example: * [ ] **API exists?** Check `packages/*/src/index.ts` exports * [ ] **Pattern validated?** Compare against `apps/quickstart/` * [ ] **Events canonical?** Use patterns from `eventGenerator.ts` * [ ] **Example compiles?** TypeScript check passes * [ ] **Imports correct?** Package names match actual packages ### Red Flags[​](#red-flags "Direct link to Red Flags") | Red Flag | What It Indicates | | -------------------------------------- | ----------------------------------- | | API name not in package exports | Hallucinated or outdated API | | Import path doesn't match package.json | Wrong package reference | | Event name with underscore | Wrong format (should be space) | | No imports shown | Context missing, harder to validate | *** ## DRY Patterns[​](#dry-patterns "Direct link to DRY Patterns") ### Configuration snippet (package pages)[​](#configuration-snippet-package-pages "Direct link to Configuration snippet (package pages)") **When to use:** Every package page for destinations, sources, transformers, and stores. The `` snippet owns the `## Configuration`, `## Settings`, `## Mapping`, and `## Examples` H2 sections. One callsite per page, after `## Installation`. ``` import data from '@walkeros/web-destination-gtag/walkerOS.json'; import Configuration from '@site/src/components/snippets/_configuration.mdx'; ## Installation ``` `type` must be one of `destination`, `source`, `transformer`, `store`. `data` is the full `walkerOS.json` import. The snippet: * emits `## Configuration` with a short pointer to the shared group-level configuration reference, * emits `## Settings` using `` when the schema is present, otherwise a "no package-specific settings" note, * emits `## Mapping` using `` when present, otherwise a pointer to the standard rule fields, * emits `## Examples` only if `data.examples.env.init` or `data.examples.step.*` is present; iterates `examples.step` and renders each through ``. **Never** hand-roll `## Configuration`, `## Settings`, `## Mapping`, or `## Examples` on a package page. Place package-specific content (vendor quirks, advanced usage, platform-specific setup) BELOW the `` block with its own sentence-case H2 heading. Fold `## Features` bullet lists into the intro paragraph as prose — no `## Features` heading on package pages. ### Shared configuration reference (group index pages)[​](#shared-configuration-reference-group-index-pages "Direct link to Shared configuration reference (group index pages)") On group index pages (`docs/destinations/index.mdx`, `sources/index.mdx`, `transformers/index.mdx`, `stores/index.mdx`), add a `## Configuration` section that documents the wrapper fields around `settings` — fields shared across all packages of that type (consent, mapping, env, id, …). Drive the table from the core Zod schemas, filtering dev-only fields at the representation layer: ``` import { schemas } from '@walkeros/core/dev'; import { omitSchemaProperties } from '@site/src/utils/schema'; export const destinationConfig = omitSchemaProperties( schemas.DestinationSchemas.configJsonSchema, ['settings', 'init', 'mock', 'onError', 'onLog'], ); ## Configuration ``` Available schemas: * `schemas.DestinationSchemas.configJsonSchema` * `schemas.SourceSchemas.configJsonSchema` * `schemas.TransformerSchemas.configJsonSchema` * `schemas.StoreSchemas.configJsonSchema` Use `omitSchemaProperties` to hide `settings` (documented per-package) and any dev-only fields (`init`, `mock`, `chainMocks`, `onError`, `onLog`) from the user-facing table. ### Code display: CodeView vs CodeBox[​](#code-display-codeview-vs-codebox "Direct link to Code display: CodeView vs CodeBox") `@walkeros/explorer` exports two components for code display: * **``** renders Monaco, the full editor. Use when the user can edit the code (playgrounds, interactive demos). Ships the IDE. * **``** renders Shiki, read-only. Use for static code in docs, marketing pages, and anywhere editing is not a requirement. Same `` frame as `` (traffic lights, header, copy button, tabs) so visuals match. Zero Monaco runtime cost. Tokens match because Shiki uses the same TextMate grammars as VS Code / Monaco, with the matching `dark-plus` / `light-plus` themes. `` and `` render their code server-side, so it appears in the static HTML and the markdown/`llms.txt` export. Code is shown exactly as authored (no automatic reformatting), so format multi-line snippets before putting them in the `code` prop. **Rule:** default to ``. Only reach for `` when the code must be editable. ### PropertyTable component[​](#propertytable-component "Direct link to PropertyTable component") `` from `@walkeros/explorer` is the primitive. It renders a pure table — no heading, no captions. Never wrap it in a custom heading on package pages; use the `` snippet instead. **When NOT to use:** * Pages without package configuration * Reference tables (Logger API, CLI commands) * Conceptual explanations ### Schema Exports (dev.ts)[​](#schema-exports-devts "Direct link to Schema Exports (dev.ts)") Every destination/source should export schemas: ``` // src/dev.ts export * as schemas from './schemas'; export * as examples from './examples'; ``` ### Don't Duplicate[​](#dont-duplicate "Direct link to Don't Duplicate") * Link to source files instead of copying type definitions * Reference `apps/quickstart/` examples instead of writing from scratch * Use PropertyTable instead of hardcoded markdown tables *** ## Writing Hints (`src/hints.ts`)[​](#writing-hints-srchintsts "Direct link to writing-hints-srchintsts") Hints are the "experienced colleague" layer in `walkerOS.json` — they tell AI agents *when*, *why*, and *what to watch out for* beyond what schemas and examples convey. Surfaced via MCP `package_get`. Not human-facing docs. **Audience:** AI agents configuring packages on behalf of users. ### Core Principle: Expand Awareness, Don't Narrow It[​](#core-principle-expand-awareness-dont-narrow-it "Direct link to Core Principle: Expand Awareness, Don't Narrow It") Hints should open up the space of what's possible, not prescribe a single path. An LLM reading hints should think "I have more options than I realized" — not "I must follow these steps exactly." ### Writing Rules[​](#writing-rules "Direct link to Writing Rules") | Rule | Do | Don't | | --------------------------------- | ------------------------------------------------------------- | --------------------------------------- | | **Describe capabilities** | "Supports SA key, ADC, and custom client" | "Use a SA key file when outside GCP" | | **Reference schemas/examples** | "See `settings.projectId` in the schema" | Repeat what the schema description says | | **Explain why behind defaults** | "Defaults to EU location; override via `location`" | "Set location to US" | | **Flag non-obvious interactions** | "When `snakeCase: true`, all data keys transform before send" | Describe obvious behavior | | **Symptoms → causes** | "Empty table? Check projectId and dataset existence" | Step-by-step fix instructions | ### Key Naming[​](#key-naming "Direct link to Key Naming") * **kebab-case**, group related hints with prefixes: `auth-*`, `storage-*`, `query-*`, `troubleshoot-*` * Keep keys descriptive enough to scan: `auth-methods` not `a1` ### When to Add Hints[​](#when-to-add-hints "Direct link to When to Add Hints") Most packages don't need hints — schemas and examples cover the common case. Add hints when: * Multiple auth or config strategies exist and it's non-obvious when to use which * Non-obvious default behaviors need explaining * Features interact in ways the schema can't express * Prerequisites outside walkerOS are required * Common troubleshooting patterns exist ### Accuracy Check[​](#accuracy-check "Direct link to Accuracy Check") Before publishing hints, verify each claim is factually correct. Don't describe features that aren't implemented. Don't assume behavior — confirm it. ### Quality Check[​](#quality-check "Direct link to Quality Check") * Each hint expands awareness (describes capabilities, not prescriptions) * Code snippets reference schema fields, not duplicate them * No hint restates what a schema `.describe()` already says * Troubleshooting hints use symptom → cause format * Hints are atomic — one concept per hint * Every claim is factually correct against current implementation ### Export Pattern[​](#export-pattern "Direct link to Export Pattern") ``` // src/hints.ts import type { Hints } from '@walkeros/core'; export const hints: Hints = { 'auth-methods': { text: 'Supports three auth methods: ...', code: [{ lang: 'json', code: '{ "settings": { ... } }' }], }, }; ``` ``` // src/dev.ts export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Note: `hints` is a direct export (not `* as`), because it's already a `Record`. *** ## Quality Checklist[​](#quality-checklist "Direct link to Quality Checklist") ### Structure[​](#structure "Direct link to Structure") * Follows appropriate Divio type (Tutorial/How-To/Reference/Explanation) * Code example within first 100 words * First example under 20 lines * Uses `
` for advanced content * Has "Next Steps" or "Related" section ### Content[​](#content "Direct link to Content") * All event names use `"entity action"` format with space * Flow config shown as primary usage pattern * Examples are complete and copy-pasteable * Includes imports in code examples ### AI Readability[​](#ai-readability "Direct link to AI Readability") * Clear semantic headers (H2, H3, H4 hierarchy - no skipped levels) * Tables for structured data * Links to source of truth TypeScript files * Static fallback content alongside dynamic components ### Consistency[​](#consistency "Direct link to Consistency") * Uses standard table formats * Follows package/skill/website templates * Terminology matches: walkerOS, collector, destination, source * Headings use sentence case (e.g., "Next steps" not "Next Steps") * [ ] `walkerOS.json` convention followed (walkerOS field in package.json, buildDev in tsup) *** ## Templates[​](#templates "Direct link to Templates") ### Package README Template (thin pointer)[​](#package-readme-template-thin-pointer "Direct link to Package README Template (thin pointer)") READMEs are **thin pointers** to the website docs, which are the single source of truth. A README carries only: logo, title, one-line description, install, ONE current `version: 4` Quick Start, and a prominent link to the package's docs page. No `## Features`, no `## Configuration Reference`, no inline API reference, no `## Examples` block. All reference content lives on the website so it can never drift in two places. The docs link MUST be an absolute `https://www.walkeros.io/docs/...` URL, never a relative path: packages publish to npm individually and relative links break on npmjs.com. These absolute docs links are validated by `npm run validate:links` (it maps the URL to the `website/docs/` source file and asserts it exists). ````

walkerOS

# @walkeros/[package-name] [one-sentence description, adapted from the docs page intro] [Documentation](https://www.walkeros.io/docs/[path]) • [NPM Package](https://www.npmjs.com/package/@walkeros/[package-name]) • [Source Code](https://github.com/elbwalker/walkerOS/tree/main/packages/[path]) ## Installation ```bash npm install @walkeros/[package-name] ``` ## Quick start ```json { "version": 4, "flows": { "default": { "config": { "platform": "[web|server]" }, "[sources|destinations]": { "[name]": { "package": "@walkeros/[package-name]", "config": {} } } } } } ``` ## Documentation Full configuration, mapping, and examples live in the docs: **https://www.walkeros.io/docs/[path]** ## Contribute Feel free to contribute by submitting an [issue](https://github.com/elbwalker/walkerOS/issues), starting a [discussion](https://github.com/elbwalker/walkerOS/discussions), or getting in [contact](https://calendly.com/elb-alexander/30min). ## License MIT ```` Verify the Quick Start against `apps/quickstart/` and the package's real export name before publishing. Platform-agnostic packages (core, collector) show their most representative current usage instead of a `config.platform` flow. ### walkerOS.json[​](#walkerosjson "Direct link to walkerOS.json") Every package should document its `walkerOS.json` convention in the README: ``` { "walkerOS": { "type": "destination", "platform": "web" } } ``` The `walkerOS` field is an object with `type` and `platform` metadata describing the package's role in the walkerOS ecosystem. ### Website Doc Template (MDX)[​](#website-doc-template-mdx "Direct link to Website Doc Template (MDX)") Website docs fold features into the intro paragraph (no `## Features` heading). Thin-pointer READMEs also have no `## Features` heading; key capabilities go into the one-line description. This template is website-only. ```` --- title: [Title] description: [SEO description] sidebar_position: [N] --- import data from '@walkeros/[package]/walkerOS.json'; import Configuration from '@site/src/components/snippets/_configuration.mdx'; # [Title] [1-sentence description that folds in key features as prose — no separate `## Features` heading on website docs.] ## Quick start ```json // Flow config example (<15 lines) ``` ```` ## Installation[​](#installation "Direct link to Installation") \ \ ## Next steps[​](#next-steps "Direct link to Next steps") * `Related guide 1` ```` --- ## Priority Matrix ### Issue Classification | Priority | Criteria | Action | |----------|----------|--------| | **P0 Critical** | Incorrect examples, wrong APIs, security issues | Fix immediately | | **P1 High** | Missing PropertyTable, outdated domains, missing sections | Fix soon | | **P2 Medium** | Inconsistent terminology, skipped headings | Plan to fix | | **P3 Low** | Style issues, minor wording | Backlog | --- ## Non-Negotiables ### Punctuation Never use em dashes (`—`). Use a comma, period, or rephrase the sentence instead. ```text CORRECT: "free and without sampling caps" WRONG: "free — without sampling caps" ``` ### Event Naming Event naming (`"entity action"`, space not underscore): see [walkeros-understanding-events](../walkeros-understanding-events/SKILL.md). ```` ### Package References[​](#package-references "Direct link to Package References") ``` CORRECT: `@walkeros/collector` (with backticks) WRONG: @walkeros/collector (no backticks) ``` ### Domain References[​](#domain-references "Direct link to Domain References") ``` CORRECT: `www.walkeros.io` or relative paths DO NOT USE: legacy domain references ``` *** ## Process[​](#process "Direct link to Process") ### For New Package Documentation[​](#for-new-package-documentation "Direct link to For New Package Documentation") 1. **Verify examples exist** in `apps/quickstart/` or create them first 2. **Write README** using template above 3. **Write website doc** using MDX template 4. **Run quality checklist** 5. **Verify all code examples** against Tier 1-3 sources ### For Documentation Updates[​](#for-documentation-updates "Direct link to For Documentation Updates") 1. **Identify issue priority** using matrix above 2. **Check current state** against source of truth 3. **Make minimal changes** - don't over-engineer 4. **Verify examples still compile** 5. **Run quality checklist** *** ## Schema naming (`.meta({id, title})`)[​](#schema-naming-metaid-title "Direct link to schema-naming-metaid-title") Every exported schema in `packages/core/src/schemas/*.ts` (and in the `*Schema` exports of destination / source / transformer / store packages) **must** carry `.meta({ id, title, description })` so the generated JSON Schema links back to the canonical TypeScript name. Without meta, the website's PropertyTable falls back to `__schema0`, `object`, or `any`. Convention: * **id** — PascalCase, namespace-prefixed so it is globally unique across every schema in `packages/core/src/schemas`. Examples: `WalkerOSConsent`, `DestinationConfig`, `CollectorPushContext`, `LoggerConfig`. * **title** — dotted form matching the VS Code TS hover: `WalkerOS.Consent`, `Destination.Config`, `Collector.PushContext`, `Logger.Config`. * **description** — one short sentence. May repeat an existing `.describe()` where both are present. Reference implementation (copy this shape for new schemas): ``` export const ConsentSchema = z.record(z.string(), z.boolean()).meta({ id: 'WalkerOSConsent', title: 'WalkerOS.Consent', description: 'Consent state mapping (group name to granted state).', }); ``` Coverage is enforced by `packages/core/src/schemas/__tests__/meta-coverage.test.ts`. When adding a new schema, run that test — it will fail with the exact schema name if meta is missing. Destination / source / transformer / store packages that ship their own `SettingsSchema` should also call `.meta({ id, title })` on it, e.g. `DestinationGtag.Settings` / `GtagDestinationSettings`. *** ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](/skills/walkeros-understanding-flow.md) - Data flow architecture * [walkeros-understanding-events](/skills/walkeros-understanding-events.md) - Event structure and naming ---