Redis Streams
Server-side event streaming to Redis Streams via the ioredis client. Each event is appended to a configurable stream via XADD, with optional approximate MAXLEN trimming for bounded memory usage, JSON or flat serialization modes, and graceful shutdown via destroy().
Redis Streams is a server destination in the walkerOS flow:
Receives events server-side from the collector, serializes them as JSON (or flat fields), and appends them to a Redis Stream for downstream consumers (stream processors, worker queues, pub/sub fan-out, XREAD consumer groups).
Installation
- Integrated
- Bundled
Configuration
This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
redis | object | Redis Streams configuration |
Mapping
Per-event rules under config.mapping. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
| Property | Type | Description | More |
|---|---|---|---|
streamKey | string | Override Redis stream key for this rule. Takes precedence over settings.redis.streamKey. |
Examples
Default stream
An event is appended to the configured Redis stream via XADD with the full event JSON as a single field.
Order event
An order complete event is appended to the Redis stream alongside other event types for downstream consumers.
Stream key override
A mapping rule routes the event to a dedicated Redis stream instead of the destination default.
Exact trim
XADD uses exact MAXLEN trimming to enforce a precise Redis stream length at the cost of extra work.
MAXLEN trim
XADD uses approximate MAXLEN trimming to cap the Redis stream length, discarding older entries efficiently.
The destination creates a single long-lived ioredis client during init(). On flow hot-swap or server shutdown, destroy() calls client.quit() to flush in-flight commands and close the TCP connection gracefully. User-provided clients (wired in via _client) are left untouched.
Stream entry format
Each event is appended via XADD <streamKey> [MAXLEN [~] <n>] * <field> <value> ...:
- streamKey: from
settings.redis.streamKey(ormapping.settings.streamKeyoverride) *: auto-generated entry ID (ms-timestamp + sequence)- fields: in
jsonmode, a singleeventfield with the full event JSON-stringified; inflatmode, top-level event keys as separate fields (nested objects JSON-encoded) - MAXLEN: when
settings.redis.maxLenis set, trims the stream to ~N entries. Approximate (~) by default; setexactTrimming: trueto trim exactly (slower)
Use mapping.settings.streamKey to route specific events to dedicated streams (e.g. orders to walkeros:orders, identities to walkeros:identities).
Serialization modes
JSON (default)
Stores the full walkerOS event as a single event field. Easiest to consume, decode with JSON.parse().
XADD walkeros:events * event '{"entity":"page","action":"view",...}'
Flat
Stores top-level event fields as separate stream entry fields. Nested objects are JSON-encoded. Useful when downstream consumers want to filter or project specific fields without parsing the full event.
XADD walkeros:events * entity page action view timestamp 1700000100 data '{"id":"Home"}'
Configure via settings.redis.serialization: 'flat'.
Authentication
Redis Cloud / Upstash (TLS + password)
AWS ElastiCache (in-VPC, no auth)
Custom options (ACL user, non-default db)
MAXLEN trimming
Redis Streams grow unbounded by default. For high-volume pipelines, cap stream length with maxLen:
Approximate trimming (~) is used by default. Redis trims to "about N" entries with minimal overhead. For exact trimming, set exactTrimming: true (slower, use only when strict bounds are required).