File
Local filesystem sink for walkerOS server flows. Appends events to a file as JSON Lines (JSONL), tab-separated values (TSV), or comma-separated values (CSV). Useful for debug logging, audit trails, replay sources, and lightweight local persistence without standing up a database or external service.
File is a server destination in the walkerOS flow:
Receives events server-side from the collector, serialises each event, and appends a line to a local file. No network calls, no SDKs, just `fs.createWriteStream` with `flag: 'a'`.
Installation
- Integrated
- Bundled
Configuration
This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
filename | string | object | Output filename. Static string or Mapping.Value (e.g. { fn: "$code:..." } for daily rotation, { key: "data.tenant" } for sharding). | |
format | 'jsonl' | 'tsv' | 'csv' | Serialisation format. Defaults to jsonl. | |
fields | Array<string> | Event paths used as columns for tsv/csv formats. Object values are JSON-stringified. Required when format is tsv or csv. |
Mapping
Per-event rules under config.mapping. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
| Property | Type | Description | More |
|---|
Examples
csv object cell
jsonl daily rotation
jsonl default
jsonl tenant shard key
tsv baersch log
The destination opens one long-lived WriteStream per resolved filename during init() (for static filenames) or on first matching event (for dynamic filenames). On flow hot-swap or server shutdown, destroy() closes all cached streams.
Filename templating
filename accepts either a static string or a standard walkerOS Mapping.Value. Common patterns:
Tenant sharding
Each event lands in a file named after the tenant value (e.g. acme, venti). Pair with a static suffix via fn: if you need a .jsonl extension.
Daily rotation
Inside the $code: function, value is the event being processed. One file per UTC day, created automatically. No logrotate, no cron.
Formats
JSONL (default)
One JSON object per line. The entire event is serialised with JSON.stringify. Easy to ingest with jq, DuckDB (read_json_auto), ClickHouse JSONEachRow, BigQuery external tables, and Athena.
TSV / CSV
Specify fields: string[] listing the event paths to extract as columns. Object values are JSON-stringified into a single cell. CSV output follows RFC 4180 quoting.
Limits
- One file handle is opened per resolved filename and kept open until
destroy(). Sharding by high-cardinality keys (e.g.user.session) can exhaust the OS file descriptor table. Be deliberate about cardinality. - External rotation (e.g.
logrotate) leaves the cached handle pointing at the rotated inode. Use the date-token pattern above instead. - No batching. Each event is written individually; Node's stream layer buffers under the hood.
- Write errors log a warning and drop the event. They never fail the flow.