File

Server Source code Package Beta

Local filesystem sink for walkerOS server flows. Appends events to a file as JSON Lines (JSONL), tab-separated values (TSV), or comma-separated values (CSV). Useful for debug logging, audit trails, replay sources, and lightweight local persistence without standing up a database or external service.

Where this fits

File is a server destination in the walkerOS flow:

Receives events server-side from the collector, serialises each event, and appends a line to a local file. No network calls, no SDKs, just `fs.createWriteStream` with `flag: 'a'`.

Installation

npm install @walkeros/server-destination-file

Integrated

import { startFlow } from '@walkeros/collector';
import { destinationFile } from '@walkeros/server-destination-file';

await startFlow({
  destinations: {
    log: {
      code: destinationFile,
      config: {
        settings: {
          filename: 'events.jsonl',
        },
      },
    },
  },
});

Bundled

Add to your flow.json destinations:

"destinations": {
  "log": {
    "package": "@walkeros/server-destination-file",
    "config": {
      "settings": {
        "filename": "events.jsonl"
      }
    }
  }
}

See bundled mode setup | CLI reference

Configuration

This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.

Settings

Property	Type	Description	More
filename*	string | object	Output filename. Static string or Mapping.Value (e.g. { fn: "$code:..." } for daily rotation, { key: "data.tenant" } for sharding).
format	'jsonl' | 'tsv' | 'csv'	Serialisation format. Defaults to jsonl.
fields	Array<string>	Event paths used as columns for tsv/csv formats. Object values are JSON-stringified. Required when format is tsv or csv.

* Required fields

Mapping

This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.

Examples

CSV with object cell

An event is written as a CSV row with the data object JSON-stringified and properly quoted for embedded commas and quotes.

Event

{
  "name": "page view",
  "data": {
    "title": "Hello, \"World\"",
    "count": 3
  },
  "context": {
    "dev": [
      "test",
      1
    ]
  },
  "globals": {
    "pagegroup": "docs"
  },
  "custom": {
    "completely": "random"
  },
  "user": {
    "id": "us3r",
    "device": "c00k13",
    "session": "s3ss10n"
  },
  "nested": [
    {
      "entity": "child",
      "data": {
        "is": "subordinated"
      }
    }
  ],
  "consent": {
    "functional": true
  },
  "id": "e64b7dc50b41b2e2",
  "trigger": "load",
  "entity": "page",
  "action": "view",
  "timestamp": 1700000000000,
  "timing": 3.14,
  "source": {
    "count": 1,
    "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
    "type": "collector",
    "schema": "4"
  }
}

Out

fs.writeFile("events.csv", "1700000000000,page view,\"{\"\"title\"\":\"\"Hello, \\\"\"World\\\"\"\"\",\"\"count\"\":3}\"\n")

Daily rotation

A mapping function derives a date-stamped filename from the event timestamp to rotate JSONL files daily.

Event

{
  "name": "order complete",
  "data": {
    "id": "ORD-1"
  },
  "context": {
    "shopping": [
      "complete",
      0
    ]
  },
  "globals": {
    "pagegroup": "shop"
  },
  "custom": {
    "completely": "random"
  },
  "user": {
    "id": "us3r",
    "device": "c00k13",
    "session": "s3ss10n"
  },
  "nested": [
    {
      "entity": "product",
      "data": {
        "id": "ers",
        "name": "Everyday Ruck Snack",
        "color": "black",
        "size": "l",
        "price": 420
      },
      "context": {
        "shopping": [
          "complete",
          0
        ]
      },
      "nested": []
    },
    {
      "entity": "product",
      "data": {
        "id": "cc",
        "name": "Cool Cap",
        "size": "one size",
        "price": 42
      },
      "context": {
        "shopping": [
          "complete",
          0
        ]
      },
      "nested": []
    },
    {
      "entity": "gift",
      "data": {
        "name": "Surprise"
      },
      "context": {
        "shopping": [
          "complete",
          0
        ]
      },
      "nested": []
    }
  ],
  "consent": {
    "functional": true
  },
  "id": "772825b42cff0d83",
  "trigger": "load",
  "entity": "order",
  "action": "complete",
  "timestamp": 1776256496000,
  "timing": 3.14,
  "source": {
    "count": 1,
    "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
    "type": "collector",
    "schema": "4"
  }
}

Out

fs.writeFile("events-2026-04-15.jsonl", "jsonl:event")

JSONL append

An event is appended as a single JSON line to a static filename using default JSONL formatting.

Event

{
  "name": "page view",
  "data": {
    "domain": "www.example.com",
    "title": "walkerOS documentation",
    "referrer": "https://www.walkeros.io/",
    "search": "?foo=bar",
    "hash": "#hash",
    "id": "/docs/"
  },
  "context": {
    "dev": [
      "test",
      1
    ]
  },
  "globals": {
    "pagegroup": "docs"
  },
  "custom": {
    "completely": "random"
  },
  "user": {
    "id": "us3r",
    "device": "c00k13",
    "session": "s3ss10n"
  },
  "nested": [
    {
      "entity": "child",
      "data": {
        "is": "subordinated"
      }
    }
  ],
  "consent": {
    "functional": true
  },
  "id": "dbe270ee1323d4a5",
  "trigger": "load",
  "entity": "page",
  "action": "view",
  "timestamp": 1700000000000,
  "timing": 3.14,
  "source": {
    "count": 1,
    "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
    "type": "collector",
    "schema": "4"
  }
}

Out

fs.writeFile("events.jsonl", "jsonl:event")

Tenant sharding

The filename is resolved from an event field so events are partitioned into per-tenant JSONL files.

Event

{
  "name": "custom event",
  "data": {
    "tenant": "acme"
  },
  "context": {
    "dev": [
      "test",
      1
    ]
  },
  "globals": {
    "lang": "elb"
  },
  "custom": {
    "completely": "random"
  },
  "user": {
    "id": "us3r",
    "device": "c00k13",
    "session": "s3ss10n"
  },
  "nested": [
    {
      "entity": "child",
      "data": {
        "is": "subordinated"
      }
    }
  ],
  "consent": {
    "functional": true
  },
  "id": "29cd1032a362d563",
  "trigger": "test",
  "entity": "custom",
  "action": "event",
  "timestamp": 1700000000000,
  "timing": 3.14,
  "source": {
    "count": 1,
    "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
    "type": "collector",
    "schema": "4"
  }
}

Out

fs.writeFile("acme", "jsonl:event")

TSV log

An event is written as a tab-separated line selecting specific fields for a compact access-log style file.

Event

{
  "name": "page view",
  "data": {
    "title": "Docs"
  },
  "context": {
    "dev": [
      "test",
      1
    ]
  },
  "globals": {
    "pagegroup": "docs"
  },
  "custom": {
    "completely": "random"
  },
  "user": {
    "session": "sess-1"
  },
  "nested": [
    {
      "entity": "child",
      "data": {
        "is": "subordinated"
      }
    }
  ],
  "consent": {
    "functional": true
  },
  "id": "0c23ce27151297be",
  "trigger": "load",
  "entity": "page",
  "action": "view",
  "timestamp": 1700000000000,
  "timing": 3.14,
  "source": {
    "count": 1,
    "trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
    "type": "browser",
    "platform": "web",
    "url": "https://example.com/docs",
    "referrer": "https://example.com/"
  }
}

Out

fs.writeFile("storage/mblog.txt", "1700000000000\tsess-1\tpage view\thttps://example.com/docs\tDocs\thttps://example.com/\n")

The destination opens one long-lived WriteStream per resolved filename during init() (for static filenames) or on first matching event (for dynamic filenames). On flow hot-swap or server shutdown, destroy() closes all cached streams.

Filename templating

filename accepts either a static string or a standard walkerOS Mapping.Value. Common patterns:

Tenant sharding

{
  "filename": { "key": "data.tenant" }
}

Each event lands in a file named after the tenant value (e.g. acme, venti). Pair with a static suffix via fn: if you need a .jsonl extension.

Daily rotation

{
  "filename": {
    "fn": "$code:`events-${new Date(value.timestamp).toISOString().slice(0,10)}.jsonl`"
  }
}

Inside the $code: function, value is the event being processed. One file per UTC day, created automatically. No logrotate, no cron.

Formats

JSONL (default)

One JSON object per line. The entire event is serialised with JSON.stringify. Easy to ingest with jq, DuckDB (read_json_auto), ClickHouse JSONEachRow, BigQuery external tables, and Athena.

TSV / CSV

Specify fields: string[] listing the event paths to extract as columns. Object values are JSON-stringified into a single cell. CSV output follows RFC 4180 quoting.

{
  "filename": "events.csv",
  "format": "csv",
  "fields": ["timestamp", "name", "data"]
}

Limits

• One file handle is opened per resolved filename and kept open until destroy(). Sharding by high-cardinality keys (e.g. user.session) can exhaust the OS file descriptor table. Be deliberate about cardinality.
• External rotation (e.g. logrotate) leaves the cached handle pointing at the rotated inode. Use the date-token pattern above instead.
• No batching. Each event is written individually; Node's stream layer buffers under the hood.
• Write errors log a warning and drop the event. They never fail the flow.