Skip to main content
Ask your AI
Server Source code Package

Express

The Express source is an HTTP event collection server built on Express.js. It runs standalone or inside an existing Express app and handles JSON POST events, pixel tracking via GET with a 1x1 transparent GIF, and configurable CORS.

Where this fits

The Express source is a server source in the walkerOS flow:

It receives events via HTTP (POST/GET) and forwards them to configured destinations.

Installation​

npm install @walkeros/server-source-express

Standalone server​

import { startFlow } from '@walkeros/collector';
import { sourceExpress } from '@walkeros/server-source-express';

const { collector } = await startFlow({
  sources: {
    express: {
      code: sourceExpress,
      config: {
        settings: {
          port: 8080,
          cors: true,
        },
      },
    },
  },
  destinations: {
    // Your destinations
  },
});

// Server running at http://localhost:8080
// POST /collect - JSON event ingestion
// GET  /collect - Pixel tracking

App-only mode​

const { collector } = await startFlow({
  sources: {
    express: {
      code: sourceExpress,
      config: {
        settings: {
          // No port = app only mode
          paths: ['/events'],
        },
      },
    },
  },
});

// Access Express app for custom integration
const app = collector.sources.express.app;
app.use(yourAuthMiddleware);
app.listen(3000);

Configuration​

This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see source configuration. Package-specific fields live under config.settings and are listed below.

Settings​

PropertyTypeDescriptionMore
portintegerHTTP server port to listen on. Use 0 for random available port. If not provided, server will not start (app only mode)
pathstringSingle route path, registered for GET and POST. Ignored when paths is set; paths takes several routes.
pathsArray<any>Route paths to register. String shorthand registers GET+POST. RouteConfig allows per-route method control.
maxBatchSizeintegerMaximum number of events accepted in a single batch request
corsboolean | objectCORS configuration: false = disabled, true = allow all origins (default), object = custom configuration

Mapping​

This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.

Examples

Pixel GET

An Express GET to /collect with name and data[...] query parameters becomes a single walker elb event for pixel tracking.

Event
{
  "method": "GET",
  "path": "/collect",
  "query": {
    "name": "page view",
    "data[title]": "Home"
  }
}
Out
elb({
  "name": "page view",
  "data": {
    "title": "Home"
  }
})

POST event

An Express POST to /collect with a JSON body becomes a single walker elb event.

Event
{
  "method": "POST",
  "path": "/collect",
  "body": {
    "name": "page view",
    "data": {
      "title": "Home",
      "url": "https://example.com/"
    }
  }
}
Out
elb({
  "name": "page view",
  "data": {
    "title": "Home",
    "url": "https://example.com/"
  }
})

HTTP methods​

MethodEndpointDescription
POST/collectJSON event ingestion
GET/collectPixel tracking (returns 1x1 GIF)
OPTIONS/collectCORS preflight

POST bodies​

Content-TypeHandling
application/jsonParsed as JSON. Malformed JSON is rejected with 400 before the pipeline runs.
text/plainRead as text, then parsed as JSON when it is JSON (a navigator.sendBeacon payload, which browsers always send as text/plain;charset=UTF-8). Any other text, such as a GA4 gtag batch of newline-separated hit lines, stays the raw string.
anything elseNot read: body is undefined.

Both parsers apply the 1mb limit. A raw text body stays the raw string and yields one empty event. When config.ingest maps it (body: { key: 'body' }), a source.before decoder such as transformer-ga4 reads it from ingest.body and turns it into events. Without a decoder that empty event is rejected as invalid input by the collector: 400 on synchronous POST, counted as rejected after the ack in respond-first mode. It never reaches a destination. This follows the shared request scope contract, so fetch, GCP and AWS sources behave the same way.

Health checks

The runtime answers /health and /ready for an Express flow. Express run standalone (your own server, no runtime) has no health endpoint. Among the other server sources, AWS Lambda answers its own healthPath, and Fetch and GCP have none. See Common features.

Response modes (async)​

config.async controls whether a request is acknowledged before or after its event settles in the pipeline, resolved per HTTP method:

"async": false                       // everything synchronous
"async": true                        // everything respond-first
"async": { "GET": false, "POST": true } // explicit per-method form (the default)
"async": { "GET": true }             // GET respond-first, POST keeps its default
ValueGETPOST
unset (default)synchronousrespond-first
truerespond-firstrespond-first
falsesynchronoussynchronous
{ "GET": true }respond-firstrespond-first (its default)
{ "POST": false }synchronous (its default)synchronous

Synchronous awaits delivery before answering. On GET, a step (the file transformer, a cache destination) can respond with real content before the pixel GIF fallback applies, so asset serving works out of the box. On POST, the response reflects the outcome: 400 for invalid input, 500 for a failure a resend could fix, 200 for one it cannot fix (see Responses), and batch envelopes answer per-index outcomes.

Respond-first answers immediately: a 2xx means "accepted", not "delivered". Delivery continues in the background and failures surface through collector.status and the logs. This is the POST default (fast ingestion) and opt-in for GET, for hot pixel endpoints that must not wait for the in-process push.

Responses​

StatusMeaning
200Accepted (respond-first mode: accepted for processing, not proof of delivery) or, in synchronous mode, processed, or failed for a reason a resend cannot fix (see below)
400Rejected client input: an unparseable application/json body (body echoes the parser message), a batch larger than maxBatchSize (default 100), or, on synchronous POST, a single event the pipeline declared invalid (body echoes the validation message, e.g. Event name is required)
404Path, or method on that path, not configured in path or paths (JSON body { "success": false, "error": "Not found" })
405HEAD request on a path that serves GET (no body)
413Body exceeds the 1mb limit
415Unsupported charset
500On synchronous POST: the pipeline failed to process a valid event for a reason a resend could fix; the body is Event was not processed. Otherwise: unexpected server fault (Internal server error)
207On synchronous batch POST: at least one event was invalid or failed for a reason a resend could fix; the body lists the failed indices

A failed event is acknowledged or retried by one rule on every server source: see Producer acknowledgement.

In respond-first mode a request whose event later fails is already answered; the source writes a DEBUG line and the collector's ERROR line is the signal.

GET pixel requests answer 200 with the GIF in every mode, unless a step responded first, because an image endpoint cannot carry a useful 400. Rejected pixel input is visible through the rejected counter instead.

Unparseable bodies and invalid events are counted on collector.status.sources.<id>.rejected. An unparseable body is logged at debug, an invalid event at error with the reason, once per source and minute. A batch over maxBatchSize is answered 400 and not counted. Apart from the invalid-event line, ambient scanner traffic never reaches the error log level.

Responses carry X-Content-Type-Options: nosniff, and the source does not send X-Powered-By.

Ingest metadata​

Extract request metadata (IP, user agent, headers) and forward it through the pipeline to transformers and destinations.

config.ingest must use the map operator. Keys are output field names; values are direct field paths on the request scope (no req. prefix). A bare object like { ip: 'ip' } has no operator: walkeros validate and the bundle reject it, and if it still runs, the collector logs one ERROR line (mapping value has no operator) and no field is extracted.

import { startFlow } from '@walkeros/collector';
import { sourceExpress } from '@walkeros/server-source-express';

const { collector } = await startFlow({
  sources: {
    express: {
      code: sourceExpress,
      config: {
        settings: { port: 8080 },
        ingest: {
          map: {
            ip: { key: 'ip' },
            userAgent: { key: 'headers.user-agent' },
            origin: { key: 'headers.origin' },
            referer: { key: 'headers.referer' },
          },
        },
      },
    },
  },
});

Ingest is opt-in: without config.ingest the source forwards no request metadata, so no client IP leaves your server by accident. The keys ip and userAgent are the ones server destinations read by default (Meta, TikTok, Snapchat, Pinterest, Bing, Reddit, Criteo and X (Twitter) conversion APIs, Google Data Manager and Piwik PRO), so this map is all they need to send the client IP and user agent. Each of those destinations can turn either off with ip: false or userAgent: false in its settings.

Available ingest paths​

PathDescription
ipExpress's req.ip. The source never enables Express's trust proxy, so this is always the socket peer address; behind a proxy that is the proxy, not the visitor, see Client IP behind a proxy
headers.*HTTP headers, lowercased names (user-agent, origin, referer, etc.)
methodUppercase HTTP method
pathPathname, no query string
query.*Query parameter, repeated keys joined with ,
urlAbsolute request URL, or '' when there is no host header
bodyParsed JSON body, the raw string for non-JSON text/plain, or undefined
rawThe Express Request itself

These are the shared request scope fields. There are no top-level protocol and hostname paths: use headers.host, headers.x-forwarded-proto, or raw.protocol / raw.hostname.

Client IP behind a proxy​

ip is Express's req.ip. This source never enables Express's trust proxy setting, so req.ip is always the socket peer address. Behind a load balancer, a CDN or a serverless platform that is the last proxy in the chain, never the visitor. The X-Forwarded-For header carries the real address, but it is a comma-separated chain whose left side is written by the client, so parsing it means deciding how many trailing hops you trust, a per-deployment security judgment.

The recommended pattern needs no source setting at all: let the infrastructure that terminates the connection hand you the client address as a single-value header it controls, and map that header with the ingest map you already have.

PlatformHeaderHow it gets there
Google external HTTPS LBX-Client-IPCustom request header X-Client-IP:{client_ip_address} on the backend service
CloudflareCF-Connecting-IPSet automatically
FastlyFastly-Client-IPMust be forced in VCL: set req.http.Fastly-Client-IP = client.ip; (Fastly preserves a client-supplied value otherwise)
AkamaiTrue-Client-IPEnable in the property
nginxX-Real-IPproxy_set_header X-Real-IP $remote_addr;
"sources": {
  "express": {
    "package": "@walkeros/server-source-express",
    "config": {
      "ingest": { "map": { "ip": { "key": "headers.x-client-ip" } } }
    }
  }
}
Trust only a header your own infrastructure controls

A client can send any header name itself, including these. The pattern is safe only when your proxy sets or overwrites the header on every request, so verify that override once against your platform before trusting it. Never map a client-reachable header like a raw X-Forwarded-For as the client IP: its leftmost entries are forgeable.

On a platform that only provides raw X-Forwarded-For and no configurable header, prefer accepting an absent IP: the pipeline treats a missing signal as "measured less", never as a wrong value. Extract a hop from the chain only when your proxy's documentation guarantees the position you select. The example below is written for the Google-style contract where the proxy appends <client-ip>, <proxy-ip> to whatever the client sent, so the second entry from the right is proxy-verified. Under a proxy that appends only its own address, the same position is attacker-controlled, so verify the contract, never infer it from observed traffic:

"ip": {
  "fn": "$code:(req)=>req.headers['x-forwarded-for']?.split(',').at(-2)?.trim()"
}

.at(-2) takes the entry one hop in from the right, which is the proxy-verified client entry under the contract above; with fewer entries the IP stays absent. Adjust the offset to your own proxy's documented behavior and confirm it with a request from a known address before trusting it.

Advanced mapping​

Use walkerOS mapping features for complex extraction:

ingest: {
  map: {
    // Simple path
    ip: { key: 'ip' },

    // Custom function
    country: { fn: (scope) => geoip.lookup(scope.ip)?.country },

    // Conditional extraction
    devMode: {
      key: 'headers.x-debug',
      condition: (scope) => scope.headers.host?.startsWith('localhost'),
    },

    // Nested structure
    request: {
      map: {
        userAgent: { key: 'headers.user-agent' },
        origin: { key: 'headers.origin' },
      },
    },
  },
}

Example request​

curl -X POST http://localhost:8080/collect \
  -H "Content-Type: application/json" \
  -d '{"name":"page view","data":{"title":"Home"}}'
💡 Need implementation support?
elbwalker offers hands-on support: setup review, measurement planning, destination mapping, and live troubleshooting. Book a 2-hour session (€399)