Fetch
The Fetch source is a Web Standard Fetch API source for walkerOS: a platform-agnostic (Request) => Response handler that runs on Cloudflare Workers, Vercel Edge, Deno, Bun, and Node.js 18+. It supports batch processing, configurable CORS, and pixel tracking via a 1x1 transparent GIF for GET requests.
The Fetch source is a server source in the walkerOS flow:
It receives events via the Fetch API and forwards them to your destinations. Works on any platform supporting Web Standards.
Installation
npm install @walkeros/server-source-fetch- Integrated
- Bundled
Cloudflare Workers
import { Source } from '@walkeros/core';
import { sourceFetch, type SourceFetch } from '@walkeros/server-source-fetch';
import { startFlow } from '@walkeros/collector';
const { collector } = await startFlow({
sources: {
fetch: {
code: sourceFetch,
config: {
settings: { paths: ['/collect'], cors: true },
},
},
},
});
export default {
fetch: Source.getSource<SourceFetch.Types>(collector, 'fetch').push,
};Add to your flow.json sources:
"sources": {
"fetch": {
"package": "@walkeros/server-source-fetch",
"config": {
"settings": {
"paths": ["/collect"],
"cors": true
}
}
}
}This bundle is for deploying to an edge or serverless platform such as Cloudflare Workers. The walkerOS runtime (runneros, the walkeros/flow image) serves only sources with an HTTP handler, such as the Express source: under it this source receives no requests, and every request answers 503.
Configuration
This source uses the standard source config wrapper (consent, data, env, id, ...). For the shared fields see source configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
path | string | Single route path, registered for GET and POST. Ignored when paths is set; paths takes several routes. | |
paths | Array<any> | Route paths to handle. String shorthand accepts GET+POST. RouteConfig allows per-route method control. | |
cors | boolean | object | CORS configuration: false = disabled, true = allow all (default), object = custom | |
maxRequestSize | integer | Maximum request body size in bytes | |
maxBatchSize | integer | Maximum events per batch request |
Mapping
This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
Examples
Batch POST
A fetch POST with a batch array produces one walker elb event per batched item preserving order.
{
"method": "POST",
"url": "http://localhost/collect",
"body": {
"batch": [
{
"name": "page view",
"data": {
"title": "Home"
}
},
{
"name": "button click",
"data": {
"id": "cta"
}
}
]
}
}elb({
"name": "page view",
"data": {
"title": "Home"
}
});
elb({
"name": "button click",
"data": {
"id": "cta"
}
})Pixel GET
A fetch GET with name and data[...] query parameters in the URL becomes a single walker elb event for pixel-style tracking.
{
"method": "GET",
"url": "http://localhost/collect?name=page+view&data%5Btitle%5D=Home"
}elb({
"name": "page view",
"data": {
"title": "Home"
}
})POST event
A fetch POST request with a JSON body becomes a single walker elb event in a fetch-based server.
{
"method": "POST",
"url": "http://localhost/collect",
"body": {
"name": "page view",
"data": {
"title": "Docs",
"url": "https://example.com/docs"
}
}
}elb({
"name": "page view",
"data": {
"title": "Docs",
"url": "https://example.com/docs"
}
})Responses
| Status | Meaning |
|---|---|
| 200 | Event processed, or failed for a reason a resend cannot fix (see below) |
| 400 | Rejected client input: the pipeline declared the event invalid (the body echoes the validation message, for example Event name is required), or a batch larger than maxBatchSize (default 100) |
| 404 | Path not configured in path or paths |
| 405 | Method not configured on that path |
| 413 | Body larger than maxRequestSize (default 102400 bytes) |
| 500 | The pipeline failed to process a valid event for a reason a resend could fix (the body is Event was not processed), or an unexpected server fault (Internal server error) |
| 207 | Batch request where at least one event was invalid or failed for a reason a resend could fix. The body carries per-index errors |
A failed event is acknowledged or retried by one rule on every server source: see Producer acknowledgement.
A per-event "not processed" in a batch logs at DEBUG, because the collector's ERROR line is the signal. A push that throws inside a batch logs an ERROR and answers Internal server error.
Invalid input, a body over maxRequestSize and a batch over maxBatchSize are
counted on collector.status.sources.<id>.rejected rather than inflating
status.failed. An invalid event is logged at error with the reason and no
stack trace, once per source and minute; an oversized request is logged at
debug.
Ingest metadata
Extract request metadata and forward it through the pipeline.
config.ingest must use the map operator. Keys are output field names; values are direct field paths on the request scope (no req. prefix). A bare object like { url: 'url' } has no operator: walkeros validate and the bundle reject it, and if it still runs, the collector logs one ERROR line (mapping value has no operator) and no field is extracted.
const { collector } = await startFlow({
sources: {
fetch: {
code: sourceFetch,
config: {
settings: { cors: true },
ingest: {
map: {
userAgent: { key: 'headers.user-agent' },
origin: { key: 'headers.origin' },
path: { key: 'path' },
},
},
},
},
},
});Ingest is opt-in: without config.ingest the source forwards no request
metadata. Server destinations with client IP and user agent fields
(Meta, TikTok, Snapchat, Pinterest, Bing, Reddit, Criteo and X (Twitter) conversion APIs, Google Data Manager and Piwik PRO) read ingest.ip and ingest.userAgent by default, so name the keys
ip and userAgent. This source has no ip path (see below); map it from a
single-value header your own infrastructure sets:
ingest: {
map: {
ip: { key: 'headers.x-client-ip' },
userAgent: { key: 'headers.user-agent' },
},
}Available ingest paths
This source builds the shared request scope, so every contract field resolves by key.
| Path | Description |
|---|---|
method | Uppercase HTTP method |
url | Absolute request URL |
path | Pathname, no query string |
query.<name> | Query parameter |
headers.<name> | Header, lowercased name |
body | Parsed request body |
raw | The WHATWG Request itself |
ip is absent on this source: a Fetch Request reports no client IP and
walkerOS does not guess one. Map the header your proxy sets, as in
Client IP behind a proxy.
Usage
Single event
fetch('https://your-endpoint.com/collect', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
name: 'page view',
data: { title: 'Home', path: '/' },
}),
});Batch events
fetch('https://your-endpoint.com/collect', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
batch: [
{ name: 'page view', data: { title: 'Home' } },
{ name: 'button click', data: { id: 'cta' } },
],
}),
});