Bot detection

This server flow scores every incoming event for bot and AI-agent traffic before it reaches your destinations. The Express source forwards the request headers, the bot transformer reads them, and a demo destination logs the annotated events. Nothing is dropped: scoring is annotation, and you filter downstream.

{
  "version": 4,
  "flows": {
    "default": {
      "config": {
        "platform": "server",
        "bundle": {
          "packages": {
            "@walkeros/collector": { "version": "^4" },
            "@walkeros/server-source-express": { "version": "^4" },
            "@walkeros/server-transformer-bot": { "version": "^4" },
            "@walkeros/destination-demo": { "version": "^4" }
          }
        }
      },
      "sources": {
        "express": {
          "package": "@walkeros/server-source-express",
          "next": "bot",
          "config": {
            "ingest": {
              "map": {
                "userAgent": { "key": "headers.user-agent" },
                "acceptLanguage": { "key": "headers.accept-language" },
                "acceptEncoding": { "key": "headers.accept-encoding" },
                "secFetchSite": { "key": "headers.sec-fetch-site" },
                "secFetchMode": { "key": "headers.sec-fetch-mode" },
                "secFetchDest": { "key": "headers.sec-fetch-dest" },
                "secChUa": { "key": "headers.sec-ch-ua" },
                "secChUaMobile": { "key": "headers.sec-ch-ua-mobile" },
                "secChUaPlatform": { "key": "headers.sec-ch-ua-platform" },
                "accept": { "key": "headers.accept" },
                "method": { "key": "method" }
              }
            }
          }
        }
      },
      "transformers": {
        "bot": {
          "package": "@walkeros/server-transformer-bot",
          "config": {
            "settings": {
              "context": "fetch",
              "input": {
                "acceptLanguage": "ingest.acceptLanguage",
                "acceptEncoding": "ingest.acceptEncoding",
                "secFetchSite": "ingest.secFetchSite",
                "secFetchMode": "ingest.secFetchMode",
                "secFetchDest": "ingest.secFetchDest",
                "secChUa": "ingest.secChUa"
              }
            }
          }
        }
      },
      "destinations": {
        "console": {
          "package": "@walkeros/destination-demo",
          "config": {
            "settings": { "name": "Event Logger" }
          }
        }
      },
      "collector": { "run": true }
    }
  }
}

Run it

Save the config as flow.json and run the bot transformer of the built flow on an event. --ingest passes the request context the Express source extracts, here the user agent of a GPTBot request. Nothing is sent anywhere:

npx walkeros push flow.json -e '{"name":"page view"}' --simulate transformer.bot \
  --ingest '{"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"}'

You should see, besides an Output: line naming the built bundle before each Simulating line:

Simulating transformer: bot
success: true
  transformer.bot
    event {"name":"page view","user":{"botScore":90,"botCategory":"ai-crawler","botProduct":"GPTBot"}}
  Duration: 15758ms

What you get

Every event now carries user.botScore (0-99, higher means more automated), user.botCategory (what kind of client it is) and user.botProduct when a named detector matched. The reason codes land on ingest.bot.reasons, off the analytics event. In production the demo destination logs each annotated event; a request from GPTBot looks like this (from the transformer's examples):

[demo] {
  "name": "page view",
  "user": {
    "botScore": 90,
    "botCategory": "ai-crawler",
    "botProduct": "GPTBot"
  },
  ...
}

Without --ingest, a simulated event has no request behind it, so it resolves no signals at all and scores null with category unknown, which means not measured rather than human. ingest.bot.reasons then names which mappings are missing. To act on the scores, add a mapping condition on your real destinations that drops or segments events where botScore exceeds 50. See Bot detection for the full filtering recipes.

Next steps

• Bot detection transformer: Score ladder, categories, and filtering recipes
• Bundled mode: Build and run the flow with the CLI