Skip to main content
Ask your AI

Deploy walkerOS to GCP Cloud Run

This guide deploys a walkerOS flow to Google Cloud Run, forwarding events to BigQuery.

Prerequisites​

1. Set Up BigQuery​

Follow the GCP BigQuery setup to:

Step 2 creates the dataset and the events table with walkeros setup.

For local testing, download a service account key. Production uses Workload Identity.

Enable Cloud Run API​

gcloud services enable run.googleapis.com

2. Bundle the Flow​

The example flow at https://www.walkeros.io/flows/gcp-bigquery.json receives events with the Express source on /collect and writes them to BigQuery:

{
"version": 4,
"flows": {
"server": {
"config": { "platform": "server" },
"sources": {
"express": {
"package": "@walkeros/server-source-express",
"config": {
"settings": { "paths": ["/collect"] }
}
}
},
"destinations": {
"bigquery": {
"package": "@walkeros/server-destination-gcp",
"import": "destinationBigQuery",
"config": {
"setup": { "location": "$env.BQ_LOCATION:EU" },
"settings": {
"projectId": "$env.GCP_PROJECT_ID",
"datasetId": "$env.BQ_DATASET:walkerOS",
"tableId": "$env.BQ_TABLE:events"
}
}
}
}
}
}
}

Install the CLI and bundle the flow into dist/:

npm install -g @walkeros/cli

walkeros bundle https://www.walkeros.io/flows/gcp-bigquery.json -o dist/

This creates dist/flow.mjs, dist/package.json, and dist/node_modules/. The CLI adds the packages named in package to the bundle, so the flow needs no separate package list.

"setup" lets walkeros setup create the dataset and the events table, in the region BQ_LOCATION names (default EU). Run it once. It finds Google credentials the way the runtime does, from GOOGLE_APPLICATION_CREDENTIALS (the key from step 1) or your gcloud application default credentials, and the account needs the setup permissions listed under Create service accounts:

export GCP_PROJECT_ID=your-project-id
export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/sa-bigquery.json

walkeros setup destination.bigquery -c https://www.walkeros.io/flows/gcp-bigquery.json
Environment variables

In a server flow, $env.NAME is read from the environment when the container starts, not at bundle time. Required: GCP_PROJECT_ID; without it the first event fails with Config settings projectId missing in the log. Optional with defaults: BQ_DATASET (walkerOS) and BQ_TABLE (events). Only walkeros setup uses BQ_LOCATION, for the dataset it creates. Only web flows bake $env values in, from config.bundle.env. See Reference Syntax.

3. Test Locally with Docker​

Before deploying, verify everything works locally. Replace <version> with the walkeros/flow version matching your CLI (see image tags):

# Run locally (uses key file for auth)
docker run --rm \
-p 8080:8080 \
-v "$(pwd)/dist:/app/flow:ro" \
-v $(pwd)/sa-bigquery.json:/app/sa-bigquery.json:ro \
-e GOOGLE_APPLICATION_CREDENTIALS=/app/sa-bigquery.json \
-e GCP_PROJECT_ID=your-project-id \
walkeros/flow:<version>

The image runs as the user walker (uid 1001), which must be able to read the mounted key. On a Linux host, a key file only your user can read (mode 600) is unreadable in the container; for this local test, make it readable with chmod 644 sa-bigquery.json, and delete it when you clean up.

In another terminal, send a test event:

curl -X POST http://localhost:8080/collect \
-H "Content-Type: application/json" \
-d '{
"name": "page view",
"data": {
"title": "Test Page",
"path": "/test"
}
}'

Expected response:

{ "success": true, "timestamp": 1700000000000 }

The Express source answers a POST before delivery, so the response says nothing about BigQuery. A failed write shows up as a Push failed line in the container log.

Check BigQuery for the event:

bq query --use_legacy_sql=false \
"SELECT * FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5"

4. Deploy to Cloud Run​

Create dist/Dockerfile so Cloud Run builds an image from the runtime image plus the whole dist/ directory. The runtime image starts /app/flow/flow.mjs on port 8080 by default:

FROM walkeros/flow:<version>
COPY . /app/flow/

To bundle inside Docker instead, use the canonical two-stage Dockerfile. Cloud Run uses Workload Identity, so no credentials file is needed in the image.

Deploy with the service account attached and the project ID set as an environment variable:

cd dist

gcloud run deploy walkeros-flow \
--source . \
--port 8080 \
--region europe-west1 \
--allow-unauthenticated \
--service-account=walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com \
--set-env-vars GCP_PROJECT_ID=YOUR_PROJECT_ID \
--memory 512Mi \
--cpu 1 \
--min-instances 0 \
--max-instances 10

Note the service URL from the output (e.g., https://walkeros-flow-xxx-ew.a.run.app).

5. Verify Deployment​

Health Check​

curl https://walkeros-flow-xxx-ew.a.run.app/health

Send Test Event​

curl -X POST https://walkeros-flow-xxx-ew.a.run.app/collect \
-H "Content-Type: application/json" \
-d '{
"name": "page view",
"data": {
"title": "Production Test",
"path": "/"
}
}'

Query BigQuery​

bq query --use_legacy_sql=false \
"SELECT name, data, timestamp FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5"

Read the Logs​

Cloud Run reads JSON lines from stdout and stderr as structured logs, with severity and message. Ask the flow for them with format: "json" in its collector block, then bundle and deploy again:

"collector": {
"logger": { "level": "INFO", "format": "json" }
}

A destination that fails to deliver shows up as one ERROR entry per cause and minute, with the step (destination.bigquery), the cause code and, when the vendor answered, the HTTP status in the context. See Logger for the codes.

Next Steps​

Cleanup​

To remove all created resources:

  • Delete the Cloud Run service (gcloud run services delete walkeros-flow --region europe-west1)
  • Clean up BigQuery resources
  • Delete local files (dist/, sa-bigquery.json)
💡 Need implementation support?
elbwalker offers hands-on support: setup review, measurement planning, destination mapping, and live troubleshooting. Book a 2-hour session (€399)