S3

Server Source code Package

S3-compatible object storage store using s3mini (~20 KB, zero dependencies). Works with AWS S3, Cloudflare R2, Scaleway, DigitalOcean Spaces, Backblaze B2, MinIO, and any S3-compatible provider.

Installation

npm install @walkeros/server-store-s3

Integrated

import { startFlow } from '@walkeros/collector';
import { storeS3Init } from '@walkeros/server-store-s3';

await startFlow({
  stores: {
    assets: {
      code: storeS3Init,
      config: {
        settings: {
          bucket: 'my-assets',
          endpoint: 'https://s3.eu-west-1.amazonaws.com',
          accessKeyId: process.env.S3_ACCESS_KEY,
          secretAccessKey: process.env.S3_SECRET_KEY,
          region: 'eu-west-1',
          prefix: 'public',
        },
      },
    },
  },
});

Bundled

Add to your flow.json:

"stores": {
  "assets": {
    "package": "@walkeros/server-store-s3",
    "config": {
      "settings": {
        "bucket": "my-assets",
        "endpoint": "https://s3.eu-west-1.amazonaws.com",
        "accessKeyId": "$env.S3_ACCESS_KEY",
        "secretAccessKey": "$env.S3_SECRET_KEY",
        "region": "eu-west-1",
        "prefix": "public"
      }
    }
  }
}

Configuration

This store uses the standard store config wrapper (consent, data, env, id, ...). For the shared fields see store configuration. Package-specific fields live under config.settings and are listed below.

Settings

Property	Type	Description	More
bucket*	string	S3 bucket name
endpoint*	string	S3-compatible endpoint URL
accessKeyId*	string	S3 access key ID
secretAccessKey*	string	S3 secret access key
region	string	AWS region for SigV4 signing
prefix	string	Key prefix prepended to all store keys for scoping

* Required fields

Mapping

This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.

Examples

Prefix scoping

Key "walker.js" with prefix "public/" resolves to S3 path "public/walker.js"

Event

{
  "operation": "get",
  "key": "walker.js",
  "settings": {
    "bucket": "my-assets",
    "prefix": "public"
  }
}

Out

get("public/walker.js", "Bytes<...>")

Read from S3

Read object from S3 and receive its raw bytes byte-exact

Event

{
  "operation": "get",
  "key": "walker.js"
}

Out

get("walker.js", "Bytes<(function(){...})()>")

Provider examples

Provider	Endpoint	Notes
AWS S3	https://s3.<region>.amazonaws.com	Set region to your actual region
Cloudflare R2	https://<account>.r2.cloudflarestorage.com	No egress fees
Scaleway	https://s3.<region>.scw.cloud	EU hosting
DigitalOcean	https://<region>.digitaloceanspaces.com	Simple pricing
Backblaze B2	https://s3.<region>.backblazeb2.com	Cheapest storage
MinIO	http://localhost:9000	Self-hosted

Credentials

Use $env. references in your flow config to avoid hardcoding secrets:

"accessKeyId": "$env.S3_ACCESS_KEY",
"secretAccessKey": "$env.S3_SECRET_KEY"

Unlike the AWS SDK, s3mini has no implicit credential chain: accessKeyId and secretAccessKey are always required.

File serving pattern

The primary use case is serving static files via the file transformer. This is the recommended pattern for managed deployments where files live in a bucket rather than needing to be baked into a Docker image. Set file: true so the store persists and returns bytes byte-exact:

{
  "stores": {
    "assets": {
      "package": "@walkeros/server-store-s3",
      "config": {
        "settings": {
          "bucket": "my-assets",
          "endpoint": "https://s3.eu-west-1.amazonaws.com",
          "accessKeyId": "$env.S3_ACCESS_KEY",
          "secretAccessKey": "$env.S3_SECRET_KEY",
          "prefix": "public"
        },
        "file": true
      }
    }
  },
  "transformers": {
    "file": {
      "package": "@walkeros/server-transformer-file",
      "config": { "settings": { "prefix": "/static" } },
      "env": { "store": "$store.assets" }
    }
  }
}

A request to /static/walker.js looks up public/walker.js in the my-assets bucket. Omit file for a structured key-value store instead.

Provisioning

The package ships an idempotent setup() lifecycle, invoked only by the explicit operator command:

walkeros setup store.<id>

It never runs automatically. It checks whether the bucket exists and creates it if not. Re-running setup is a no-op when the bucket already exists in your account.

Setup options

Option	Type	Default	Notes
region	string	eu-central-1	Region the bucket is created in (LocationConstraint). Falls back to settings.region when concrete (not auto).

bucket is taken from settings.bucket and is NOT duplicated under setup.

Enable provisioning

Set setup: true in the component config to enable provisioning with defaults, or pass an object to override:

{
  "stores": {
    "assets": {
      "package": "@walkeros/server-store-s3",
      "config": {
        "settings": {
          "bucket": "my-assets",
          "endpoint": "https://s3.eu-central-1.amazonaws.com",
          "accessKeyId": "$env.S3_ACCESS_KEY",
          "secretAccessKey": "$env.S3_SECRET_KEY",
          "region": "eu-central-1"
        },
        "setup": true
      }
    }
  }
}

What setup does NOT apply

s3mini is a minimal S3 client that exposes createBucket and bucketExists only. It does NOT expose PutBucketEncryption, PutPublicAccessBlock, PutBucketVersioning, PutBucketLifecycleConfiguration, or PutBucketTagging. Configure those once via the AWS Console or aws s3api.

Behavior

• Idempotent create: BucketAlreadyOwnedByYou (concurrent caller, your account) is treated as success and returns { bucketCreated: false }. BucketAlreadyExists (different AWS account owns the global name) fails loud with an actionable message so you pick a different name.
• Region resolution: explicit setup.region wins; otherwise settings.region is used when concrete (not auto); otherwise the EU default eu-central-1.

Runtime hard-fail

storeS3Init probes bucketExists() once when the collector wires the store. On a missing bucket it throws with an actionable message:

S3 bucket not found: my-assets at https://s3.eu-central-1.amazonaws.com. Run "walkeros setup store.assets" to create it.

Run walkeros setup store.<id> once to provision the bucket, then redeploy.

Security

• Key validation: Path traversal attempts (.., absolute paths) are rejected
• Prefix scoping: The prefix setting restricts all operations to a subdirectory
• No credential chain: Credentials must be explicitly provided (no ambient AWS credentials)

Structured vs file mode

The S3 store has two modes, decided once at init by config.file:

• Structured (default): values are structured StoreValue data, serialized by the shared core codec and stored with Content-Type: application/json. Use for session state, lookups, and cached responses.
• File (file: true): values persist as raw bytes byte-exact, stored with the real mime derived from the key (or application/octet-stream when unknown). set() accepts a Uint8Array or string; get() hands the exact bytes back. Use for serving assets such as walker.js.

One store instance is exactly one mode.

API

const file = await store.get('walker.js');           // StoreValue | undefined
await store.set('data.json', { ok: true });          // structured value
await store.set('walker.js', new Uint8Array([/*…*/])); // file mode: raw bytes
await store.delete('old-file.txt');                   // void

In file mode, get() returns the bytes as a Uint8Array leaf and set() rejects non-Uint8Array, non-string values with a clear error.