> Part of the walkerOS documentation. Project overview and full index: # Understanding walkerOS Stores ## Overview[​](#overview "Direct link to Overview") Stores provide key-value storage that other components (sources, transformers, destinations) consume via environment injection. They are the 4th component type in Flow\.Json alongside sources, transformers, and destinations. **Core principle:** Stores are passive infrastructure. They don't process events or participate in chains — they provide state that other components read and write. ## Store interface[​](#store-interface "Direct link to Store interface") See [packages/core/src/types/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/store.ts) for the canonical interface. ### Instance[​](#instance "Direct link to Instance") | Property | Type | Purpose | Required | | --------- | ---------------------------------- | ---------------------------- | ------------ | | `type` | `string` | Store type identifier | **Required** | | `config` | `Store.Config` | Settings and env | **Required** | | `get` | `(key) => StoreValue \| undefined` | Read a value | **Required** | | `set` | `(key, value, ttl?) => void` | Write a value (optional TTL) | **Required** | | `delete` | `(key) => void` | Remove a value | **Required** | | `destroy` | `DestroyFn` | Cleanup on shutdown | Optional | All methods can be sync or async (return `Promise`). ### Value type and `file` mode[​](#value-type-and-file-mode "Direct link to value-type-and-file-mode") Stores hold one canonical value type: structured data (`StoreValue`), with binary (`Uint8Array`) as a first-class leaf. `StoreValue` is `string | number | boolean | null | Uint8Array | StoreValue[] | { [key: string]: StoreValue }` (`undefined` is reserved as the "miss" sentinel and is never a stored value). A shared core codec (`serializeStoreValue` / `deserializeStoreValue`) round-trips that value to and from each backing. `Store.Config.file?: boolean` (default `false`) picks the mode, decided once at init: * **Structured (default):** values are `StoreValue` data, serialized by the shared codec. * **File (`file: true`):** a byte-native backend (fs, S3, GCS) persists raw bytes byte-exact. `set()` accepts a `Uint8Array` or `string` and stores it untouched; `get()` hands the exact bytes back. Use for serving assets such as walker.js. The Sheets store is structured-only and rejects `file: true` at init. TTL is owned by the cache layer, not the store. The store persists values; a `cache` wrapper manages expiry. `flow_validate` warns when a store sets both `file: true` and `cache`, and when a `@walkeros/server-transformer-file` is wired to a byte-native store that does not set `file: true`. ### Init function (context pattern)[​](#init-function-context-pattern "Direct link to Init function (context pattern)") Stores use the same context-based init pattern as other components: ``` import type { Store } from '@walkeros/core'; export const storeMyStore: Store.Init = (context) => { const { config, env, logger, id } = context; const settings = config.settings || {}; return { type: 'my-store', config: context.config as Store.Config, get(key) { /* ... */ }, set(key, value, ttl) { /* ... */ }, delete(key) { /* ... */ }, destroy() { /* cleanup */ }, }; }; ``` **Context contains:** | Property | Type | Purpose | | ----------- | -------------------- | ------------------------- | | `config` | `Store.Config` | Settings from flow config | | `env` | `Store.Env` | Environment dependencies | | `logger` | `Logger.Instance` | Scoped logger | | `id` | `string` | Store identifier | | `collector` | `Collector.Instance` | Reference to collector | ## Lifecycle[​](#lifecycle "Direct link to Lifecycle") Stores have the simplest lifecycle of all component types: ``` Startup: Stores → Destinations → Transformers → Sources Shutdown: Sources → Destinations → Transformers → Stores ``` * **Init first:** Stores init before all other components so they're available when sources/transformers/destinations start * **Destroy last:** Stores destroy after all other components so transformers can flush during their own destroy * **No lazy init:** Unlike destinations, stores don't support `require` or deferred activation — they are always eager ## Wiring stores via `$store.`[​](#wiring-stores-via-store "Direct link to wiring-stores-via-store") ### Bundled mode (flow\.json)[​](#bundled-mode-flowjson "Direct link to Bundled mode (flow.json)") Use `$store.storeId` in a component's `env` to inject a store instance: ``` { "stores": { "data": { "package": "@walkeros/server-store-fs", "config": { "settings": { "basePath": "./data" } }, "cache": { "rules": [{ "ttl": 60 }] } } }, "transformers": { "fingerprint": { "package": "@walkeros/server-transformer-fingerprint", "env": { "store": "$store.data" } } } } ``` The bundler resolves `$store.data` to a runtime reference. Invalid references are caught at build time. `walkeros validate` also catches typos at validation time, including unknown store names and the colon-instead-of-dot mistake (e.g. `$store:data` is flagged with the suggested form `$store.data`). The `cache` field enables the built-in in-memory cache tier (`__cache`) on top of any backing store. No separate memory store declaration is needed. ### Integrated mode (TypeScript)[​](#integrated-mode-typescript "Direct link to Integrated mode (TypeScript)") Pass store instances directly — no `$store.` prefix needed: ``` import { startFlow } from '@walkeros/collector'; import { storeFsInit } from '@walkeros/server-store-fs'; import { transformerFingerprint } from '@walkeros/server-transformer-fingerprint'; const { collector } = await startFlow({ stores: { data: { code: storeFsInit, config: { settings: { basePath: './data' } }, cache: { rules: [{ ttl: 60 }] }, }, }, transformers: { fingerprint: { code: transformerFingerprint, env: { store: collector.stores.data }, // Direct reference }, }, }); ``` Note: In integrated mode, you wire the store instance directly in `env` rather than using the `$store.` string prefix (that's a bundler feature). ## Available stores[​](#available-stores "Direct link to Available stores") ### Built-in cache tier (`__cache`)[​](#built-in-cache-tier-__cache "Direct link to built-in-cache-tier-__cache") The collector ships a built-in in-memory cache with LRU eviction, TTL, and entry/byte caps. Enable it on any store by setting `Flow.Store.cache`. No separate package import needed: ``` { "stores": { "files": { "package": "@walkeros/server-store-fs", "cache": { "rules": [{ "ttl": 60 }] } } } } ``` Use this for the common "cache in front of a slow backing store" pattern (API, GCS, Sheets, etc.). ### `@walkeros/server-store-fs` (filesystem)[​](#walkerosserver-store-fs-filesystem "Direct link to walkerosserver-store-fs-filesystem") File-based store for serving static assets. Server-only. ``` import { storeFsInit } from '@walkeros/server-store-fs'; ``` ### `@walkeros/server-store-s3` (S3-compatible object storage)[​](#walkerosserver-store-s3-s3-compatible-object-storage "Direct link to walkerosserver-store-s3-s3-compatible-object-storage") S3-compatible store using `s3mini` (\~20 KB, zero dependencies). Works with AWS S3, Cloudflare R2, Scaleway, DigitalOcean Spaces, Backblaze B2, MinIO, and any S3-compatible provider. Structured by default (stored as `application/json`); set `file: true` to serve raw bytes byte-exact with the real mime. Server-only. ``` import { storeS3Init } from '@walkeros/server-store-s3'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | ----------------- | -------- | -------- | -------- | -------------------------- | | `bucket` | `string` | Yes | — | S3 bucket name | | `endpoint` | `string` | Yes | — | S3-compatible endpoint URL | | `accessKeyId` | `string` | Yes | — | S3 access key ID | | `secretAccessKey` | `string` | Yes | — | S3 secret access key | | `region` | `string` | No | `"auto"` | AWS region (SigV4 signing) | | `prefix` | `string` | No | — | Key prefix for scoping | **Primary use case:** Serving static files in managed deployments (Mode D) where files live in a bucket rather than being baked into a Docker image. ### `@walkeros/server-store-gcs` (Google Cloud Storage)[​](#walkerosserver-store-gcs-google-cloud-storage "Direct link to walkerosserver-store-gcs-google-cloud-storage") Zero-dependency GCS store using raw `fetch` + GCS JSON API. Built-in auth: ADC on Cloud Run / GKE, or explicit service account JWT. Server-only. ``` import { storeGcsInit } from '@walkeros/server-store-gcs'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | -------- | -------- | -------- | ------- | ---------------------- | | `bucket` | `string` | Yes | — | GCS bucket name | | `prefix` | `string` | No | — | Key prefix for scoping | Credentials live at `config.credentials` (sibling of `settings`): `string | object` SA JSON for non-GCP envs, `$env`-resolvable; omit for ADC. The deprecated `settings.credentials` still works. **Primary use case:** Serving static files on GCP infrastructure (Cloud Run, GKE) where ADC provides seamless authentication. ### `@walkeros/server-store-sheets` (Google Sheets)[​](#walkerosserver-store-sheets-google-sheets "Direct link to walkerosserver-store-sheets-google-sheets") Zero-dependency Google Sheets store using raw `fetch` + Sheets v4 REST API. One row per key, one cell per value (JSON-serialized). Built-in auth shared with the GCS store. Server-only. Structured-only: cells hold structured JSON, so it rejects `file: true` at init and rejects values carrying a binary (`Uint8Array`) leaf. Use fs, S3, or GCS for byte-exact serving. ``` import { storeSheetsInit } from '@walkeros/server-store-sheets'; ``` **Settings:** | Setting | Type | Required | Default | Purpose | | ------------ | -------- | -------- | ---------- | -------------------------------- | | `id` | `string` | Yes | — | Spreadsheet ID (segment in URL) | | `sheet` | `string` | No | `'Sheet1'` | Sheet (tab) name | | `key` | `string` | No | `'A'` | Column letter for keys | | `value` | `string` | No | `'B'` | Column letter for JSON values | | `headerRows` | `number` | No | `1` | Header rows to skip when reading | Credentials live at `config.credentials` (sibling of `settings`): `string | object` SA JSON for non-GCP envs, `$env`-resolvable; omit for ADC. The deprecated `settings.credentials` still works. **Primary use case:** Demos and small prototypes where the spreadsheet is the operator-facing UI for tweaking lookup data. Quota: 60 reads/min and 60 writes/min per project. Enable the built-in cache via `Flow.Store.cache` on the store declaration to absorb the quota, otherwise quota burns in seconds. **Not a production CRM substitute.** See [Website: Sheets Store](https://www.walkeros.io/docs/stores/server/sheets.md) for the cache-wiring example. ## Stores in flow config[​](#stores-in-flow-config "Direct link to Stores in flow config") Declare any store consumed by one or more components in the `stores` section of the flow config and wire it via `$store.` in component `env`. The built-in cache tier (`Flow.Store.cache`) covers the "fast in-memory cache in front of a slow backing store" pattern without a separate memory store. ## Accessing stores at runtime[​](#accessing-stores-at-runtime "Direct link to Accessing stores at runtime") After `startFlow()`, stores are available on the collector instance: ``` const { collector } = await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './data' } } }, }, }); // Read/write await collector.stores.files.set('key', 'value', 60000); // 60s TTL const value = await collector.stores.files.get('key'); await collector.stores.files.delete('key'); ``` ## Hooks[​](#hooks "Direct link to Hooks") Store operations (`get`, `set`, `delete`) are wrapped with `useHooks` during initialization, enabling pre/post interception via the collector's hooks system. **Available hook names:** | Hook name | Pre hook | Post hook | | ------------- | ---------------- | ----------------- | | `StoreGet` | `preStoreGet` | `postStoreGet` | | `StoreSet` | `preStoreSet` | `postStoreSet` | | `StoreDelete` | `preStoreDelete` | `postStoreDelete` | Hooks fire on every store operation regardless of which component triggered it (cache system, transformer via env, destination via env, direct access on `collector.stores`). ``` const { collector } = await startFlow({ stores: { files: { code: storeFsInit, config: { settings: { basePath: './data' } } }, }, }); // Intercept all store reads collector.hooks.preStoreGet = ({ fn }, key) => { console.log('Reading key:', key); return fn(key); }; ``` ## Key differences from other components[​](#key-differences-from-other-components "Direct link to Key differences from other components") | Aspect | Sources/Transformers/Destinations | Stores | | ------------- | --------------------------------- | ------------------------ | | Event flow | Participate in push chain | No push, no chain | | `next/before` | Chain connection fields | None (passive) | | Lifecycle | Init after stores | Init first, destroy last | | `require` | Deferred activation supported | Always eager | | Interface | `push(event, context)` | `get/set/delete(key)` | ## Dev exports (`dev.ts`)[​](#dev-exports-devts "Direct link to dev-exports-devts") Every store package must export schemas and examples via `dev.ts`, matching the convention used by sources and destinations: ### Required files[​](#required-files "Direct link to Required files") ``` src/ ├── schemas/ │ ├── settings.ts # Zod schema for store settings │ └── index.ts # Re-export + zodToSchema() conversion ├── examples/ │ └── index.ts # Example Store.Config objects └── dev.ts # export * as schemas; export * as examples ``` ### Schema pattern[​](#schema-pattern "Direct link to Schema pattern") ``` // src/schemas/settings.ts import { z } from '@walkeros/core/dev'; export const SettingsSchema = z.object({ mySetting: z.string().describe('Setting description for docs and MCP'), }); export type Settings = z.infer; // src/schemas/index.ts import { zodToSchema } from '@walkeros/core/dev'; import { SettingsSchema } from './settings'; export { SettingsSchema, type Settings } from './settings'; export const settings = zodToSchema(SettingsSchema); ``` ### tsup + package.json[​](#tsup--packagejson "Direct link to tsup + package.json") ``` // tsup.config.ts import { defineConfig, buildModules, buildDev } from '@walkeros/config/tsup'; export default defineConfig([buildModules(), buildDev()]); ``` Add `./dev` export to package.json: ``` "./dev": { "types": "./dist/dev.d.ts", "import": "./dist/dev.mjs", "require": "./dist/dev.js" } ``` ### Why this matters[​](#why-this-matters "Direct link to Why this matters") * `buildDev()` generates `dist/walkerOS.json` at build time * MCP tools fetch `walkerOS.json` from CDN for package discovery and schema validation * Website docs use `` instead of hardcoded markdown tables * Without `dev.ts`, a store is invisible to MCP and docs tables rot ### Hints (Optional)[​](#hints-optional "Direct link to Hints (Optional)") Stores can optionally export hints — lightweight, actionable context for AI agents beyond what schemas and examples already provide. Create `src/hints.ts`: ``` import type { Hint } from '@walkeros/core'; export const hints: Hint.Hints = { 'persistence-behavior': { text: 'Describes persistence guarantees. See settings schema for options.', }, }; ``` Export from `src/dev.ts` alongside schemas and examples: ``` export * as schemas from './schemas'; export * as examples from './examples'; export { hints } from './hints'; ``` Most stores don't need hints — only add them for non-obvious behaviors, prerequisites, or troubleshooting patterns. ## Setup (optional)[​](#setup-optional "Direct link to Setup (optional)") Stores can implement an optional `setup()` lifecycle to provision external resources, for example creating a SQLite table, initializing an S3 bucket, or running a one-off schema migration. Setup is **never** invoked by the runtime, push, init, or deploy. It runs only when an operator explicitly types `walkeros setup store.`. The signature is `(ctx: LifecycleContext, Env>) => Promise`, where `LifecycleContext` carries `{ id, config, env, logger }`. Idempotency is the package's responsibility: the framework adds no opinion. Use `resolveSetup(ctx.config.setup, DEFAULTS)` from `@walkeros/core` to normalize the `boolean | object` shape into a concrete options object. See [walkeros-create-destination](https://www.walkeros.io/skills/walkeros-create-destination.md), [walkeros-create-source](https://www.walkeros.io/skills/walkeros-create-source.md), [walkeros-understanding-destinations](https://www.walkeros.io/skills/walkeros-understanding-destinations.md), [walkeros-understanding-sources](https://www.walkeros.io/skills/walkeros-understanding-sources.md), and the `walkeros setup` CLI documentation for the authoring template and operator workflow. ## Related Skills[​](#related-skills "Direct link to Related Skills") * [walkeros-understanding-flow](https://www.walkeros.io/skills/walkeros-understanding-flow.md) - Architecture and Flow structure * [walkeros-understanding-transformers](https://www.walkeros.io/skills/walkeros-understanding-transformers.md) - Transformer interface and env pattern * [walkeros-using-store-cache](https://www.walkeros.io/skills/walkeros-using-store-cache.md) - Store-level cache and the declarative `state` block for fetch/stash without `$code:` * [walkeros-using-cli](https://www.walkeros.io/skills/walkeros-using-cli.md) - Bundling flows with stores **Source files:** * [packages/core/src/types/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/core/src/types/store.ts) - Store types * [packages/collector/src/store.ts](https://github.com/elbwalker/walkerOS/blob/main/packages/collector/src/store.ts) - initStores implementation * [packages/server/stores/fs/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/fs/src) - Filesystem store package * [packages/server/stores/s3/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/s3/src) - S3 store package * [packages/server/stores/gcs/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/gcs/src) - GCS store package * [packages/server/stores/sheets/src/](https://github.com/elbwalker/walkerOS/blob/main/packages/server/stores/sheets/src) - Google Sheets store package **Documentation:** * [Website: Flow](https://www.walkeros.io/docs/getting-started/modes/bundled.md) - Flow configuration with stores section * [Website: Stores](https://www.walkeros.io/docs/stores.md) - Stores overview * [Website: S3 Store](https://www.walkeros.io/docs/stores/server/s3.md) - S3 store documentation * [Website: FS Store](https://www.walkeros.io/docs/stores/server/fs.md) - Filesystem store documentation * [Website: GCS Store](https://www.walkeros.io/docs/stores/server/gcs.md) - GCS store documentation * [Website: Sheets Store](https://www.walkeros.io/docs/stores/server/sheets.md) - Google Sheets store documentation