Storage Backends

Every service in the platform defines a store interface that describes its data operations. The actual storage mechanism is selected at runtime via the SHIFT_STORAGE environment variable. This separation allows services to work identically across different backends without changing application code.

Backend Selection

SHIFT_STORAGE=convex    # Default -- uses Convex HTTP API
SHIFT_STORAGE=file      # JSON files in local dotfile directories
SHIFT_STORAGE=gateway   # Routes through the gateway API (used by CLI)

The createStore() factory function in each service reads SHIFT_STORAGE and returns the appropriate implementation.

Store Interface Pattern

Each service defines a ServiceStore interface in its packages/shared/src/store/ directory. This interface declares all data operations the service needs (CRUD, queries, etc.) without specifying how they are implemented.

Service (CLI / API / Web)
  --> ServiceStore interface (packages/shared/src/store/index.ts)
        --> FileStore      (store/file.ts)
        --> ConvexStore    (store/convex.ts)
        --> GatewayStore   (store/gateway.ts)

Per-Service Store Files

Each service follows a consistent file layout:

File	Purpose
store/index.ts	Store interface (the contract)
store/file.ts	FileStore -- reads/writes JSON files on disk
store/convex.ts	ConvexStore -- calls Convex HTTP API
store/gateway.ts	GatewayStore -- REST calls through the gateway
store/factory.ts	createStore() -- selects backend based on SHIFT_STORAGE

Convex Backend (Default)

When SHIFT_STORAGE=convex (or unset in API/gateway context), services use the Convex HTTP API for persistence. All services share a single Convex deployment with table prefixes to prevent collisions:

Prefix	Service
yp_	Yellow Pages
passport_	Passport
pulse_	Pulse
ledger_	Ledger
palette_	Palette
stage_	Stage
billing_	Billing
inference_	Inference
git_	Git
auth_	Gateway auth state
compute_	Compute

See Convex Backend for a deep dive on schema, functions, and deployment.

SID Bridging

The application layer uses nanoid(8) identifiers in the id field. Convex uses its own internal _id system. Every Convex table includes a sid (shift ID) field with an index, bridging the two ID systems transparently.

When a ConvexStore receives an app-level id, it queries the Convex table by the sid index to find the corresponding Convex document. When returning data, it maps the sid field back to id. This bridging is invisible to the service code above the store layer.

App layer:  { id: "a1b2c3d4", name: "my-service", ... }
                    |
                    v  (ConvexStore translates)
Convex:     { _id: "k17abc...", sid: "a1b2c3d4", name: "my-service", ... }

File Backend

When SHIFT_STORAGE=file, services store data as JSON files in dotfile directories within the project root. Each service has its own directory:

.yellowpages/
  services/
    <id>.json
  systems/
    <id>.json

.ledger/
  events/
    <id>.json
  policies/
    <id>.json

.passport/
  providers/
    <id>.json

This backend is designed to be git-native: files can be committed, diffed, and reviewed in pull requests. Cache files are gitignored.

The FileStore wraps synchronous read/write operations on these JSON files, implementing the same store interface as the ConvexStore.

Gateway Backend

When SHIFT_STORAGE=gateway (the default for the CLI), store operations are routed through the gateway's REST API. The GatewayStore translates store method calls into HTTP requests to the appropriate /api/v1/<service>/ endpoints.

This is the standard mode for the shift-cli binary, which communicates with the deployed platform at app.the-shift.dev.

Environment Variables

Variable	Purpose
SHIFT_STORAGE	Backend selection: convex, file, or gateway
CONVEX_URL	Convex Cloud deployment URL
CONVEX_SELF_HOSTED_URL	Self-hosted Convex URL (e.g., http://127.0.0.1:3210)
CONVEX_ADMIN_KEY	Admin key for self-hosted Convex
SHIFT_GATEWAY_URL	Gateway URL for SHIFT_STORAGE=gateway mode