Files
Yiling-J 4ac274cdd4 feat(runner): replace server-side bash sandbox with remote runner (#530)
* feat(runner): replace server-side bash sandbox with remote runner

* fix(runner): address review findings for output bounds, key rotation, defaults, and generated uploads

* chore(docker): remove sandbox privileges and env vars from remaining compose files

* feat(cli): add runner start with bundled SRT sandboxing (#535)

* refactor(agent): expose runner bash without skill gating

* feat(cli): add runner start with optional SRT sandboxing

* refactor(cli): remove runner Python dependency

* feat(cli): bundle SRT and use platform-native runner shells

* refactor(cli): delegate runner processes to Execa

* chore(cli): upgrade @anthropic-ai/sandbox-runtime to 0.0.83

* feat(cli): discover Bash for Windows runners

Use Pi-style Bash discovery and launch behavior, exercise the packaged CLI shell, and wait for pending ZIP metadata reads before closing cancelled downloads.

* fix(runner): enforce ownership and safe reconnects

Gate dispatch on reconciliation, retain cancellations across disconnects, and stop replaced clients. Guard the single-server runner topology with a database lease. Restage missing generated inputs, report signal exits, and bound generated-file cleanup.

* feat(webui): add Korean translations for agent runners

* feat(cli): add runner event logging and --verbose flag (#536)

* feat(cli): add runner event logging and --verbose flag

* fix(cli): log runner capacity rejections accurately and use monotonic duration
2026-11-18 15:58:57 +08:00

120 lines
7.4 KiB
Plaintext

---
title: Deploy with Distributed Workers
description: Run media transcoding on dedicated workers while agents run with the main server.
---
Shumai runs agent, embedding, and archive queues with the main web server. Media transcoding can run on dedicated workers through PostgreSQL queues managed by DBOS Transact.
```mermaid
graph TD
User[User] --> Server[Shumai Web UI / API + Agents + Archives]
Server --> DB[(PostgreSQL / DBOS Queues)]
DB --> Transcode[Dedicated Transcode Workers]
```
## Main Server
Set `SHUMAI_WORKERS=none` (or `--workers=none`) to run agents and archives with the main server and send transcoding jobs to dedicated workers. `all` and `transcode` also run the transcode queue in the server. The older `agent`, `ai`, and `archive` values remain accepted and run agents and archives.
```env
DATABASE_URL=postgresql://shumai:shumai_password@localhost:5432/shumai_db?schema=public
STORAGE_BACKEND=s3
AWS_ENDPOINT_URL_S3=https://your-s3-endpoint.com
S3_BUCKET=shumai
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key-id
S3_SECRET_ACCESS_KEY=your-secret-access-key
SHUMAI_WORKERS=none
BETTER_AUTH_SECRET=your-random-32-character-secret-key
```
Agent tools run on the server, so install its sandbox dependencies (`bubblewrap` on Linux or `sandbox-exec` on macOS). Standalone agent workers are no longer supported.
## Dedicated Transcode Workers
Configure workers with the same PostgreSQL database and S3 storage as the server. A shared S3-compatible backend is required when machines do not share a filesystem.
```bash
npm install -g @shumai-one/shumai-transcode
shumai-transcode
```
Or use the dedicated image:
```bash
docker run -d --name shumai-transcode-worker --env-file .env shumaione/shumai-transcode:latest
```
Bun users can run `bun run --bun shumai-transcode`.
## Upgrading Existing Agent Sessions
Stop and drain old agent workers before upgrading the server. The normal database migration copies existing session context into independent histories automatically. No separate migration command is required. Legacy records remain in the database during this transition.
Each session uses a local SQLite working copy, restored from PostgreSQL when missing. `AGENT_SESSION_CACHE_DIR` optionally selects the cache directory. Within each server process, history reads and the active agent share the same SQLite database. History can show locally committed entries before PostgreSQL persistence completes; these entries may disappear after recovery if persistence fails. The server mirrors compact session commits to PostgreSQL before acknowledging them. Keep the database in your backup policy; local SQLite copies are disposable. Image and skill placeholders remain compact in both copies and are resolved when the agent needs them.
---
## Worker Concurrency & Multi-Instance Deployments
### Concurrency Tuning
Workers pull tasks from PostgreSQL queues based on configurable concurrency limits:
- `CONCURRENCY_TRANSCODE`: Maximum concurrent video transcoding jobs per worker (defaults to `1`).
- `CONCURRENCY_AGENT`: Maximum concurrent agent and embedding workflows per main server (defaults to `5`, recommend setting to `1` when paired with a local `embedding-server`).
### Replica Identity
When running multiple replicas of the transcode workers or main servers across multiple machines or pods:
- `DbosExecutor` automatically derives a unique per-instance ID from the machine hostname and process ID to ensure startup recovery only re-enqueues tasks belonging to that specific instance.
- **Custom Worker ID**: Set `SHUMAI_WORKERS_ID` (or `SHUMAI_WORKER_ID`) if you wish to assign an explicit identifier.
- **Kubernetes**: In Kubernetes Pods, set `POD_NAME` (via Downward API `fieldRef: { fieldPath: metadata.name }`) to provide stable, unique per-pod identity across pod restarts.
Agent runners require a single main server per database. With runners configured, stop the old main server before starting its replacement. For Kubernetes, use a single main server replica with the Recreate deployment strategy. Transcode workers can still scale independently.
---
## Scalable Deployment on Kubernetes & AWS EKS
For production cloud deployments requiring high-throughput media processing, Shumai can be deployed on Kubernetes or **AWS Elastic Kubernetes Service (AWS EKS)** with automated scale-to-zero queue-driven scaling via **KEDA (Kubernetes Event-driven Autoscaling)**.
```mermaid
graph TD
User([User / Web Browser]) -->|Port 80/3000| Server[Pod: shumai-server<br/>SHUMAI_WORKERS=agent]
Server -->|Enqueue to transcode_queue| DB[(PostgreSQL: pgvector18<br/>dbos.workflow_status)]
Server -->|Direct Uploads / Stream Proxies| Storage[(S3 / Garage Storage)]
KEDA[KEDA ScaledJob<br/>PostgreSQL Trigger<br/>min: 0, max: 10] -.->|Monitors Queue| DB
KEDA -->|Spawns Jobs for queue| Job1[Job: shumai-transcode-1<br/>Auto-drain worker]
KEDA -->|Spawns Jobs for queue| JobN[Job: shumai-transcode-N<br/>Auto-drain worker]
Job1 -->|Fetch Source / Store Proxies| Storage
JobN -->|Fetch Source / Store Proxies| Storage
Job1 -->|Mark Complete & Exit 0| DB
JobN -->|Mark Complete & Exit 0| DB
CA[Cluster Autoscaler / Karpenter] -.->|Scale-to-Zero On-Demand Nodes<br/>0 to 10 instances| JobN
```
### Key Architectural Highlights
- **Decoupled Stateless Workers**: The main web and API server, database, and storage run on reliable On-Demand compute (`c9g.large`, `SHUMAI_WORKERS=agent`), delegating media transcoding to background workers orchestrated via DBOS task queues in PostgreSQL.
- **Dedicated Graviton5 On-Demand Compute (`c9g.xlarge`)**: Media encoding runs on 4 vCPU / 8 GiB RAM AWS Graviton5 On-Demand instances (~$0.174/hr), utilizing ARM64 vector optimizations and high-speed DDR5 memory without involuntary Spot interruptions.
- **Queue-Driven Scale-to-Zero via KEDA**: KEDA monitors `dbos.workflow_status` for enqueued and pending transcode tasks and dynamically provisions Kubernetes `ScaledJob` resources (scaling 0 to 10). When the queue is empty, transcode nodes scale down to zero (`minSize: 0`), incurring **$0/hr** in worker costs.
- **Idle-Timeout Auto-Drain**: Worker pods continuously process queued tasks with concurrency `1`. When the queue is fully drained and a pod has been idle for 10 seconds (`SHUMAI_WORKER_IDLE_TIMEOUT_MS=10000`), it exits `0` cleanly so the node can scale down, processing batch uploads without cold-start pod churn.
- **Automated Stale Task Recovery**: The core web server runs an integrated recovery sweep that detects stale or orphaned tasks (e.g. from node crash) and resumes them automatically in DBOS, with a 3-attempt cap to safely handle poison-pill media.
### Complete Deployment Example & Source Code
For the full, production-ready manifests, declarative `eksctl` cluster configuration, and step-by-step walkthrough, see the **[EKS Deployment Example](https://github.com/shumaiOne/shumai/tree/main/eks)** in the Shumai repository:
👉 **[View the EKS Deployment Guide & Manifests on GitHub](https://github.com/shumaiOne/shumai/tree/main/eks)**
The repository includes:
- [`eks/eks-cluster.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/eks-cluster.yaml): Complete `eksctl` ClusterConfig with compute tiering.
- [`eks/kustomization.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/kustomization.yaml): Single-command Kustomize deployment for all services.
- Detailed instructions for setting up public endpoints, automated Cluster Autoscaler, and zero-downtime graceful shutdown.