mirror of
https://github.com/shumaiOne/shumai.git
synced 2026-11-22 22:18:43 +00:00
118 lines
7.1 KiB
Plaintext
118 lines
7.1 KiB
Plaintext
---
|
|
title: Deploy with Distributed Workers
|
|
description: Run media transcoding on dedicated workers while agents run with the main server.
|
|
---
|
|
|
|
Shumai runs agent, embedding, and archive queues with the main web server. Media transcoding can run on dedicated workers through PostgreSQL queues managed by DBOS Transact.
|
|
|
|
```mermaid
|
|
graph TD
|
|
User[User] --> Server[Shumai Web UI / API + Agents + Archives]
|
|
Server --> DB[(PostgreSQL / DBOS Queues)]
|
|
DB --> Transcode[Dedicated Transcode Workers]
|
|
```
|
|
|
|
## Main Server
|
|
|
|
Set `SHUMAI_WORKERS=none` (or `--workers=none`) to run agents and archives with the main server and send transcoding jobs to dedicated workers. `all` and `transcode` also run the transcode queue in the server. The older `agent`, `ai`, and `archive` values remain accepted and run agents and archives.
|
|
|
|
```env
|
|
DATABASE_URL=postgresql://shumai:shumai_password@localhost:5432/shumai_db?schema=public
|
|
STORAGE_BACKEND=s3
|
|
AWS_ENDPOINT_URL_S3=https://your-s3-endpoint.com
|
|
S3_BUCKET=shumai
|
|
S3_REGION=us-east-1
|
|
S3_ACCESS_KEY_ID=your-access-key-id
|
|
S3_SECRET_ACCESS_KEY=your-secret-access-key
|
|
SHUMAI_WORKERS=none
|
|
BETTER_AUTH_SECRET=your-random-32-character-secret-key
|
|
```
|
|
|
|
Agent tools run on the server, so install its sandbox dependencies (`bubblewrap` on Linux or `sandbox-exec` on macOS). Standalone agent workers are no longer supported.
|
|
|
|
## Dedicated Transcode Workers
|
|
|
|
Configure workers with the same PostgreSQL database and S3 storage as the server. A shared S3-compatible backend is required when machines do not share a filesystem.
|
|
|
|
```bash
|
|
npm install -g @shumai-one/shumai-transcode
|
|
shumai-transcode
|
|
```
|
|
|
|
Or use the dedicated image:
|
|
|
|
```bash
|
|
docker run -d --name shumai-transcode-worker --env-file .env shumaione/shumai-transcode:latest
|
|
```
|
|
|
|
Bun users can run `bun run --bun shumai-transcode`.
|
|
|
|
## Upgrading Existing Agent Sessions
|
|
|
|
Stop and drain old agent workers before upgrading the server. The normal database migration copies existing session context into independent histories automatically. No separate migration command is required. Legacy records remain in the database during this transition.
|
|
|
|
Each session uses a local SQLite working copy, restored from PostgreSQL when missing. `AGENT_SESSION_CACHE_DIR` optionally selects the cache directory. Within each server process, history reads and the active agent share the same SQLite database. History can show locally committed entries before PostgreSQL persistence completes; these entries may disappear after recovery if persistence fails. The server mirrors compact session commits to PostgreSQL before acknowledging them. Keep the database in your backup policy; local SQLite copies are disposable. Image and skill placeholders remain compact in both copies and are resolved when the agent needs them.
|
|
|
|
---
|
|
|
|
## Worker Concurrency & Multi-Instance Deployments
|
|
|
|
### Concurrency Tuning
|
|
|
|
Workers pull tasks from PostgreSQL queues based on configurable concurrency limits:
|
|
|
|
- `CONCURRENCY_TRANSCODE`: Maximum concurrent video transcoding jobs per worker (defaults to `1`).
|
|
- `CONCURRENCY_AGENT`: Maximum concurrent agent and embedding workflows per main server (defaults to `5`, recommend setting to `1` when paired with a local `embedding-server`).
|
|
|
|
### Replica Identity
|
|
|
|
When running multiple replicas of the transcode workers or main servers across multiple machines or pods:
|
|
|
|
- `DbosExecutor` automatically derives a unique per-instance ID from the machine hostname and process ID to ensure startup recovery only re-enqueues tasks belonging to that specific instance.
|
|
- **Custom Worker ID**: Set `SHUMAI_WORKERS_ID` (or `SHUMAI_WORKER_ID`) if you wish to assign an explicit identifier.
|
|
- **Kubernetes**: In Kubernetes Pods, set `POD_NAME` (via Downward API `fieldRef: { fieldPath: metadata.name }`) to provide stable, unique per-pod identity across pod restarts.
|
|
|
|
---
|
|
|
|
## Scalable Deployment on Kubernetes & AWS EKS
|
|
|
|
For production cloud deployments requiring high-throughput media processing, Shumai can be deployed on Kubernetes or **AWS Elastic Kubernetes Service (AWS EKS)** with automated scale-to-zero queue-driven scaling via **KEDA (Kubernetes Event-driven Autoscaling)**.
|
|
|
|
```mermaid
|
|
graph TD
|
|
User([User / Web Browser]) -->|Port 80/3000| Server[Pod: shumai-server<br/>SHUMAI_WORKERS=agent]
|
|
Server -->|Enqueue to transcode_queue| DB[(PostgreSQL: pgvector18<br/>dbos.workflow_status)]
|
|
Server -->|Direct Uploads / Stream Proxies| Storage[(S3 / Garage Storage)]
|
|
|
|
KEDA[KEDA ScaledJob<br/>PostgreSQL Trigger<br/>min: 0, max: 10] -.->|Monitors Queue| DB
|
|
KEDA -->|Spawns Jobs for queue| Job1[Job: shumai-transcode-1<br/>Auto-drain worker]
|
|
KEDA -->|Spawns Jobs for queue| JobN[Job: shumai-transcode-N<br/>Auto-drain worker]
|
|
|
|
Job1 -->|Fetch Source / Store Proxies| Storage
|
|
JobN -->|Fetch Source / Store Proxies| Storage
|
|
Job1 -->|Mark Complete & Exit 0| DB
|
|
JobN -->|Mark Complete & Exit 0| DB
|
|
|
|
CA[Cluster Autoscaler / Karpenter] -.->|Scale-to-Zero On-Demand Nodes<br/>0 to 10 instances| JobN
|
|
```
|
|
|
|
### Key Architectural Highlights
|
|
|
|
- **Decoupled Stateless Workers**: The main web and API server, database, and storage run on reliable On-Demand compute (`c9g.large`, `SHUMAI_WORKERS=agent`), delegating media transcoding to background workers orchestrated via DBOS task queues in PostgreSQL.
|
|
- **Dedicated Graviton5 On-Demand Compute (`c9g.xlarge`)**: Media encoding runs on 4 vCPU / 8 GiB RAM AWS Graviton5 On-Demand instances (~$0.174/hr), utilizing ARM64 vector optimizations and high-speed DDR5 memory without involuntary Spot interruptions.
|
|
- **Queue-Driven Scale-to-Zero via KEDA**: KEDA monitors `dbos.workflow_status` for enqueued and pending transcode tasks and dynamically provisions Kubernetes `ScaledJob` resources (scaling 0 to 10). When the queue is empty, transcode nodes scale down to zero (`minSize: 0`), incurring **$0/hr** in worker costs.
|
|
- **Idle-Timeout Auto-Drain**: Worker pods continuously process queued tasks with concurrency `1`. When the queue is fully drained and a pod has been idle for 10 seconds (`SHUMAI_WORKER_IDLE_TIMEOUT_MS=10000`), it exits `0` cleanly so the node can scale down, processing batch uploads without cold-start pod churn.
|
|
- **Automated Stale Task Recovery**: The core web server runs an integrated recovery sweep that detects stale or orphaned tasks (e.g. from node crash) and resumes them automatically in DBOS, with a 3-attempt cap to safely handle poison-pill media.
|
|
|
|
### Complete Deployment Example & Source Code
|
|
|
|
For the full, production-ready manifests, declarative `eksctl` cluster configuration, and step-by-step walkthrough, see the **[EKS Deployment Example](https://github.com/shumaiOne/shumai/tree/main/eks)** in the Shumai repository:
|
|
|
|
👉 **[View the EKS Deployment Guide & Manifests on GitHub](https://github.com/shumaiOne/shumai/tree/main/eks)**
|
|
|
|
The repository includes:
|
|
|
|
- [`eks/eks-cluster.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/eks-cluster.yaml): Complete `eksctl` ClusterConfig with compute tiering.
|
|
- [`eks/kustomization.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/kustomization.yaml): Single-command Kustomize deployment for all services.
|
|
- Detailed instructions for setting up public endpoints, automated Cluster Autoscaler, and zero-downtime graceful shutdown.
|