Compare commits

..
13 Commits
Author SHA1 Message Date
yiling c2af7494ec Release v0.6.1 2026-10-24 13:21:50 +08:00
yiling b7d2748b00 docs(changelog): add bun 1.4.4 upgrade to unreleased changes 2026-10-24 12:02:22 +08:00
Yiling-J 3467c0d92b chore(docker): update Aliyun compose images and document transcode worker scaling in README_CN (#491) 2026-10-24 11:51:37 +08:00
Yiling-J 22fede0290 fix(webui): prevent clipping of long text overlay and date picker in cards (#490)
* fix(webui): prevent clipping of long text overlay and date picker in cards

* docs: update DeepWiki badge url in README and README_CN

* fix(webui): preserve expanded overlay height and textarea style in LongTextField

* fix(webui): add bottom padding to file list to prevent screen clipping of expanded fields

* fix(webui): expand directly into edit mode on single click for LongTextField

* fix(webui): prevent file selection when interacting with metadata fields in cards and list rows
2026-10-24 11:40:47 +08:00
Yiling-J d87fe43b0f feat(dashboard): add workflow tasks monitoring and stopping to team dashboard (#489)
* feat(dashboard): add workflow tasks monitoring and stopping to team dashboard

* fix(asset): retain workflow tasks on asset purge and mark active tasks as failed

* style(webui): add edge padding to audit logs and workflow tasks tables

* fix(workflow): atomically update asset status to failed when task is stopped

* fix(workflow): propagate cancellation errors, guard concurrent task stop, and localize UI

* docs(changelog): keep only added log for workflow tasks dashboard

* test(provider): mock upstream catalog api calls using saved fixtures

* feat(dashboard): show file name, size, and project name in workflow tasks list
2026-10-23 16:44:52 +08:00
Yiling-J 7a0ffca629 Add feedback invitation to README
Added a feedback section to encourage user engagement.
2026-10-22 20:39:56 +08:00
Yiling-J 5cf6d31794 chore(deps): upgrade bun to 1.4.4 and migrate to bun fmt (#488)
- Upgrade Bun to 1.4.4 across Dockerfiles, CI workflows, and release configurations.
- Update @types/bun and bun-types.
- Remove eslint-plugin-prettier and update format script to use native bun fmt.
- Configure .prettierignore to exclude packages/webui/public and minified files.
2026-10-22 20:25:54 +08:00
Yiling-J 0d5fe48a3a docs(docker): add guide for scaling transcoding workers horizontally (#486) 2026-10-21 12:45:09 +08:00
Yiling-J 19eed521ae fix(webui): prevent unwanted redirects to login on reset-password page (#487)
* fix(webui): allow reset-password route with token when stale auth exists

* fix(webui): prevent 401 redirect to login on reset-password page
2026-10-21 12:12:25 +08:00
yiling e05a23bcc2 Release v0.6.0 2026-10-20 20:02:34 +08:00
Yiling-J d957ab2339 fix(webui): fix loading skeleton layout mismatches and keep left carousel visible (#485)
* fix(webui): fix loading skeleton layout mismatches and keep left carousel visible

* fix(webui): update mobile file list skeleton to match card layout

* fix(webui): blur video poster before playback with instant cut on play

* docs: consolidate webui changelog entries into single line
2026-10-20 17:39:40 +08:00
Yiling-J 1fcd7245ea fix(transcode): ensure durable file exchange and crash recovery across workflow steps (#484)
* fix(transcode): ensure durable file exchange and crash recovery across workflow steps

* fix(transcode): recognize AWS SDK NoSuchKey by name and add workflow E2E recovery test
2026-10-20 11:29:22 +08:00
Yiling-J 176b7f0eab feat(workflow): migrate workflow engine from temporal to dbos (#481)
* feat(workflow): migrate workflow engine from temporal to dbos

Replace Temporal and LocalExecutor with DBOS Transact (@dbos-inc/dbos-sdk) as the unified workflow engine. Background tasks, media transcoding, and AI agent workflows now execute with durable steps and task queues directly against PostgreSQL, eliminating the need for an external Temporal cluster and separate polling executors.

* fix(workflow): submit tasks via DBOSClient to allow enqueueing from steps

Enqueuing workflows using DBOS.enqueueWorkflowWithOptions threw a
DBOSInvalidWorkflowTransitionError when triggered from within a running
step or activity (e.g. creating embedding or autofill tasks during
video transcoding).

Use DBOSClient to submit and cancel workflow tasks directly against the
system database, detaching execution from the ambient workflow/step
context.

* feat(web): support SHUMAI_WORKERS environment variable

* fix(workflow): ensure CLI tasks are enqueued to DBOS and run DBOS migrations in global setup

- Initialize workflows and wait for pending DBOS submissions in shumai regenerate-embeddings CLI
- Remove no-op submit bypass and enqueue tasks via DBOSClient in producer and CLI contexts
- Add DBOS migration runner in test global setup to avoid concurrent migration deadlocks across test forks
- Update changelog and add regression tests

* fix(workflow): ensure unique per-instance executor identity for worker replicas

- Derive unique per-instance DBOS executor IDs across worker replicas

- Support SHUMAI_WORKER_ID, DBOS__VMID, and POD_NAME (Kubernetes Downward API)

- Disambiguate multi-replica bare metal/container deployments with host, PID, and instance suffix

- Prevent startup recovery from re-enqueueing workflows actively executed by other live workers

- Add multi-instance startup recovery tests and update deployment documentation

* fix(workflow): only register listened queues and update concurrency on change

- Only register queues that the executor instance actively listens to

- Use always_update conflict resolution so environment variable changes take effect on restart

- Prevent dedicated workers from overwriting unmanaged queue concurrency configurations

- Add tests verifying concurrency updates and queue isolation across worker roles

* refactor(workflow): simplify activities to use defineStep and direct calls (#482)

* refactor(workflow): simplify activities to use defineStep and direct calls

- Wrap activities using defineStep from @shumai/workflow-core
- Remove obsolete getActivities, registerActivities, dynamic activity proxies, and workerQueue arguments
- Remove executor.ts and registry.ts, consolidating workflow registration into workflow.ts
- Call activities directly across transcode and agent workflows
- Update all corresponding workflow tests and developer guide documentation

* fix(workflow): properly detect DBOSWorkflowCancelledError in runOnCancel fallback

- DBOSWorkflowCancelledError in DBOS SDK does not set name, inheriting 'Error'
- Implement isDbosWorkflowCancelledError checking instanceof, constructor.name, and dbosErrorCode
- Update unit tests to verify real DBOS SDK cancellation error instances

* docs(workflow): update documentation and readme for dbos migration

- Rename docs/configuration/temporal.mdx to docs/configuration/distributed.mdx
- Document DBOS-based distributed worker architecture and task queues
- Remove obsolete docker-compose/temporal/
- Update environment variables documentation in docs/configuration/env-variables.mdx
- Update references in README.md, README_CN.md, docs/docs.json, and related guides

* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding (#483)

* feat(deploy): add shumai-transcode docker image and k8s autoscaling example with garage s3

* docs(k8s): document port-forwarding and in-cluster s3 domain mapping

* feat(storage): support public s3 endpoint for browser uploads and update k8s example

* fix(k8s): configure s3 bucket cors rules in garage initialization

* fix(k8s): increase transcode worker memory limit to 4Gi for 4K video encoding

* feat(deploy): add AWS EKS deployment with Graviton3 Spot nodegroups and autoscaling

* docs(deploy): redesign EKS documentation, rename k8s to eks, and add scalable deployment feature

* docs: rephrase DBOS feature bullet to Durable Workflows via DBOS

* docs(eks): upgrade default instances to Graviton5 c9g.large and c9g.xlarge

* feat(eks): use On-Demand for general nodegroup and add spot capacity diversification

* docs(eks): fix Architecture diagram to locate all transcode worker pods in Transcode Compute Pool

* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding

* feat(transcode): support idle-timeout auto-drain for KEDA scaled jobs

* fix(deploy): fix EKS deployment manifests and documentation based on cluster verification
2026-10-19 15:35:08 +08:00
188 changed files with 13391 additions and 9128 deletions
+4 -4
View File
@@ -24,7 +24,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.2
bun-version: 1.4.4
- name: Install dependencies
run: bun install
@@ -81,7 +81,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.2
bun-version: 1.4.4
- name: Install dependencies
run: bun install
@@ -126,7 +126,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.2
bun-version: 1.4.4
- name: Install dependencies
run: bun install
@@ -163,7 +163,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.2
bun-version: 1.4.4
- name: Install dependencies
run: bun install
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.2
bun-version: 1.4.4
- name: Setup Node.js
uses: actions/setup-node@v6
+28
View File
@@ -74,3 +74,31 @@ jobs:
provenance: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# Extract metadata for Docker (Transcode Worker)
- name: Extract metadata for Docker (Transcode Worker)
id: meta-transcode
uses: docker/metadata-action@v5
with:
images: |
shumaione/shumai-transcode
crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai-transcode
tags: |
type=semver,pattern={{version}}
type=ref,event=branch,enable=${{ github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && !inputs.pr && !inputs.tag) }}
type=raw,value=pr-${{ inputs.pr }},enable=${{ github.event_name == 'workflow_dispatch' && inputs.pr != '' && inputs.tag == '' }}
type=raw,value=${{ inputs.tag }},enable=${{ github.event_name == 'workflow_dispatch' && inputs.tag != '' }}
labels: |
org.opencontainers.image.revision=${{ steps.checkout.outputs.commit }}
# Build and push Docker image (Transcode Worker)
- name: Build and push Docker image (Transcode Worker)
uses: docker/build-push-action@v7
with:
platforms: linux/amd64,linux/arm64
context: .
file: ./Dockerfile.transcode
push: true
provenance: false
tags: ${{ steps.meta-transcode.outputs.tags }}
labels: ${{ steps.meta-transcode.outputs.labels }}
+2
View File
@@ -25,3 +25,5 @@ code_quality.md
# Paraglide JS
packages/webui/paraglide
runner-package.json
+3
View File
@@ -9,3 +9,6 @@ packages/webui/paraglide
*.md
**/*.md
docs/changelog.mdx
packages/webui/public
**/*.min.*
+41 -30
View File
@@ -159,7 +159,7 @@ The project is a monorepo managed by **Bun Workspaces**. It follows a strictly d
We follow Bun's monorepo conventions for dependency management:
1. **Self-Contained Packages**: Every workspace package MUST declare its own runtime `dependencies` in its local `package.json`. Do not rely on dependencies being available via the root.
2. **Shared DevDependencies**: Common development tools (e.g., `typescript`, `eslint`, `vitest`, `prettier`, `prisma`) MUST be declared in the root `package.json` to ensure version consistency across the workspace.
2. **Shared DevDependencies**: Common development tools (e.g., `typescript`, `eslint`, `vitest`, `prisma`) MUST be declared in the root `package.json` to ensure version consistency across the workspace.
3. **Local DevDependencies**: Tools specific to a single package (e.g., `@vitejs/plugin-react` for `webui`) should be declared in that package's local `package.json`.
4. **Workspace Imports**: Use `workspace:*` for internal package dependencies.
5. **Clean Root**: The root `package.json` must not contain runtime `dependencies`. It is reserved for shared `devDependencies` and workspace-wide `scripts`.
@@ -299,7 +299,7 @@ describe('Team API', () => {
### Workflow E2E Tests
- Located in `packages/e2e/workflow/**/*.test.ts`.
- Run via `bun run test:e2e:workflow` (which executes tests against both `local` and `temporal` executors dynamically).
- Run via `bun run test:e2e:workflow`.
- **Mandatory test coverage**: When creating or updating workflows or activities, you MUST create or update the corresponding E2E tests.
- **Mocking**: Mock ONLY AI API calls. Do not mock S3, databases, or media/transcode extraction services.
- Mock the AI response by spying on `AgentHarness.prototype.prompt`:
@@ -377,64 +377,75 @@ model Example {
- The default `limit` should be **20**.
- Always use the `paginateQuery` helper function from `packages/core/src/pagination.ts` for consistent cursor pagination.
## Temporal Workflow & Activities
## DBOS Workflow & Activities
We use a custom workflow engine that supports both **Local** (polling-based) and **Temporal** (production-grade) execution.
We use **DBOS Transact** (`@dbos-inc/dbos-sdk`) as our unified workflow engine for reliable, distributed background execution.
### Architecture
1. **Workflows**: Orchestrate multiple activities. They must be deterministic and compatible with Temporal's V8 isolate. Defined in worker packages (e.g., `@shumai/agent/src/workflows`, `@shumai/transcode/src/workflows`).
2. **Activities**: Perform the actual work (DB updates, API calls, media processing). Grouped by domain in worker packages (e.g., `@shumai/agent/src/activities`, `@shumai/transcode/src/activities`).
1. **Workflows**: Orchestrate multiple steps/activities reliably. Defined in worker packages (e.g., `@shumai/agent/src/workflows`, `@shumai/transcode/src/workflows`).
2. **Activities (Steps)**: Perform the actual work (DB updates, API calls, media processing). Grouped by domain in worker packages (e.g., `@shumai/agent/src/activities`, `@shumai/transcode/src/activities`).
3. **Executors**:
- `LocalExecutor` (in `@shumai/workflow-core`): Polls the database for `pending` tasks and executes them directly.
- `TemporalExecutor` (in `@shumai/workflow-core`): Submits tasks to a Temporal cluster.
4. **Automatic Submission**: New `WorkflowTask` records are automatically submitted to the `WorkflowService` via a Prisma Client Extension defined in `@shumai/db`.
- `DbosExecutor` (in `@shumai/workflow-core`): Coordinates workflow and step dispatching via DBOS queues against PostgreSQL.
4. **Automatic Submission**: New `WorkflowTask` records are automatically enqueued to `WorkflowService` via a Prisma Client Extension defined in `@shumai/db`.
### Dynamic Registration
To avoid circular dependencies, workflows and activities are dynamically registered onto the `LocalExecutor` registry at bootstrap time via initializer functions (e.g., `initAgentWorkflows()`, `initTranscodeWorkflows()`).
To avoid circular dependencies, workflows are dynamically registered onto the workflow registry at bootstrap time via initializer functions (e.g., `initAgentWorkflows()`, `initTranscodeWorkflows()`).
### Non-Retryable Error Handling
To prevent Temporal from indefinitely retrying fatal, expected business validation failures (e.g., missing records, invalid configurations, missing parameters), **never throw standard `Error` objects from workflow or activity logic.** Instead, throw a non-retryable `ApplicationFailure`.
To prevent the engine from indefinitely retrying fatal, expected business validation failures (e.g., missing records, invalid configurations, missing parameters), **never throw standard `Error` objects from workflow or activity logic.** Instead, throw a non-retryable `ApplicationFailure` (alias of `WorkflowFailure`).
**Mandatory Requirement**: You **MUST** set `nonRetryable: true` in the options object passed to `ApplicationFailure.create`. Standard errors or `ApplicationFailure` objects without `nonRetryable: true` will cause Temporal to retry the activity or workflow indefinitely.
**Mandatory Requirement**: You **MUST** set `nonRetryable: true` in the options object passed to `ApplicationFailure.create`. Standard errors or failure objects without `nonRetryable: true` will cause DBOS to retry the activity or workflow according to its retry policy.
- **Workflow Boundary**: In workflows, import `ApplicationFailure` from `@temporalio/workflow`:
- **Import**: Import `ApplicationFailure` from `@shumai/workflow-core`:
```typescript
import { ApplicationFailure } from '@temporalio/workflow'
import { ApplicationFailure } from '@shumai/workflow-core'
throw ApplicationFailure.create({ message: 'agentId missing in payload', nonRetryable: true })
```
- **Activity Boundary**: In activities, import `ApplicationFailure` from `@temporalio/activity`:
```typescript
import { ApplicationFailure } from '@temporalio/activity'
throw ApplicationFailure.create({
message: 'no autofill agent found for team',
nonRetryable: true,
})
```
### Temporal Workflow Patterns
### Workflow Patterns
- **Definition**: Define workflows as exported async functions.
```typescript
import { activityA, activityB } from '../activities/my-activities'
export async function myWorkflow(task: WorkflowTask): Promise<void> {
const { activityA, activityB } = getActivities()
await activityA({ ... })
await activityB({ ... })
}
```
- **Environment Compatibility**: Always use `getActivities()` from `@shumai/workflow-core` to ensure the code works in both Local and Temporal environments.
- **Activity Access**: Activities should be accessed via `getActivities()` within a workflow. Do not call services directly inside a workflow function to maintain Temporal compatibility.
- **Queue Redirection**: Production Temporal workers use worker-specific unique queues for data locality (e.g., local disk access) and consistent state.
- Every workflow MUST first call `getAgentWorkerQueueActivity` or `getTranscodeWorkerQueueActivity` via the shared domain queue (e.g., `TaskQueueAgent` or `TaskQueueTranscode`) to obtain the specific queue name for that worker instance.
- ALL subsequent activities (including shared DB activities like updating task status) MUST be executed on that specific discovered queue.
- The shared domain worker ONLY registers the queue discovery activities. All other activities (shared and domain-specific) are registered on the specific worker.
- **Activity Invocation**: Workflows import and call activities directly as standard async functions.
- **Queues**: Workflows are dispatched to dedicated task queues (e.g. `TaskQueueTranscode`, `TaskQueueAgent`) managed by DBOS.
### Activity Patterns
- **Definition**: Wrap exported activity functions with `defineStep` from `@shumai/workflow-core`.
```typescript
export const myActivity = defineStep(
async function myActivity(params: MyActivityParams): Promise<MyActivityResult> {
...
},
{ runOnCancel: false }, // optional: set runOnCancel: true for cleanup/status handlers
)
```
- **Implementation**: Activities _can_ and _should_ call other services (e.g., `aiService`, `transcodeService`) or interact with the database directly. Database (Prisma) queries or updates are allowed in all activity types (e.g., agent or transcode) for simplicity.
### Worker Identity & Multi-Instance Deployments
In multi-replica deployments (e.g., multiple transcode or agent worker instances), each worker instance MUST have a unique executor ID to prevent startup recovery from re-enqueueing workflows being actively processed by other live workers.
- **Identity Derivation**: `DbosExecutor` automatically derives a unique per-instance ID:
- If `SHUMAI_WORKER_ID` or `DBOS__VMID` is set, it is used as the instance identifier.
- In Kubernetes, set `POD_NAME` (via Downward API: `fieldRef: { fieldPath: metadata.name }`) to provide stable, per-pod identity across StatefulSet restarts.
- In Docker/container environments, `HOSTNAME` (container ID) is automatically used.
- On bare metal/multi-process hosts, `process.pid` and an instance suffix are appended to guarantee uniqueness.
### Durable Step File Exchange
Intermediate files passed across steps must be saved to durable storage (S3) by the producing step, consuming steps must use `ensureLocalMediaFile` (or S3 fetch) rather than assuming local `/tmp` presence, and any purely temporary S3 scratch files must be cleaned up in a `finally` block.
## Import conventions
- When importing across workspace packages, always use absolute workspace imports (e.g. `import { prisma } from '@shumai/db'`, `import { teamService } from '@shumai/core/src/team/team'`).
+30
View File
@@ -12,6 +12,36 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Changed
## [0.6.1] - 2026-10-24
### Added
* **dashboard**: Add Workflow Tasks section to the Team Dashboard allowing team owners to monitor background workflow jobs, inspect execution payloads and outputs, and stop in-flight tasks
### Fixed
* **metadata**: Fix an issue where clicking to edit a metadata field on a card or list row inadvertently selected the file, and prevent expanded text overlays or date pickers from being clipped by container borders
### Changed
* **deps**: Upgrade Bun to 1.4.4
## [0.6.0] - 2026-10-20
### Added
* **deploy**: Add dedicated `shumaione/shumai-transcode` Docker image and a complete AWS EKS deployment example (`eks/`) featuring KEDA queue-driven autoscaling, idle-timeout auto-drain worker mode, and Scale-to-Zero On-Demand Graviton5 instances for media processing
* **storage**: Support configuring a separate public endpoint (`AWS_PUBLIC_ENDPOINT_URL_S3`) for browser-facing presigned URLs while preserving internal cluster endpoints for backend workers
### Fixed
* **transcode**: Fix an issue where PDF, 3D model, and image background transcode workflows could fail with missing temporary file errors if a worker restarted or recovered midway through processing
* **webui**: Improve file list and detail loading with matching skeleton layouts, a persistent sidebar carousel to eliminate layout shift, and smoothed video poster previews
### Changed
* **workflow**: Migrate the background workflow engine to DBOS, unifying all-in-one and dedicated worker deployments using the existing PostgreSQL database without requiring a separate Temporal cluster
## [0.5.3] - 2026-10-16
### Added
+6 -4
View File
@@ -1,6 +1,6 @@
#syntax=docker/dockerfile:1.7-labs
# Stage 1: Builder
FROM oven/bun:1.4.2 AS builder
FROM oven/bun:1.4.4 AS builder
WORKDIR /app
@@ -28,7 +28,7 @@ RUN bun run build.ts
RUN bun run scripts/extract-runtime-deps.ts
# Stage 2: Runner
FROM oven/bun:1.4.2 AS runner
FROM oven/bun:1.4.4 AS runner
# Install system dependencies and Jellyfin-FFmpeg (with VA-API support)
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -94,9 +94,11 @@ USER root
COPY scripts/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# Expose "bun index.js" as "shumai" CLI command (and alias "shuami")
# Expose "bun index.js" as "shumai" CLI command (and worker commands "shumai-transcode", "shumai-agent")
RUN printf '#!/bin/sh\nexec bun /app/index.js "$@"\n' > /usr/local/bin/shumai \
&& chmod +x /usr/local/bin/shumai \
&& printf '#!/bin/sh\nexec bun /app/transcode-worker.js "$@"\n' > /usr/local/bin/shumai-transcode \
&& printf '#!/bin/sh\nexec bun /app/agent-worker.js "$@"\n' > /usr/local/bin/shumai-agent \
&& chmod +x /usr/local/bin/shumai /usr/local/bin/shumai-transcode /usr/local/bin/shumai-agent \
&& ln -s /usr/local/bin/shumai /usr/local/bin/shuami
ENTRYPOINT ["docker-entrypoint.sh"]
+80
View File
@@ -0,0 +1,80 @@
#syntax=docker/dockerfile:1.7-labs
# Stage 1: Builder
FROM oven/bun:1.4.4 AS builder
WORKDIR /app
# Copy package files
COPY package.json bun.lock ./
COPY --parents packages/*/package.json ./
COPY --parents apps/*/package.json ./
COPY patches/ ./patches/
# Install dependencies (including devDependencies for build tools)
RUN bun install --frozen-lockfile
# Copy source code
COPY . .
# Build transcode worker bundle
RUN bun run build.ts transcode
# Generate runner-package.json for the runner stage
RUN bun run scripts/extract-runtime-deps.ts
# Stage 2: Runner
FROM oven/bun:1.4.4 AS runner
# Install transcoding system dependencies and Jellyfin-FFmpeg (with VA-API/hardware acceleration support)
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
ca-certificates \
gosu \
poppler-utils \
imagemagick \
fonts-noto-cjk \
libraw-bin \
libimage-exiftool-perl \
&& . /etc/os-release \
&& TARGETARCH=$(dpkg --print-architecture) \
&& FFMPEG_VERSION="8.1.2-5" \
&& curl -fsSL -o /tmp/jellyfin-ffmpeg.deb "https://github.com/jellyfin/jellyfin-ffmpeg/releases/download/v${FFMPEG_VERSION}/jellyfin-ffmpeg8_${FFMPEG_VERSION}-${VERSION_CODENAME}_${TARGETARCH}.deb" \
&& apt-get install -y --no-install-recommends /tmp/jellyfin-ffmpeg.deb \
&& rm -f /tmp/jellyfin-ffmpeg.deb \
&& ldconfig /usr/lib/jellyfin-ffmpeg/lib \
&& ln -sf /usr/lib/jellyfin-ffmpeg/ffmpeg /usr/local/bin/ffmpeg \
&& ln -sf /usr/lib/jellyfin-ffmpeg/ffprobe /usr/local/bin/ffprobe \
&& ln -sf /usr/lib/jellyfin-ffmpeg/ffmpeg /usr/bin/ffmpeg \
&& ln -sf /usr/lib/jellyfin-ffmpeg/ffprobe /usr/bin/ffprobe \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Set up working directory and make it owned by the bun user, ensure video/render group access
RUN usermod -a -G video,render bun 2>/dev/null || true \
&& mkdir -p /app/data \
&& chown -R bun:bun /app
# Switch to the non-root bun user
USER bun
WORKDIR /app
# Install runtime dependencies with exact matching versions extracted from the builder stage
COPY --chown=bun:bun --from=builder /app/runner-package.json ./package.json
RUN bun install
# Copy build artifacts
COPY --chown=bun:bun --from=builder /app/dist ./
# Switch back to root at the end so the container runs the entrypoint as root
USER root
COPY scripts/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# Expose "bun transcode-worker.js" as "shumai-transcode" CLI command
RUN printf '#!/bin/sh\nexec bun /app/transcode-worker.js "$@"\n' > /usr/local/bin/shumai-transcode \
&& chmod +x /usr/local/bin/shumai-transcode
ENTRYPOINT ["docker-entrypoint.sh"]
# Default command: start transcode worker
CMD ["shumai-transcode"]
+39 -4
View File
@@ -12,7 +12,7 @@
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
<a target="_blank" href="https://deepwiki.com/shumaiOne/shumai"><img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg" /></a>
<a target="_blank" href="https://deepwiki.com/shumaiOne/shumai"><img alt="Ask DeepWiki" src="https://img.shields.io/badge/Ask-DeepWiki-6E56CF" /></a>
</p>
<p align="center">
@@ -29,7 +29,8 @@ Shumai is an open-source Frame.io alternative built for creative teams to collab
* **Frame-by-Frame Annotations & Comments**: Leave precise feedback with frame-specific drawing tools and timestamped comments on both video and image assets.
* **Secure Sharing & Collections**: Create private share links and curated media collections for clients, stakeholders, and collaborators.
* **Granular Access Control**: Manage workspace permissions with team-level and project-level role-based access control.
* **Distributed Transcoding via Temporal**: Offload resource-intensive video transcoding to a background worker pool orchestrated by Temporal.
* **Durable Workflows via DBOS**: Run background tasks and AI agent workflows orchestrated reliably by DBOS on PostgreSQL with automatic recovery and fault tolerance.
* **Scalable Deployment on Kubernetes**: Horizontally autoscale video transcoding on multiple instances. See [EKS Deployment Example](eks/README.md).
* **Custom Asset Metadata**: Define dynamic metadata fields tailored to your production pipeline.
* **Share Link Watermarking**: Protect shared media with customizable text and image watermarks, and save watermark as reusable templates.
* **Wide File Format Support**: Preview common creative file formats, including images (with PSD support), videos, audio, and documents, while allowing any file type to be uploaded and stored.
@@ -69,11 +70,13 @@ The ultimate goal is an **agentic Kanban**, where agents and team members can wo
> * **For Images:** Shumai generates a high-quality WebP proxy at the exact same resolution as your original file for optimized viewing.
> * **For Videos:** It generates at least one proxy video (depending on your transcoding settings), alongside a sprite image and a small preview video (capped at 1600 frames) to enable fast timeline scrubbing and lightning-quick previews.
---
Have feedback or ideas? [Share them with us](https://github.com/shumaiOne/shumai/issues/328), and if you find Shumai useful, please give us a ⭐ or recommend Shumai to someone who might find it useful, it really helps!
## Installation
Below is a quickstart guide for running Shumai with local storage. For advanced configuration options (including S3-compatible storage and Temporal workflow orchestration), see our [Documentation](https://docs.shumai.one/getting-started/overview).
Below is a quickstart guide for running Shumai with local storage. For advanced configuration options (including S3-compatible storage and distributed worker orchestration), see our [Documentation](https://docs.shumai.one/getting-started/overview).
### Option 1: Docker Compose
@@ -141,6 +144,38 @@ Docker Compose is the fastest way to get Shumai running. You do not need to clon
```
6. Open your browser and access Shumai at `http://localhost:3000` (or `http://<your-server-ip>:3000` for remote deployments).
#### **Scaling Transcode Workers (Parallel Processing)**
By default, the Docker Compose deployment transcodes videos one-by-one. This is by design, as video transcoding is compute-intensive and can saturate available CPU or GPU resources.
However, when uploading large batches (e.g., 100+ videos at once) or massive raw camera files that take hours to process and block subsequent uploads, you can scale transcoding throughput across additional machines. Each machine still processes tasks sequentially, but adding more worker instances increases overall parallel capacity.
Shumai makes this seamless with a dedicated [`shumaione/shumai-transcode`](https://hub.docker.com/r/shumaione/shumai-transcode) Docker image:
1. **Connect**: Launch a temporary machine and ensure network access to your PostgreSQL database and storage backend (e.g., S3 or shared storage).
2. **Start the worker**: Run the container with matching database and storage credentials:
```bash
docker run -d --name shumai_transcode_worker \
-e DATABASE_URL="postgres://shumai:shumai_password@<postgres-host>:5432/shumai_db?sslmode=disable" \
-e STORAGE_BACKEND="s3" \
-e S3_BUCKET="your-bucket-name" \
-e AWS_ENDPOINT_URL_S3="https://s3.us-east-1.amazonaws.com" \
-e S3_ACCESS_KEY_ID="your-access-key-id" \
-e S3_SECRET_ACCESS_KEY="your-secret-access-key" \
shumaione/shumai-transcode:latest
```
The worker will automatically attach to the task queue and begin processing pending transcode jobs in parallel with your primary instance.
3. **Stop anytime**: Once the transcoding backlog finishes (or whenever you no longer need the extra compute), simply stop or remove the worker:
```bash
docker stop shumai_transcode_worker
```
There is no downtime or impact on the main Shumai application.
### Option 2: Run from Source (Development)
To set up Shumai locally for development:
+39 -3
View File
@@ -12,7 +12,7 @@
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
<a target="_blank" href="https://deepwiki.com/shumaiOne/shumai"><img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg" /></a>
<a target="_blank" href="https://deepwiki.com/shumaiOne/shumai"><img alt="Ask DeepWiki" src="https://img.shields.io/badge/Ask-DeepWiki-6E56CF" /></a>
</p>
<p align="center">
@@ -29,7 +29,8 @@ Shumai 是一个开源的 Frame.io 替代方案,专为创意团队打造,让
* **逐帧标注和评论**:支持按帧进行精确反馈,可以直接在视频和图片素材上画重点、写评论,还能留下时间戳。
* **安全分享和合集管理**:可以生成私密分享链接,也能整理成精选合集,方便给客户、同事和合作方查看。
* **细粒度权限控制**:支持按团队、项目设置角色权限,方便管理不同成员的访问范围。
* **基于 Temporal 的分布式转码**:把吃资源的视频转码任务交给后台 worker 池处理,由 Temporal 统一调度。
* **基于 DBOS 的持久化工作流**:后台任务与 AI 工作流基于 PostgreSQL 由 DBOS 统一持久化编排与调度,具备自动恢复和容错能力。
* **Kubernetes 弹性扩缩容部署**:支持在多个实例上横向自动扩缩容视频转码工作负载。详见 [EKS 部署示例](eks/README.md)。
* **自定义素材元数据**:可以按你的制作流程,自由定义动态元数据字段。
* **分享链接水印**:通过可自定义的文字和图片水印保护共享媒体,并将水印配置保存为模板以便重复使用。
* **丰富的文件格式支持**:支持预览常见创意文件格式,包括图片(支持 PSD)、视频、音频和文档,同时可上传和存储任意类型的文件。
@@ -69,7 +70,7 @@ Shumai 也将项目管理融入了创意工作流程。 内置的看板可以帮
## 安装指南
以下是使用本地存储运行 Shumai 的快速上手指南。如需了解高级配置(包括 S3 兼容存储配置、Temporal 工作流编排等),请参阅我们的[官方文档](https://docs.shumai.one/getting-started/overview)。
以下是使用本地存储运行 Shumai 的快速上手指南。如需了解高级配置(包括 S3 兼容存储配置、分布式 worker 编排等),请参阅我们的[官方文档](https://docs.shumai.one/getting-started/overview)。
### 方案一:使用 Docker Compose(推荐)
<details>
@@ -164,6 +165,41 @@ https://gh-proxy.com/https://raw.githubusercontent.com/shumaiOne/shumai/main/REA
5. 打开浏览器,访问 `http://localhost:3000` 即可开始使用(若是远程部署,请访问 `http://<您的服务器IP>:3000`)。
#### **扩展转码 Worker(并行处理)**
默认情况下,Docker Compose 部署会按顺序逐个处理视频转码任务。这是出于系统设计考虑,因为视频转码属于计算密集型任务,容易耗尽可用的 CPU 或 GPU 资源。
但在批量上传(例如一次性上传 100+ 个视频)或处理需要耗费数小时的大型摄像机原始素材时,长时间的转码任务可能会阻塞后续文件的处理。此时,您可以通过增加额外的机器来横向扩展转码吞吐量。每台机器依然按顺序执行任务,但增加 worker 实例能够显著提升整体的并行处理能力。
Shumai 提供了独立的转码 Worker 镜像(支持 Docker Hub 与阿里云镜像源):
1. **连接准备**:启动一台临时机器,确保其能够正常访问您的 PostgreSQL 数据库和存储后端(例如 S3 或共享存储)。
2. **启动 Worker**:配置相同的数据库与存储连接凭据并运行容器:
**阿里云镜像(国内推荐):**
```bash
docker run -d --name shumai_transcode_worker \
-e DATABASE_URL="postgres://shumai:shumai_password@<postgres-host>:5432/shumai_db?sslmode=disable" \
-e STORAGE_BACKEND="s3" \
-e S3_BUCKET="your-bucket-name" \
-e AWS_ENDPOINT_URL_S3="https://s3.us-east-1.amazonaws.com" \
-e S3_ACCESS_KEY_ID="your-access-key-id" \
-e S3_SECRET_ACCESS_KEY="your-secret-access-key" \
crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai-transcode:latest
```
*如使用 Docker Hub,镜像为 [`shumaione/shumai-transcode:latest`](https://hub.docker.com/r/shumaione/shumai-transcode)。*
Worker 启动后会自动接入任务队列,并与主实例并行消费并处理积压的转码任务。
3. **随时停止**:当转码队列处理完毕(或不再需要额外的算力)时,只需停止或移除 Worker 容器:
```bash
docker stop shumai_transcode_worker
```
主 Shumai 服务的运行不会受到任何影响,亦无停机时间。
### 方案二:源码编译运行(开发环境)
如果您希望在本地参与 Shumai 的开发,请按以下步骤配置开发环境:
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"bin": {
+1 -10
View File
@@ -21,16 +21,7 @@ async function run() {
console.log('🚀 Starting Agent Worker...')
const options: { workflowBundle?: unknown; workflowsPath?: string } = {}
if (process.env.NODE_ENV === 'production') {
// @ts-ignore: Bun temporal plugin macro
const { default: workflowBundle } = await import('./workflows.ts:::workflow')
options.workflowBundle = workflowBundle
} else {
options.workflowsPath = new URL('./workflows.ts', import.meta.url).pathname
}
await workflowService.startWorkers(TaskQueueAgent, options)
await workflowService.startWorkers(TaskQueueAgent)
}
handleDaemonCommands('shumai-agent', run).catch((err) => {
-3
View File
@@ -1,3 +0,0 @@
export * from '@shumai/agent/src/workflows/agent-autofill'
export * from '@shumai/agent/src/workflows/agent-chat'
export * from '@shumai/agent/src/workflows/agent-embedding'
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/premiere-uxp",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "commonjs",
"scripts": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"bin": {
+76 -9
View File
@@ -14,18 +14,85 @@ async function run() {
// Initialize workflows and activities
initTranscodeWorkflows()
console.log('🚀 Starting Transcode Worker...')
let idleTimeoutMs = process.env.SHUMAI_WORKER_IDLE_TIMEOUT_MS
? parseInt(process.env.SHUMAI_WORKER_IDLE_TIMEOUT_MS, 10)
: 0
const options: { workflowBundle?: unknown; workflowsPath?: string } = {}
if (process.env.NODE_ENV === 'production') {
// @ts-ignore: Bun temporal plugin macro
const { default: workflowBundle } = await import('./workflows.ts:::workflow')
options.workflowBundle = workflowBundle
} else {
options.workflowsPath = new URL('./workflows.ts', import.meta.url).pathname
const idleTimeoutArg = process.argv.find((arg) => arg.startsWith('--idle-timeout='))
if (idleTimeoutArg) {
const val = parseInt(idleTimeoutArg.split('=')[1], 10)
if (!isNaN(val)) idleTimeoutMs = val
}
await workflowService.startWorkers(TaskQueueTranscode, options)
if (idleTimeoutMs > 0) {
console.log(
`🚀 Starting Transcode Worker with idle timeout (${idleTimeoutMs / 1000}s auto-drain)...`,
)
} else {
console.log('🚀 Starting Transcode Worker (persistent daemon mode)...')
}
await workflowService.startWorkers(TaskQueueTranscode)
let isShuttingDown = false
let idleTimer: Timer | null = null
let activeWorkflows = 0
const shutdown = async () => {
if (isShuttingDown) return
isShuttingDown = true
if (idleTimer) {
clearTimeout(idleTimer)
idleTimer = null
}
console.log('Shutting down Transcode Worker gracefully...')
try {
await workflowService.shutdownWorkers()
} catch (err) {
console.error('Error during Transcode Worker shutdown:', err)
}
process.exit(0)
}
if (idleTimeoutMs > 0) {
const resetIdleTimer = () => {
if (idleTimer) {
clearTimeout(idleTimer)
idleTimer = null
}
if (activeWorkflows === 0 && !isShuttingDown) {
idleTimer = setTimeout(async () => {
console.log(
`[Transcode Worker] Inactive for ${idleTimeoutMs / 1000}s with no active workflows, shutting down.`,
)
await shutdown()
}, idleTimeoutMs)
}
}
// Start idle timer immediately on startup in case no tasks are enqueued
resetIdleTimer()
workflowService.getExecutor().on('workflowStarted', (task) => {
activeWorkflows++
console.log(
`[Transcode Worker] Claimed task ${task.id} (active: ${activeWorkflows}), cancelling idle timer.`,
)
if (idleTimer) {
clearTimeout(idleTimer)
idleTimer = null
}
})
workflowService.getExecutor().on('workflowFinished', async ({ taskId }) => {
activeWorkflows = Math.max(0, activeWorkflows - 1)
console.log(`[Transcode Worker] Completed task ${taskId} (active: ${activeWorkflows}).`)
resetIdleTimer()
})
}
process.on('SIGINT', shutdown)
process.on('SIGTERM', shutdown)
}
handleDaemonCommands('shumai-transcode', run).catch((err) => {
-1
View File
@@ -1 +0,0 @@
export * from '@shumai/transcode/src/workflows/transcode'
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"dependencies": {
+25 -22
View File
@@ -53,8 +53,11 @@ const regenCmdIndex = cliArgs.findIndex((arg) => arg === 'regenerate-embeddings'
if (regenCmdIndex !== -1) {
try {
initAgentWorkflows()
initTranscodeWorkflows()
console.log('🔄 Regenerating embeddings...')
const result = await embeddingService.regenerateEmbeddings()
await workflowService.waitForPendingSubmissions()
console.log(
`✅ Successfully queued ${result.tasksCreated} embedding tasks for ${result.totalAssets} assets.`,
)
@@ -86,30 +89,28 @@ async function run() {
await agentService.ensureSystemEmbeddingAgents().catch(console.error)
assetService.startCleanupJob()
notificationJobService.start()
workflowService.start()
if (process.env.WORKFLOW_EXECUTOR === 'temporal') {
const args = process.argv.slice(2)
let workersOption = ''
for (let i = 0; i < args.length; i++) {
if (args[i] === '--workers') {
workersOption = args[i + 1] || ''
} else if (args[i].startsWith('--workers=')) {
workersOption = args[i].split('=')[1]
}
const args = process.argv.slice(2)
let workersOption = process.env.SHUMAI_WORKERS || 'all'
for (let i = 0; i < args.length; i++) {
if (args[i] === '--workers') {
workersOption = args[i + 1] || workersOption
} else if (args[i].startsWith('--workers=')) {
workersOption = args[i].split('=')[1] || workersOption
}
const queuesToStart: string[] = []
if (workersOption === 'agent' || workersOption === 'ai') {
queuesToStart.push('agent_queue')
} else if (workersOption === 'transcode') {
queuesToStart.push('transcode_queue')
} else if (workersOption === 'all') {
queuesToStart.push('agent_queue', 'transcode_queue')
}
Promise.all(queuesToStart.map((q) => workflowService.startWorkers(q))).catch(console.error)
}
let listenQueues: string[] = ['agent_queue', 'transcode_queue']
if (workersOption === 'none') {
listenQueues = []
} else if (workersOption === 'agent' || workersOption === 'ai') {
listenQueues = ['agent_queue']
} else if (workersOption === 'transcode') {
listenQueues = ['transcode_queue']
}
workflowService.start(listenQueues)
workflowService.startStaleTaskRecovery()
const isProd = process.env.NODE_ENV === 'production'
const port = process.env.SHUMAI_SERVER_PORT ? parseInt(process.env.SHUMAI_SERVER_PORT) : 3000
@@ -206,8 +207,10 @@ async function run() {
console.log(`🚀 Server running at ${server.url}`)
const shutdown = () => {
const shutdown = async () => {
console.log('\nShutting down gracefully...')
workflowService.stopStaleTaskRecovery()
await workflowService.shutdownWorkers().catch(() => {})
assetService.stopCleanupJob()
server.stop(true)
process.exit(0)
+45 -34
View File
@@ -3,47 +3,58 @@ import { bunPluginPino } from 'bun-plugin-pino'
import tailwindPlugin from 'bun-plugin-tailwind'
import { existsSync } from 'fs'
import { rm } from 'fs/promises'
import { temporalWorkflow } from './packages/workflow-core/src/bun-temporal-plugin'
console.log('\n🚀 Starting build process...\n')
const outdir = 'dist'
const target = process.argv[2] || 'all'
if (existsSync(outdir)) {
if (existsSync(outdir) && target === 'all') {
console.log(`🗑️ Cleaning previous build at ${outdir}`)
await rm(outdir, { recursive: true, force: true })
}
await Bun.build({
entrypoints: ['./apps/web/src/index.ts'],
root: '.',
target: 'bun',
outdir: outdir,
minify: true,
splitting: true,
naming: '[name].[ext]',
plugins: [
temporalWorkflow({
bundleOptions: {},
}),
tailwindPlugin,
bunPluginPino({ transports: ['pino-pretty'] }),
],
external: [
'@temporalio/activity',
'@temporalio/client',
'@temporalio/worker',
'@temporalio/workflow',
'pdfkit',
],
// compile: {
// outfile: 'Shumai',
// autoloadBunfig: true,
// autoloadDotenv: true,
// autoloadTsconfig: true,
// autoloadPackageJson: true,
// },
define: {
'process.env.NODE_ENV': JSON.stringify('production'),
if (target === 'all' || target === 'web') {
await Bun.build({
entrypoints: ['./apps/web/src/index.ts'],
root: '.',
target: 'bun',
outdir: outdir,
minify: true,
splitting: true,
naming: '[name].[ext]',
plugins: [tailwindPlugin, bunPluginPino({ transports: ['pino-pretty'] })],
external: ['@dbos-inc/dbos-sdk', 'pg', 'pdfkit'],
define: {
'process.env.NODE_ENV': JSON.stringify('production'),
},
})
}
const allWorkers = [
{
name: 'transcode',
entrypoint: './apps/transcode/src/index.ts',
naming: 'transcode-worker.[ext]',
},
})
{ name: 'agent', entrypoint: './apps/agent/src/index.ts', naming: 'agent-worker.[ext]' },
]
const workersToBuild = target === 'all' ? allWorkers : allWorkers.filter((w) => w.name === target)
for (const worker of workersToBuild) {
await Bun.build({
entrypoints: [worker.entrypoint],
root: '.',
target: 'bun',
outdir: outdir,
minify: true,
splitting: false,
naming: worker.naming,
plugins: [bunPluginPino({ transports: ['pino-pretty'] })],
external: ['@dbos-inc/dbos-sdk', 'pg', 'pdfkit'],
define: {
'process.env.NODE_ENV': JSON.stringify('production'),
},
})
}
+34 -283
View File
@@ -11,14 +11,13 @@
"@testcontainers/postgresql": "12.1.0",
"@testing-library/dom": "10.4.1",
"@testing-library/react": "16.3.3",
"@types/bun": "1.4.0",
"@types/bun": "1.4.3",
"@types/node": "25.9.1",
"bun-plugin-pino": "1.5.0",
"bun-plugin-tailwind": "0.1.2",
"bun-types": "1.4.0",
"bun-types": "1.4.3",
"eslint": "10.9.1",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-prettier": "5.5.6",
"eslint-plugin-react": "7.37.5",
"globals": "17.11.0",
"happy-dom": "20.12.0",
@@ -33,7 +32,7 @@
},
"apps/agent": {
"name": "@shumai/agent-app",
"version": "0.5.3",
"version": "0.6.1",
"bin": {
"shumai-agent": "./src/index.ts",
},
@@ -45,7 +44,7 @@
},
"apps/cli": {
"name": "@shumai/cli",
"version": "0.5.3",
"version": "0.6.1",
"bin": {
"shumai-cli": "./src/index.ts",
},
@@ -60,7 +59,7 @@
},
"apps/premiere-uxp": {
"name": "@shumai/premiere-uxp",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@shumai/api": "workspace:*",
"@shumai/dtos": "workspace:*",
@@ -114,7 +113,7 @@
},
"apps/transcode": {
"name": "@shumai/transcode-app",
"version": "0.5.3",
"version": "0.6.1",
"bin": {
"shumai-transcode": "./src/index.ts",
},
@@ -126,7 +125,7 @@
},
"apps/web": {
"name": "@shumai/web-app",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@shumai/agent": "workspace:*",
"@shumai/api": "workspace:*",
@@ -146,7 +145,7 @@
},
"packages/agent": {
"name": "@shumai/agent",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.74",
"@earendil-works/pi-agent-core": "0.82.1",
@@ -156,9 +155,6 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@temporalio/activity": "1.23.0",
"@temporalio/worker": "1.23.0",
"@temporalio/workflow": "1.23.0",
"adm-zip": "0.6.0",
"jittered-fractional-indexing": "1.0.1",
"sharp": "0.35.4",
@@ -172,7 +168,7 @@
},
"packages/api": {
"name": "@shumai/api",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@hono/zod-validator": "0.7.6",
"@shumai/core": "workspace:*",
@@ -186,7 +182,7 @@
},
"packages/core": {
"name": "@shumai/core",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@ai-sdk/alibaba": "2.0.38",
"@ai-sdk/amazon-bedrock": "5.0.67",
@@ -216,8 +212,6 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@sinclair/typebox": "0.34.52",
"@temporalio/activity": "1.24.0",
"@temporalio/workflow": "1.24.0",
"adm-zip": "0.6.0",
"ai": "7.0.84",
"better-auth": "1.6.11",
@@ -249,7 +243,7 @@
},
"packages/db": {
"name": "@shumai/db",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@prisma/adapter-pg": "7.10.0",
@@ -265,7 +259,7 @@
},
"packages/dtos": {
"name": "@shumai/dtos",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"zod": "4.5.4",
@@ -273,7 +267,7 @@
},
"packages/e2e": {
"name": "@shumai/e2e",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@google/genai": "2.28.0",
@@ -288,25 +282,22 @@
},
"packages/tableprinter": {
"name": "@shumai/tableprinter",
"version": "0.5.3",
"version": "0.6.1",
},
"packages/transcode": {
"name": "@shumai/transcode",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@temporalio/activity": "1.23.0",
"@temporalio/worker": "1.23.0",
"@temporalio/workflow": "1.23.0",
"ulid": "3.0.2",
},
},
"packages/webui": {
"name": "@shumai/webui",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@base-ui/react": "1.7.0",
"@dnd-kit/abstract": "0.5.0",
@@ -408,14 +399,12 @@
},
"packages/workflow-core": {
"name": "@shumai/workflow-core",
"version": "0.5.3",
"version": "0.6.1",
"dependencies": {
"@dbos-inc/dbos-sdk": "5.2.11",
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@temporalio/client": "1.23.0",
"@temporalio/worker": "1.23.0",
"@temporalio/workflow": "1.23.0",
"ulid": "3.0.2",
},
},
@@ -602,6 +591,8 @@
"@date-fns/tz": ["@date-fns/tz@1.5.0", "", {}, "sha512-lwYN/vDPeNRULcepoE/LO2Pgx+7/RV+S9ARfbc9lr2DtGkOD7pAiruHvbR1RX3Qyf6ja47EWJDMsNK5vK08DJg=="],
"@dbos-inc/dbos-sdk": ["@dbos-inc/dbos-sdk@5.2.11", "", { "dependencies": { "commander": "^12.0.0", "pg": "^8.11.3", "serialize-error": "^8.1.0", "superjson": "^1.13.3", "ws": "^8.18.1", "yaml": "^2.8.3" }, "bin": { "dbos": "dist/src/cli/cli.js", "dbos-sdk": "dist/src/cli/cli.js" } }, "sha512-qrqWGA+7bt48viESoxRIEt518pcYbJXGy1KZJ3GRaK4WI0sR3f2xmvTqc8+Rhs/tbfROso5zFb9bgnUnMKNqbg=="],
"@dnd-kit/abstract": ["@dnd-kit/abstract@0.5.0", "", { "dependencies": { "@dnd-kit/geometry": "^0.5.0", "@dnd-kit/state": "^0.5.0", "tslib": "^2.6.2" } }, "sha512-hi13iMJgjPX/KDYVKg5VeDIhmYiV6buc9bAX+tCLYf4QdyYjPbsXjn2sPo6m7fQ6SGJBEFgHJ2PemeKDUbwBaA=="],
"@dnd-kit/collision": ["@dnd-kit/collision@0.5.0", "", { "dependencies": { "@dnd-kit/abstract": "^0.5.0", "@dnd-kit/geometry": "^0.5.0", "tslib": "^2.6.2" } }, "sha512-xUqRn3lS7oqLkT0AnnHS/STh/Czvwe1UapZFYiLbsUGxopMsQd4teaPCzPouOThoMdGEe+dHWjfqJl6t9iG4mQ=="],
@@ -742,42 +733,12 @@
"@jridgewell/resolve-uri": ["@jridgewell/resolve-uri@3.1.2", "", {}, "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw=="],
"@jridgewell/source-map": ["@jridgewell/source-map@0.3.11", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.25" } }, "sha512-ZMp1V8ZFcPG5dIWnQLr3NSI1MiCU7UETdS/A0G8V/XWHvJv3ZsFqutJn1Y5RPmAPX6F3BiE397OqveU/9NCuIA=="],
"@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.6.0", "", {}, "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw=="],
"@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="],
"@js-sdsl/ordered-map": ["@js-sdsl/ordered-map@4.4.2", "", {}, "sha512-iUKgm52T8HOE/makSxjqoWhe95ZJA1/G1sYsGev2JDKUSS14KAgg1LHb+Ba+IPow0xflbnSkOsZcO08C7w1gYw=="],
"@jsonjoy.com/base64": ["@jsonjoy.com/base64@1.1.2", "", { "peerDependencies": { "tslib": "2" } }, "sha512-q6XAnWQDIMA3+FTiOYajoYqySkO+JSat0ytXGSuRdq9uXE7o92gzuQwQM14xaCRlBLGq3v5miDGC4vkVTn54xA=="],
"@jsonjoy.com/buffers": ["@jsonjoy.com/buffers@17.67.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-tfExRpYxBvi32vPs9ZHaTjSP4fHAfzSmcahOfNxtvGHcyJel+aibkPlGeBB+7AoC6hL7lXIE++8okecBxx7lcw=="],
"@jsonjoy.com/codegen": ["@jsonjoy.com/codegen@1.0.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-E8Oy+08cmCf0EK/NMxpaJZmOxPqM+6iSe2S4nlSBrPZOORoDJILxtbSUEDKQyTamm/BVAhIGllOBNU79/dwf0g=="],
"@jsonjoy.com/fs-core": ["@jsonjoy.com/fs-core@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-node-builtins": "4.68.2", "@jsonjoy.com/fs-node-utils": "4.68.2", "thingies": "^2.5.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-PoBeUNEbjyLKKwCap2z8LkkqdhdfttS4rTHCALVuP65BdF+sAoyBqHo1m+uGTRBiQWv3H7MGfr8f7lY4pDwanw=="],
"@jsonjoy.com/fs-fsa": ["@jsonjoy.com/fs-fsa@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-core": "4.68.2", "@jsonjoy.com/fs-node-builtins": "4.68.2", "@jsonjoy.com/fs-node-utils": "4.68.2", "thingies": "^2.5.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-h6eGXlLGGMyPfNliDQrbuHKTB9Z29ksCLjreAmdBqrDOBdfrTrHssi+b9WLfrF+J2KzAbIt9OMjJu6AEpTnYkg=="],
"@jsonjoy.com/fs-node": ["@jsonjoy.com/fs-node@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-core": "4.68.2", "@jsonjoy.com/fs-node-builtins": "4.68.2", "@jsonjoy.com/fs-node-utils": "4.68.2", "@jsonjoy.com/fs-print": "4.68.2", "@jsonjoy.com/fs-snapshot": "4.68.2", "glob-to-regex.js": "^1.0.0", "thingies": "^2.5.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-Kpj519Qk4OXG4+mZ8vTf9BEflliJrPueIDEVcbx8tAOufMJJ8IxR0WwdbcEvJol2KQog3PJjtALXVclorE+xbQ=="],
"@jsonjoy.com/fs-node-builtins": ["@jsonjoy.com/fs-node-builtins@4.68.2", "", { "peerDependencies": { "tslib": "2" } }, "sha512-V8WzQsW2YIrH3RxBGY6HZisxn+dDUltHgksVRuCdPEOXEkAfXuhL/01eHFrabNu84Dn13XuLqvcQUKOYVKAUOA=="],
"@jsonjoy.com/fs-node-to-fsa": ["@jsonjoy.com/fs-node-to-fsa@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-fsa": "4.68.2", "@jsonjoy.com/fs-node-builtins": "4.68.2", "@jsonjoy.com/fs-node-utils": "4.68.2" }, "peerDependencies": { "tslib": "2" } }, "sha512-4O1K4w5G4oJIpKpoa3WSLG83AsQYVnGv+aUSBGOvIUph9Axm6bB1mPlvldkNg2tBx/4dkiTlZnqNrK5SQ21Wtg=="],
"@jsonjoy.com/fs-node-utils": ["@jsonjoy.com/fs-node-utils@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-node-builtins": "4.68.2", "glob-to-regex.js": "^1.0.1" }, "peerDependencies": { "tslib": "2" } }, "sha512-CxFwyG9fJr7dAKAd1uanNRNrRMtDDqbYJA8do53+M4LY7SxcBEEmMjC1zi9MOsBlVLHTXvA7OP9+n639UqtIcw=="],
"@jsonjoy.com/fs-print": ["@jsonjoy.com/fs-print@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-node-utils": "4.68.2", "tree-dump": "^1.1.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-cBABQmZJXig6bahwNka3+yPNGUF1GeMWbR76ZM1N2ajgCd+S+twZigCoe9+0ueZmkhM9xd2a7688Gy/ch7T+2w=="],
"@jsonjoy.com/fs-snapshot": ["@jsonjoy.com/fs-snapshot@4.68.2", "", { "dependencies": { "@jsonjoy.com/buffers": "^17.65.0", "@jsonjoy.com/fs-node-utils": "4.68.2", "@jsonjoy.com/json-pack": "^17.65.0", "@jsonjoy.com/util": "^17.65.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-Aix7+NM38LzvewM0T3PICSgFdF5BVCtVgsjNsrlDPGc9hiMgJzReTDDl2/elgl0A9USMl3QP27x5WwxZSOtrfw=="],
"@jsonjoy.com/json-pack": ["@jsonjoy.com/json-pack@1.21.0", "", { "dependencies": { "@jsonjoy.com/base64": "^1.1.2", "@jsonjoy.com/buffers": "^1.2.0", "@jsonjoy.com/codegen": "^1.0.0", "@jsonjoy.com/json-pointer": "^1.0.2", "@jsonjoy.com/util": "^1.9.0", "hyperdyperid": "^1.2.0", "thingies": "^2.5.0", "tree-dump": "^1.1.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-+AKG+R2cfZMShzrF2uQw34v3zbeDYUqnQ+jg7ORic3BGtfw9p/+N6RJbq/kkV8JmYZaINknaEQ2m0/f693ZPpg=="],
"@jsonjoy.com/json-pointer": ["@jsonjoy.com/json-pointer@1.0.2", "", { "dependencies": { "@jsonjoy.com/codegen": "^1.0.0", "@jsonjoy.com/util": "^1.9.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-Fsn6wM2zlDzY1U+v4Nc8bo3bVqgfNTGcn6dMgs6FjrEnt4ZCe60o6ByKRjOGlI2gow0aE/Q41QOigdTqkyK5fg=="],
"@jsonjoy.com/util": ["@jsonjoy.com/util@1.9.0", "", { "dependencies": { "@jsonjoy.com/buffers": "^1.0.0", "@jsonjoy.com/codegen": "^1.0.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-pLuQo+VPRnN8hfPqUTLTHk126wuYdXVxE6aDmjSeV4NCAgyxWbiOIeNJVtID3h1Vzpoi9m4jXezf73I6LgabgQ=="],
"@kwsites/file-exists": ["@kwsites/file-exists@1.1.1", "", { "dependencies": { "debug": "^4.1.1" } }, "sha512-m9/5YGR18lIwxSFDwfE3oA7bWuq9kdau6ugN4H2rJeyhFQZcG9AgSHkQtSD15a8WvTgfz9aikZMrKPHvbpqFiw=="],
"@lexical/a11y": ["@lexical/a11y@0.49.0", "", { "dependencies": { "@lexical/extension": "0.49.0", "@lexical/utils": "0.49.0", "lexical": "0.49.0" }, "peerDependencies": { "typescript": ">=5.2" }, "optionalPeers": ["typescript"] }, "sha512-ypvO0SI9DCzEAjxkRLq+gNzRqfjg632RCkwu16Hk94PCBrEmcDhx5iUGg3SRJC7WnM2wXC/F/SB8yWDEfq7tEw=="],
@@ -920,8 +881,6 @@
"@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="],
"@pkgr/core": ["@pkgr/core@0.3.6", "", {}, "sha512-SEeaJLb3qBNF/OaXnaR1NmmBbFYk1zC0ZH/52fATcRPLFg/p791YrcyFFy44Bo9sLaGuSuLp5Q6axbb/O+v/RA=="],
"@playwright/test": ["@playwright/test@1.61.1", "", { "dependencies": { "playwright": "1.61.1" }, "bin": { "playwright": "cli.js" } }, "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig=="],
"@pondwader/socks5-server": ["@pondwader/socks5-server@1.0.10", "", {}, "sha512-bQY06wzzR8D2+vVCUoBsr5QS2U6UgPUQRmErNwtsuI6vLcyRKkafjkr3KxbtGFf9aBBIV2mcvlsKD1UYaIV+sg=="],
@@ -1382,38 +1341,8 @@
"@swc-uxp-wrappers/utils": ["@swc-uxp-wrappers/utils@2.0.1", "", { "dependencies": { "@swc-uxp-internal/base": "npm:@spectrum-web-components/base@0.37.0", "@swc-uxp-internal/icon": "npm:@spectrum-web-components/icon@0.37.0", "@swc-uxp-internal/icons": "npm:@spectrum-web-components/icons@0.37.0", "@swc-uxp-internal/icons-ui": "npm:@spectrum-web-components/icons-ui@0.37.0", "@swc-uxp-internal/icons-workflow": "npm:@spectrum-web-components/icons-workflow@0.37.0", "@swc-uxp-internal/iconset": "npm:@spectrum-web-components/iconset@0.37.0", "@swc-uxp-internal/reactive-controllers": "npm:@spectrum-web-components/reactive-controllers@0.37.0", "@swc-uxp-internal/shared": "npm:@spectrum-web-components/shared@0.37.0", "@swc-uxp-internal/styles": "npm:@spectrum-web-components/styles@0.37.0", "@swc-uxp-internal/theme": "npm:@spectrum-web-components/theme@0.37.0" } }, "sha512-8vbd6fOok2/f8Tp0KPlBPUJYlMhnMGmEhiiySf/dhsOR85kiFMKf3VwILWkd0No2+fViARviDCVZbk+5F8EtNA=="],
"@swc/core": ["@swc/core@1.16.1", "", { "dependencies": { "@swc/counter": "^0.1.3", "@swc/types": "^0.1.28" }, "optionalDependencies": { "@swc/core-darwin-arm64": "1.16.1", "@swc/core-darwin-x64": "1.16.1", "@swc/core-linux-arm-gnueabihf": "1.16.1", "@swc/core-linux-arm64-gnu": "1.16.1", "@swc/core-linux-arm64-musl": "1.16.1", "@swc/core-linux-ppc64-gnu": "1.16.1", "@swc/core-linux-s390x-gnu": "1.16.1", "@swc/core-linux-x64-gnu": "1.16.1", "@swc/core-linux-x64-musl": "1.16.1", "@swc/core-win32-arm64-msvc": "1.16.1", "@swc/core-win32-ia32-msvc": "1.16.1", "@swc/core-win32-x64-msvc": "1.16.1" }, "peerDependencies": { "@swc/helpers": ">=0.5.17" }, "optionalPeers": ["@swc/helpers"] }, "sha512-nUaeu91O5QZKrQdaDCHd402ogUIoNOOjpkZNq0UomWK0G6gDaGmLhvddF1/3BXf5O8aLyo6ZPY/aMDWvaJQ/hg=="],
"@swc/core-darwin-arm64": ["@swc/core-darwin-arm64@1.16.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-zlJblJ8ncErD43lKdxjbUaUskJQf+LxiPXYcWXD8/8ZMV+7uuAT+CwjciLXpyZBd5Pq/S726bMpeeAwSeL1hhg=="],
"@swc/core-darwin-x64": ["@swc/core-darwin-x64@1.16.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-IN0BmPWb0YAh/17mmlWB/HDBtTw2MfuW4hulf/tQAgTQBRH17l+z499bNJLK6LizSjqs0P7V+jU38Zj+vJC1DA=="],
"@swc/core-linux-arm-gnueabihf": ["@swc/core-linux-arm-gnueabihf@1.16.1", "", { "os": "linux", "cpu": "arm" }, "sha512-EYgrx2YOCQ2Twz2S793kqNjPkpvYVUPzzR95bIb7by+VQcyaai4lZZ2iz/tZvcFVKSNcN3/JTKwx+aBn2ZL52A=="],
"@swc/core-linux-arm64-gnu": ["@swc/core-linux-arm64-gnu@1.16.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-moyKm0YZlHdHohzm1YwgAyesqnE853rO0REMfJLFAova51wF9BNi+3ZW2PeS7Vqvn6HeJuepLpAHbBdZctxpHA=="],
"@swc/core-linux-arm64-musl": ["@swc/core-linux-arm64-musl@1.16.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-kKGBO9wdapiSzuf5ZzZ2fYtlu1BNSYtIIUxvH1ir/gcelTOREEHGDCLTDFx/2Knf878nU11A40z7LxwasEFxqA=="],
"@swc/core-linux-ppc64-gnu": ["@swc/core-linux-ppc64-gnu@1.16.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-nZ6qahtLxC3PM54cWOQZHxt4lTCF/3J4LIoWWzz6v7A+rLs8Dx54anYQf7mH3eIi8KlNpgKci/ie8ZSqFN8O7A=="],
"@swc/core-linux-s390x-gnu": ["@swc/core-linux-s390x-gnu@1.16.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-4ji5PNzhYq193Z4/4xUaSoNJza6iCkDJSzhetrbB6KOYxsr+kxtQr8ePWhMJUiMt6JUWtXaZ1PYT8FhtED+nGA=="],
"@swc/core-linux-x64-gnu": ["@swc/core-linux-x64-gnu@1.16.1", "", { "os": "linux", "cpu": "x64" }, "sha512-VJQxqrisHV+B394IgrOu8YsIIXZgffnf5tO+yc9Z/hoUpuZEvuQTjWwlnpZdpyD+0nx6LTD1/3k646JYm43yJA=="],
"@swc/core-linux-x64-musl": ["@swc/core-linux-x64-musl@1.16.1", "", { "os": "linux", "cpu": "x64" }, "sha512-r9oV1mwxxsIGcLV1IQ/tw76MW3doatKze1QFWuC+a7QqJUkhY/bKTSVk6NpKKUGm2LDsE33Va8VqSClfA7vSiQ=="],
"@swc/core-win32-arm64-msvc": ["@swc/core-win32-arm64-msvc@1.16.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-6huNRessoBLxWEqBm5zJXyCQ27TO7anvkdiuQ5MDO4CJni0nOXEqKtV9RllQ2TdyENKKsUMXVnIfW2hIXx/R5Q=="],
"@swc/core-win32-ia32-msvc": ["@swc/core-win32-ia32-msvc@1.16.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-OVKJFUzphrGmsh+BGtcZDesx0YryV7/Yvy5XGgTqnrZfjnyfcr5uaqYQugCckdIlupc5Vs3XtDjRAj12z4ZPlw=="],
"@swc/core-win32-x64-msvc": ["@swc/core-win32-x64-msvc@1.16.1", "", { "os": "win32", "cpu": "x64" }, "sha512-Bt+VIhWYCGk4urklnkkteLUOeLv1VxigwTCeB/xC6rBZxY6IIKdDwCJf6on3E3SUGsIqmQS6QqtuJQc1VxF4Aw=="],
"@swc/counter": ["@swc/counter@0.1.3", "", {}, "sha512-e2BR4lsJkkRlKZ/qCHPw9ZaSxc0MVUd7gtbtaB7aMvHeJVYe8sOB8DBZkP2DtISHGSku9sCK6T6cnY0CtXrOCQ=="],
"@swc/helpers": ["@swc/helpers@0.5.23", "", { "dependencies": { "tslib": "^2.8.0" } }, "sha512-5lSsMOTXURePglDfvuAQUqkGek9Hg2kksOYay2m0+XR++b2NWYL/4sWyuvVBIs8oKnJaxkdi9whaL/sqN13afw=="],
"@swc/types": ["@swc/types@0.1.28", "", { "dependencies": { "@swc/counter": "^0.1.3" } }, "sha512-V6Mnml8v09QALx6K0elJ7o9K/MkVDtW3t6L+7Ou/JcWtb3xwId2AH4FeOceySd2JaO87IMw4+6vSZxLm34LPbw=="],
"@tailwindcss/node": ["@tailwindcss/node@4.3.3", "", { "dependencies": { "@jridgewell/remapping": "^2.3.5", "enhanced-resolve": "^5.24.1", "jiti": "^2.7.0", "lightningcss": "1.32.0", "magic-string": "^0.30.21", "source-map-js": "^1.2.1", "tailwindcss": "4.3.3" } }, "sha512-/T8IKEsf9VTU6tLjgC7+sv2mOPtQxzE2jMw7u4Tt40Tx+QSZxpzh95/H6cMKoja9XuW7iMdLJYBB0o9G1CaAgg=="],
"@tailwindcss/oxide": ["@tailwindcss/oxide@4.3.3", "", { "optionalDependencies": { "@tailwindcss/oxide-android-arm64": "4.3.3", "@tailwindcss/oxide-darwin-arm64": "4.3.3", "@tailwindcss/oxide-darwin-x64": "4.3.3", "@tailwindcss/oxide-freebsd-x64": "4.3.3", "@tailwindcss/oxide-linux-arm-gnueabihf": "4.3.3", "@tailwindcss/oxide-linux-arm64-gnu": "4.3.3", "@tailwindcss/oxide-linux-arm64-musl": "4.3.3", "@tailwindcss/oxide-linux-x64-gnu": "4.3.3", "@tailwindcss/oxide-linux-x64-musl": "4.3.3", "@tailwindcss/oxide-wasm32-wasi": "4.3.3", "@tailwindcss/oxide-win32-arm64-msvc": "4.3.3", "@tailwindcss/oxide-win32-x64-msvc": "4.3.3" } }, "sha512-krXjAikiaFSPaK/FkAQT5UTx3VormQaiZ5hBFlJZ9UFQGB/rwg1MZIhHAG9smMQRTdyJxP6Qt5MwMtdyU5FWrA=="],
@@ -1484,22 +1413,6 @@
"@tanstack/virtual-file-routes": ["@tanstack/virtual-file-routes@1.162.0", "", {}, "sha512-uhOeFyxLcU41HzvrxsGpiWdcMbScY1EDgbZ5K7DVRMYInbLYWAC0EA/kx9wXAoSM8q82bUG2hRl8+EAjE6XAbA=="],
"@temporalio/activity": ["@temporalio/activity@1.23.0", "", { "dependencies": { "@temporalio/client": "1.23.0", "@temporalio/common": "1.23.0" } }, "sha512-BVeiizVD08yk0Ls6MhKCK+g3fPIWbVM9MLx+o990KwhCaZtRM6o1+du1/qfIIShHl6YjHk+H0ewkx2x14QQ6bA=="],
"@temporalio/client": ["@temporalio/client@1.23.0", "", { "dependencies": { "@grpc/grpc-js": "^1.12.4", "@temporalio/common": "1.23.0", "@temporalio/proto": "1.23.0", "abort-controller": "^3.0.0", "long": "^5.3.2", "nexus-rpc": "^0.0.3", "uuid": "^11.1.0" } }, "sha512-E9nXZPiKDGEDbPpRi0EyQ9i1x70fEIG26aMXZO88kKslKZnEKNPphs06xIic7bhuDoSrPsgVfdDap6o3l6IO2A=="],
"@temporalio/common": ["@temporalio/common@1.23.0", "", { "dependencies": { "@temporalio/proto": "1.23.0", "long": "^5.3.2", "ms": "3.0.0-canary.1", "nexus-rpc": "^0.0.3", "protobufjs": "^8.7.1" } }, "sha512-TaZYb/ruzgzJ8jd42EqRUCI4ispLZETt2Jigc9szES6q9edDmfncRw143HyvPi7b8n9naohYmA/IxTe9COH/dg=="],
"@temporalio/core-bridge": ["@temporalio/core-bridge@1.23.0", "", { "dependencies": { "@grpc/grpc-js": "^1.12.4", "@temporalio/common": "1.23.0" } }, "sha512-sid14odlYnci9WgB+iDLyxFbzkq+qBbRPp+CW9U1v+DLRfHlFJwpznQ1AZl4mHjtAZRGQFvDsDG0XNncl9ESSA=="],
"@temporalio/nexus": ["@temporalio/nexus@1.23.0", "", { "dependencies": { "@temporalio/client": "1.23.0", "@temporalio/common": "1.23.0", "@temporalio/proto": "1.23.0", "long": "^5.3.2", "nexus-rpc": "^0.0.3" } }, "sha512-mObGNp3thR+mTXdtKXpUvFpsdjEI9xpVjewhblKXao7Nqv6xrs7RpFiLdKfJhkEvDyfRgIk/dH5RkiCbZ58I9w=="],
"@temporalio/proto": ["@temporalio/proto@1.23.0", "", { "dependencies": { "long": "^5.3.2", "protobufjs": "^8.7.1" } }, "sha512-90JWnYEs1wyq97/SXkQGknRVMr/BCV9sZ0pOnK3fHTedVinfvz3/PBVFkeAJyU3CJqMQGnzRs9IIfsBTT1icrA=="],
"@temporalio/worker": ["@temporalio/worker@1.23.0", "", { "dependencies": { "@grpc/grpc-js": "^1.12.4", "@swc/core": "^1.3.102", "@temporalio/activity": "1.23.0", "@temporalio/client": "1.23.0", "@temporalio/common": "1.23.0", "@temporalio/core-bridge": "1.23.0", "@temporalio/nexus": "1.23.0", "@temporalio/proto": "1.23.0", "@temporalio/workflow": "1.23.0", "heap-js": "^2.6.0", "memfs": "^4.6.0", "nexus-rpc": "^0.0.3", "protobufjs": "^8.7.1", "rxjs": "^7.8.1", "source-map": "^0.7.4", "source-map-loader": "^5.0.0", "supports-color": "^8.1.1", "swc-loader": "^0.2.3", "unionfs": "^4.5.1", "webpack": "^5.108.4" } }, "sha512-k1MrK10lD1s7U66PFqAwGlQMRsVrfU5nCB+CjIAg5RyL7JpOzidDCz/E/jB2HobXzZnxtMcDBfBHh/BYUoyhxw=="],
"@temporalio/workflow": ["@temporalio/workflow@1.23.0", "", { "dependencies": { "@temporalio/common": "1.23.0", "@temporalio/proto": "1.23.0", "nexus-rpc": "^0.0.3" } }, "sha512-dHQnYmxU/QN7VfTmdfCfpdfTS1w0hoW/NQ7Zefr55QnqLwZw+E0Vn6S4fAbJrpfHrsAmezUZQFVLi+Z4Sq+jzA=="],
"@testcontainers/postgresql": ["@testcontainers/postgresql@12.1.0", "", { "dependencies": { "testcontainers": "^12.1.0" } }, "sha512-Pjf2VSVNirEPfz36nidyrVAnZvc2YhajOznY4VgyEsvfTd5qiMNOuPq96drREvxAUtXl5SFLX7vXj7sSq4aTcA=="],
"@testing-library/dom": ["@testing-library/dom@10.4.1", "", { "dependencies": { "@babel/code-frame": "^7.10.4", "@babel/runtime": "^7.12.5", "@types/aria-query": "^5.0.1", "aria-query": "5.3.0", "dom-accessibility-api": "^0.5.9", "lz-string": "^1.5.0", "picocolors": "1.1.1", "pretty-format": "^27.0.2" } }, "sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg=="],
@@ -1512,7 +1425,7 @@
"@types/aria-query": ["@types/aria-query@5.0.4", "", {}, "sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw=="],
"@types/bun": ["@types/bun@1.4.0", "", { "dependencies": { "bun-types": "1.4.0" } }, "sha512-K+lZULY23vRgK/CfTjFIV+tyifaNdSMlPh9j+6mQ/cLfpOznLyAuzgV/JQysyECpkBQLVMSyvjlr2fBUSA9wFQ=="],
"@types/bun": ["@types/bun@1.4.3", "", { "dependencies": { "bun-types": "1.4.3" } }, "sha512-N67Pqy8JXbAY/j0tNraS3V5mubGvc6rrrNDmDCtZLlFzccbUJPDb0tvgUBC5q4m8ybkP93Jcn1QB+RYQ6PvBhw=="],
"@types/chai": ["@types/chai@5.2.3", "", { "dependencies": { "@types/deep-eql": "*", "assertion-error": "^2.0.1" } }, "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA=="],
@@ -1666,42 +1579,8 @@
"@vitest/utils": ["@vitest/utils@4.1.11", "", { "dependencies": { "@vitest/pretty-format": "4.1.11", "convert-source-map": "^2.0.0", "tinyrainbow": "^3.1.0" } }, "sha512-zTCVGpyFsGWBhllOyKlTw/vnr6D9qxsfSDyfbyZmTyjHw5N/VuvzHpHoQjm2ZJzn4RJgx5w4r7V0er69CmLgPQ=="],
"@webassemblyjs/ast": ["@webassemblyjs/ast@1.14.1", "", { "dependencies": { "@webassemblyjs/helper-numbers": "1.13.2", "@webassemblyjs/helper-wasm-bytecode": "1.13.2" } }, "sha512-nuBEDgQfm1ccRp/8bCQrx1frohyufl4JlbMMZ4P1wpeOfDhF6FQkxZJ1b/e+PLwr6X1Nhw6OLme5usuBWYBvuQ=="],
"@webassemblyjs/floating-point-hex-parser": ["@webassemblyjs/floating-point-hex-parser@1.13.2", "", {}, "sha512-6oXyTOzbKxGH4steLbLNOu71Oj+C8Lg34n6CqRvqfS2O71BxY6ByfMDRhBytzknj9yGUPVJ1qIKhRlAwO1AovA=="],
"@webassemblyjs/helper-api-error": ["@webassemblyjs/helper-api-error@1.13.2", "", {}, "sha512-U56GMYxy4ZQCbDZd6JuvvNV/WFildOjsaWD3Tzzvmw/mas3cXzRJPMjP83JqEsgSbyrmaGjBfDtV7KDXV9UzFQ=="],
"@webassemblyjs/helper-buffer": ["@webassemblyjs/helper-buffer@1.14.1", "", {}, "sha512-jyH7wtcHiKssDtFPRB+iQdxlDf96m0E39yb0k5uJVhFGleZFoNw1c4aeIcVUPPbXUVJ94wwnMOAqUHyzoEPVMA=="],
"@webassemblyjs/helper-numbers": ["@webassemblyjs/helper-numbers@1.13.2", "", { "dependencies": { "@webassemblyjs/floating-point-hex-parser": "1.13.2", "@webassemblyjs/helper-api-error": "1.13.2", "@xtuc/long": "4.2.2" } }, "sha512-FE8aCmS5Q6eQYcV3gI35O4J789wlQA+7JrqTTpJqn5emA4U2hvwJmvFRC0HODS+3Ye6WioDklgd6scJ3+PLnEA=="],
"@webassemblyjs/helper-wasm-bytecode": ["@webassemblyjs/helper-wasm-bytecode@1.13.2", "", {}, "sha512-3QbLKy93F0EAIXLh0ogEVR6rOubA9AoZ+WRYhNbFyuB70j3dRdwH9g+qXhLAO0kiYGlg3TxDV+I4rQTr/YNXkA=="],
"@webassemblyjs/helper-wasm-section": ["@webassemblyjs/helper-wasm-section@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@webassemblyjs/helper-buffer": "1.14.1", "@webassemblyjs/helper-wasm-bytecode": "1.13.2", "@webassemblyjs/wasm-gen": "1.14.1" } }, "sha512-ds5mXEqTJ6oxRoqjhWDU83OgzAYjwsCV8Lo/N+oRsNDmx/ZDpqalmrtgOMkHwxsG0iI//3BwWAErYRHtgn0dZw=="],
"@webassemblyjs/ieee754": ["@webassemblyjs/ieee754@1.13.2", "", { "dependencies": { "@xtuc/ieee754": "^1.2.0" } }, "sha512-4LtOzh58S/5lX4ITKxnAK2USuNEvpdVV9AlgGQb8rJDHaLeHciwG4zlGr0j/SNWlr7x3vO1lDEsuePvtcDNCkw=="],
"@webassemblyjs/leb128": ["@webassemblyjs/leb128@1.13.2", "", { "dependencies": { "@xtuc/long": "4.2.2" } }, "sha512-Lde1oNoIdzVzdkNEAWZ1dZ5orIbff80YPdHx20mrHwHrVNNTjNr8E3xz9BdpcGqRQbAEa+fkrCb+fRFTl/6sQw=="],
"@webassemblyjs/utf8": ["@webassemblyjs/utf8@1.13.2", "", {}, "sha512-3NQWGjKTASY1xV5m7Hr0iPeXD9+RDobLll3T9d2AO+g3my8xy5peVyjSag4I50mR1bBSN/Ct12lo+R9tJk0NZQ=="],
"@webassemblyjs/wasm-edit": ["@webassemblyjs/wasm-edit@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@webassemblyjs/helper-buffer": "1.14.1", "@webassemblyjs/helper-wasm-bytecode": "1.13.2", "@webassemblyjs/helper-wasm-section": "1.14.1", "@webassemblyjs/wasm-gen": "1.14.1", "@webassemblyjs/wasm-opt": "1.14.1", "@webassemblyjs/wasm-parser": "1.14.1", "@webassemblyjs/wast-printer": "1.14.1" } }, "sha512-RNJUIQH/J8iA/1NzlE4N7KtyZNHi3w7at7hDjvRNm5rcUXa00z1vRz3glZoULfJ5mpvYhLybmVcwcjGrC1pRrQ=="],
"@webassemblyjs/wasm-gen": ["@webassemblyjs/wasm-gen@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@webassemblyjs/helper-wasm-bytecode": "1.13.2", "@webassemblyjs/ieee754": "1.13.2", "@webassemblyjs/leb128": "1.13.2", "@webassemblyjs/utf8": "1.13.2" } }, "sha512-AmomSIjP8ZbfGQhumkNvgC33AY7qtMCXnN6bL2u2Js4gVCg8fp735aEiMSBbDR7UQIj90n4wKAFUSEd0QN2Ukg=="],
"@webassemblyjs/wasm-opt": ["@webassemblyjs/wasm-opt@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@webassemblyjs/helper-buffer": "1.14.1", "@webassemblyjs/wasm-gen": "1.14.1", "@webassemblyjs/wasm-parser": "1.14.1" } }, "sha512-PTcKLUNvBqnY2U6E5bdOQcSM+oVP/PmrDY9NzowJjislEjwP/C4an2303MCVS2Mg9d3AJpIGdUFIQQWbPds0Sw=="],
"@webassemblyjs/wasm-parser": ["@webassemblyjs/wasm-parser@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@webassemblyjs/helper-api-error": "1.13.2", "@webassemblyjs/helper-wasm-bytecode": "1.13.2", "@webassemblyjs/ieee754": "1.13.2", "@webassemblyjs/leb128": "1.13.2", "@webassemblyjs/utf8": "1.13.2" } }, "sha512-JLBl+KZ0R5qB7mCnud/yyX08jWFw5MsoalJ1pQ4EdFlgj9VdXKGuENGsiCIjegI1W7p91rUlcB/LB5yRJKNTcQ=="],
"@webassemblyjs/wast-printer": ["@webassemblyjs/wast-printer@1.14.1", "", { "dependencies": { "@webassemblyjs/ast": "1.14.1", "@xtuc/long": "4.2.2" } }, "sha512-kPSSXE6De1XOR820C90RIo2ogvZG+c3KiHzqUoO/F34Y2shGzesfqv7o57xrxovZJH/MetF5UjroJ/R/3isoiw=="],
"@workflow/serde": ["@workflow/serde@4.1.0", "", {}, "sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ=="],
"@xtuc/ieee754": ["@xtuc/ieee754@1.2.0", "", {}, "sha512-DX8nKgqcGwsc0eJSqYt5lwP4DH5FlHnmuWWBRy7X0NcaGR0ZtuyeESgMwTYVEtxmsNGY+qit4QYT/MIYTOTPeA=="],
"@xtuc/long": ["@xtuc/long@4.2.2", "", {}, "sha512-NuHqBY1PB/D8xU6s/thBgOAiAP7HOYDQ32+BFZILJ8ivkUkAHQnWfn6WhL79Owj1qmUnoN/YPhktdIoucipkAQ=="],
"abort-controller": ["abort-controller@3.0.0", "", { "dependencies": { "event-target-shim": "^5.0.0" } }, "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg=="],
"acorn": ["acorn@8.18.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-lGq+9yr1/GuAWaVYIHRjvvySG5/4VfKIvC8EWxStPdcDh/Ka7FG3twP6v4d5BkravUilhIAsG4Qj83t02LWUPQ=="],
@@ -1716,10 +1595,6 @@
"ajv": ["ajv@6.15.0", "", { "dependencies": { "fast-deep-equal": "^3.1.1", "fast-json-stable-stringify": "^2.0.0", "json-schema-traverse": "^0.4.1", "uri-js": "^4.2.2" } }, "sha512-fgFx7Hfoq60ytK2c7DhnF8jIvzYgOMxfugjLOSMHjLIPgenqa7S7oaagATUq99mV6IYvN2tRmC0wnTYX6iPbMw=="],
"ajv-formats": ["ajv-formats@2.1.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-Wx0Kx52hxE7C18hkMEggYlEifqWZtYaRgouJor+WMdPnQyEK13vgEWyVNup7SoeeoLMsr4kf5h6dOW11I15MUA=="],
"ajv-keywords": ["ajv-keywords@5.1.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3" }, "peerDependencies": { "ajv": "^8.8.2" } }, "sha512-YCS/JNFAUyr5vAuhk1DWm1CBxRHW9LbJ2ozWeemrIqpbsqKjHVxYPyi5GC0rjZIT5JxJ3virVTS8wk4i/Z+krw=="],
"ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
"ansi-styles": ["ansi-styles@5.2.0", "", {}, "sha512-Cxwpt2SfTzTtXcfOlzGEee8O+c+MmUgGrNiBcXnuWxuFJHe6a5Hz7qwhwe5OgaSYI0IJvkLqWX1ASG+cJOkEiA=="],
@@ -1820,8 +1695,6 @@
"buffer-equal-constant-time": ["buffer-equal-constant-time@1.0.1", "", {}, "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA=="],
"buffer-from": ["buffer-from@1.1.2", "", {}, "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ=="],
"buffer-image-size": ["buffer-image-size@0.6.4", "", { "dependencies": { "@types/node": "*" } }, "sha512-nEh+kZOPY1w+gcCMobZ6ETUp9WfibndnosbpwB1iJk/8Gt5ZF2bhS6+B6bPYz424KtwsR6Rflc3tCz1/ghX2dQ=="],
"buildcheck": ["buildcheck@0.0.7", "", {}, "sha512-lHblz4ahamxpTmnsk+MNTRWsjYKv965MwOrSJyeD588rR3Jcu7swE+0wN5F+PbL5cjgu/9ObkhfzEPuofEMwLA=="],
@@ -1832,7 +1705,7 @@
"bun-plugin-tailwind": ["bun-plugin-tailwind@0.1.2", "", { "peerDependencies": { "bun": ">=1.0.0" } }, "sha512-41jNC1tZRSK3s1o7pTNrLuQG8kL/0vR/JgiTmZAJ1eHwe0w5j6HFPKeqEk0WAD13jfrUC7+ULuewFBBCoADPpg=="],
"bun-types": ["bun-types@1.4.0", "", { "dependencies": { "@types/node": "*" } }, "sha512-iIKw23BspnQQYd3prITOBxeUsxBHnwzX6YJfGMuNOZzeNcMmVqzIIVGRm1l69ogaPQmb4wB6BN8mA5bE9YuC5Q=="],
"bun-types": ["bun-types@1.4.3", "", { "dependencies": { "@types/node": "*", "undici-types": "*" } }, "sha512-tinRJNGOmUN+tEAsn2y2e942n9JHRHR42iUYzDH75NmhJqVBokGSQkkfSFdNWwsANIdrK6ACH9t01M452YyTwg=="],
"byline": ["byline@5.0.0", "", {}, "sha512-s6webAy+R4SR8XVuJWt2V2rGvhnrhxN+9S15GNuTK3wKPOXFF6RNc+8ug2XhH+2s4f+uudG4kUVYmYOQWL2g0Q=="],
@@ -1864,8 +1737,6 @@
"chownr": ["chownr@1.1.4", "", {}, "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg=="],
"chrome-trace-event": ["chrome-trace-event@1.0.4", "", {}, "sha512-rNjApaLzuwaOTjCiT8lSDdGN1APCiqkChLMJxJPWLunPAt5fy8xgU9/jNOchV84wfIxrA0lRQB7oCT8jrn/wrQ=="],
"class-variance-authority": ["class-variance-authority@0.7.1", "", { "dependencies": { "clsx": "^2.1.1" } }, "sha512-Ka+9Trutv7G8M6WT6SeiRWz792K5qEqIGEGzXKhAE6xOWAY6pPH8U+9IY3oCMv6kqTmLsv7Xh/2w2RigkePMsg=="],
"classnames": ["classnames@2.5.1", "", {}, "sha512-saHYOzhIQs6wy2sVxTM6bUDsQO4F50V9RQ22qBpEdCW+I+/Wmke2HOl6lS6dTpdxVhb88/I6+Hs+438c3lfUow=="],
@@ -1904,6 +1775,8 @@
"cookie-es": ["cookie-es@3.1.1", "", {}, "sha512-UaXxwISYJPTr9hwQxMFYZ7kNhSXboMXP+Z3TRX6f1/NyaGPfuNUZOWP1pUEb75B2HjfklIYLVRfWiFZJyC6Npg=="],
"copy-anything": ["copy-anything@3.0.5", "", { "dependencies": { "is-what": "^4.1.8" } }, "sha512-yCEafptTtb4bk7GLEQoM8KVJpxAfdBJYaXyzQEgQQQgYrZiDp8SJmGKlYza6CYjEDNstAdNdKA3UuoULlEbS6w=="],
"core-util-is": ["core-util-is@1.0.3", "", {}, "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ=="],
"cpu-features": ["cpu-features@0.0.10", "", { "dependencies": { "buildcheck": "~0.0.6", "nan": "^2.19.0" } }, "sha512-9IkYqtX3YHPCzoVg1Py+o9057a3i0fp7S530UWokCSaFVTc7CwXPRiOjRjBQQ18ZCNafx78YfnG+HALxtVmOGA=="],
@@ -2066,8 +1939,6 @@
"eslint-config-prettier": ["eslint-config-prettier@10.1.8", "", { "peerDependencies": { "eslint": ">=7.0.0" }, "bin": { "eslint-config-prettier": "bin/cli.js" } }, "sha512-82GZUjRS0p/jganf6q1rEO25VSoHH0hKPCTrgillPjdI/3bgBhAE1QzHrHTizjpRvy6pGAvKjDJtk2pF9NDq8w=="],
"eslint-plugin-prettier": ["eslint-plugin-prettier@5.5.6", "", { "dependencies": { "prettier-linter-helpers": "^1.0.1", "synckit": "^0.11.13" }, "peerDependencies": { "@types/eslint": ">=8.0.0", "eslint": ">=8.0.0", "eslint-config-prettier": ">= 7.0.0 <10.0.0 || >=10.1.0", "prettier": ">=3.0.0" }, "optionalPeers": ["@types/eslint", "eslint-config-prettier"] }, "sha512-ifetmTcxWfz+4qRW3pH/ujdTq2jQIj59AxJMIN26K5avYgU8dxycUETQonWiW+wPrYXA0j3Try0l1CnwVQtDqQ=="],
"eslint-plugin-react": ["eslint-plugin-react@7.37.5", "", { "dependencies": { "array-includes": "^3.1.8", "array.prototype.findlast": "^1.2.5", "array.prototype.flatmap": "^1.3.3", "array.prototype.tosorted": "^1.1.4", "doctrine": "^2.1.0", "es-iterator-helpers": "^1.2.1", "estraverse": "^5.3.0", "hasown": "^2.0.2", "jsx-ast-utils": "^2.4.1 || ^3.0.0", "minimatch": "^3.1.2", "object.entries": "^1.1.9", "object.fromentries": "^2.0.8", "object.values": "^1.2.1", "prop-types": "^15.8.1", "resolve": "^2.0.0-next.5", "semver": "^6.3.1", "string.prototype.matchall": "^4.0.12", "string.prototype.repeat": "^1.0.0" }, "peerDependencies": { "eslint": "^3 || ^4 || ^5 || ^6 || ^7 || ^8 || ^9.7" } }, "sha512-Qteup0SqU15kdocexFNAJMvCJEfa2xUKNV4CC1xsVMrIIqEy3SQ/rqyxCWNzfrd3/ldy6HMlD2e0JDVpDg2qIA=="],
"eslint-scope": ["eslint-scope@9.1.2", "", { "dependencies": { "@types/esrecurse": "^4.3.1", "@types/estree": "^1.0.8", "esrecurse": "^4.3.0", "estraverse": "^5.2.0" } }, "sha512-xS90H51cKw0jltxmvmHy2Iai1LIqrfbw57b79w/J7MfvDfkIkFZ+kj6zC3BjtUwh150HsSSdxXZcsuv72miDFQ=="],
@@ -2124,8 +1995,6 @@
"fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="],
"fast-diff": ["fast-diff@1.3.0", "", {}, "sha512-VxPP4NqbUjj6MaAOafWeUn2cXWLcCtljklUtZf0Ind4XQ+QPtmA0b18zZy0jIQx+ExRVCR/ZQpBmik5lXshNsw=="],
"fast-fifo": ["fast-fifo@1.3.2", "", {}, "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ=="],
"fast-json-stable-stringify": ["fast-json-stable-stringify@2.1.0", "", {}, "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw=="],
@@ -2172,8 +2041,6 @@
"fs-constants": ["fs-constants@1.0.0", "", {}, "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow=="],
"fs-monkey": ["fs-monkey@1.1.0", "", {}, "sha512-QMUezzXWII9EV5aTFXW1UBVUO77wYPpjqIF8/AviUCThNeSYZykpoTixUeaNNBwmCev0AMDWMAni+f8Hxb1IFw=="],
"fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="],
"function-bind": ["function-bind@1.1.2", "", {}, "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA=="],
@@ -2214,8 +2081,6 @@
"glob-parent": ["glob-parent@6.0.2", "", { "dependencies": { "is-glob": "^4.0.3" } }, "sha512-XxwI8EOhVQgWp6iDL+3b0r86f4d6AX6zSU55HfB4ydCEuXLXc5FcYeOu+nnGftS4TEju/11rt4KJPTMgbfmv4A=="],
"glob-to-regex.js": ["glob-to-regex.js@1.2.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-QMwlOQKU/IzqMUOAZWubUOT8Qft+Y0KQWnX9nK3ch0CJg0tTp4TvGZsTfudYKv2NzoQSyPcnA6TYeIQ3jGichQ=="],
"globals": ["globals@17.11.0", "", {}, "sha512-Z2I8hM+PbJDXQDq3Icgpzv+mPdwr68iZUU9d5WW4FuXfDUQfkZaZuvjMv42/5crNyw154+9+VWXbYrUgDXbxNw=="],
"globalthis": ["globalthis@1.0.4", "", { "dependencies": { "define-properties": "^1.2.1", "gopd": "^1.0.1" } }, "sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ=="],
@@ -2238,8 +2103,6 @@
"has-bigints": ["has-bigints@1.1.0", "", {}, "sha512-R3pbpkcIqv2Pm3dUwgjclDRVmWpTJW2DcMzcIhEXEx1oh/CEMObMm3KLmRJOdvhM7o4uQBnwr8pzRK2sJWIqfg=="],
"has-flag": ["has-flag@4.0.0", "", {}, "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ=="],
"has-property-descriptors": ["has-property-descriptors@1.0.2", "", { "dependencies": { "es-define-property": "^1.0.0" } }, "sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg=="],
"has-proto": ["has-proto@1.2.0", "", { "dependencies": { "dunder-proto": "^1.0.0" } }, "sha512-KIL7eQPfHQRC8+XluaIw7BHUwwqL19bQn4hzNgdr+1wXoU0KKj6rufu47lhY7KbJR2C6T6+PfyN0Ea7wkSS+qQ=="],
@@ -2256,8 +2119,6 @@
"he": ["he@1.2.0", "", { "bin": { "he": "bin/he" } }, "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw=="],
"heap-js": ["heap-js@2.7.1", "", {}, "sha512-EQfezRg0NCZGNlhlDR3Evrw1FVL2G3LhU7EgPoxufQKruNBSYA8MiRPHeWbU+36o+Fhel0wMwM+sLEiBAlNLJA=="],
"help-me": ["help-me@5.0.0", "", {}, "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg=="],
"hls.js": ["hls.js@1.7.3", "", {}, "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A=="],
@@ -2270,8 +2131,6 @@
"https-proxy-agent": ["https-proxy-agent@7.0.6", "", { "dependencies": { "agent-base": "^7.1.2", "debug": "4" } }, "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw=="],
"hyperdyperid": ["hyperdyperid@1.2.0", "", {}, "sha512-Y93lCzHYgGWdrJ66yIktxiaGULYc6oGiABxhcO5AufBeOyoIdZF7bIfLaOrbM0iGIOXQQgxxRrFEnb+Y6w1n4A=="],
"iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="],
"ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="],
@@ -2360,6 +2219,8 @@
"is-weakset": ["is-weakset@2.0.4", "", { "dependencies": { "call-bound": "^1.0.3", "get-intrinsic": "^1.2.6" } }, "sha512-mfcwb6IzQyOKTs84CQMrOwW4gQcaTOAWJ0zzJCl2WSPDrWk/OzDaImWFH3djXhb24g4eudZfLRozAvPGw4d9hQ=="],
"is-what": ["is-what@4.1.16", "", {}, "sha512-ZhMwEosbFJkA0YhFnNDgTM4ZxDRsS6HqTo7qsZM08fehyRYIYa0yHu5R6mgo1n/8MgaPBXiPimPD77baVFYg+A=="],
"isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="],
"isbot": ["isbot@5.2.2", "", {}, "sha512-iQcBXcd+Rv/pkubRyGh2utW2j1oPG5hZY6TUhVPpqK4G+o3IbxpJNx04hgksjc/N7GK5pEorUxDeg31cFgEk/w=="],
@@ -2374,8 +2235,6 @@
"jackspeak": ["jackspeak@3.4.3", "", { "dependencies": { "@isaacs/cliui": "^8.0.2" }, "optionalDependencies": { "@pkgjs/parseargs": "^0.11.0" } }, "sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw=="],
"jest-worker": ["jest-worker@27.5.1", "", { "dependencies": { "@types/node": "*", "merge-stream": "^2.0.0", "supports-color": "^8.0.0" } }, "sha512-7vuh85V5cdDofPyxn58nrPjBktZo0u9x1g8WtjQol+jZDaE+fhN+cIvTj11GndBnMnyfrUOG1sZQxCdjKh+DKg=="],
"jiti": ["jiti@2.7.0", "", { "bin": { "jiti": "lib/jiti-cli.mjs" } }, "sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ=="],
"jittered-fractional-indexing": ["jittered-fractional-indexing@1.0.1", "", { "dependencies": { "fractional-indexing": "^3.2.0" } }, "sha512-OpKFkVr4hU5ivd1ZCjZfHvVpWekraJvcePcMusBmgBmCVQK5JiRCA+4TT1vAUTLqGD9MkhqFwO0l3QspvlZgzw=="],
@@ -2498,10 +2357,6 @@
"mdast-util-to-string": ["mdast-util-to-string@4.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0" } }, "sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg=="],
"memfs": ["memfs@4.68.2", "", { "dependencies": { "@jsonjoy.com/fs-core": "4.68.2", "@jsonjoy.com/fs-fsa": "4.68.2", "@jsonjoy.com/fs-node": "4.68.2", "@jsonjoy.com/fs-node-builtins": "4.68.2", "@jsonjoy.com/fs-node-to-fsa": "4.68.2", "@jsonjoy.com/fs-node-utils": "4.68.2", "@jsonjoy.com/fs-print": "4.68.2", "@jsonjoy.com/fs-snapshot": "4.68.2", "@jsonjoy.com/json-pack": "^1.11.0", "@jsonjoy.com/util": "^1.9.0", "glob-to-regex.js": "^1.0.1", "thingies": "^2.5.0", "tree-dump": "^1.0.3", "tslib": "^2.0.0" } }, "sha512-Un1ElEBoIdPI9kg0sm3LebVEuEViodfIvlaG8Z1MFXa7ZnR9vWuPKUo3dt/vuWY2ivc05l76B5QOjdgCzAzmGw=="],
"merge-stream": ["merge-stream@2.0.0", "", {}, "sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w=="],
"micromark": ["micromark@4.0.2", "", { "dependencies": { "@types/debug": "^4.0.0", "debug": "^4.0.0", "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-core-commonmark": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-combine-extensions": "^2.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-encode": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA=="],
"micromark-core-commonmark": ["micromark-core-commonmark@2.0.3", "", { "dependencies": { "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-factory-destination": "^2.0.0", "micromark-factory-label": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-factory-title": "^2.0.0", "micromark-factory-whitespace": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-html-tag-name": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg=="],
@@ -2544,16 +2399,12 @@
"micromark-util-types": ["micromark-util-types@2.0.2", "", {}, "sha512-Yw0ECSpJoViF1qTU4DC6NwtC4aWGt1EkzaQB8KPPyCRR8z9TWeV0HbEFGTO+ZY1wB22zmxnJqhPyTpOVCpeHTA=="],
"mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="],
"mime-match": ["mime-match@1.0.2", "", { "dependencies": { "wildcard": "^1.1.0" } }, "sha512-VXp/ugGDVh3eCLOBCiHZMYWQaTNUHv2IJrut+yXA6+JbLPXHglHwfS/5A5L0ll+jkCY7fIzRJcH6OIunF+c6Cg=="],
"minimatch": ["minimatch@10.2.6", "", { "dependencies": { "brace-expansion": "^5.0.8" } }, "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A=="],
"minimist": ["minimist@1.2.8", "", {}, "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA=="],
"minimizer-webpack-plugin": ["minimizer-webpack-plugin@5.8.0", "", { "dependencies": { "@jridgewell/trace-mapping": "^0.3.31", "jest-worker": "^27.4.5", "schema-utils": "^4.3.3", "terser": "^5.51.0" }, "peerDependencies": { "@minify-html/node": "*", "@swc/core": "*", "@swc/css": "*", "@swc/html": "*", "clean-css": "*", "cssnano": "*", "csso": "*", "esbuild": "*", "html-minifier-terser": "*", "lightningcss": "*", "postcss": "*", "uglify-js": "*", "webpack": "^5.1.0" }, "optionalPeers": ["@minify-html/node", "@swc/core", "@swc/css", "@swc/html", "clean-css", "cssnano", "csso", "esbuild", "html-minifier-terser", "lightningcss", "postcss", "uglify-js"] }, "sha512-2cT9+goJfBhtMz+gJqejSf09ClgmYhPhccRb/fb0ztVbixt0BkO8mRuI26FoPPuUNkRk6iEDryWAIouc5W8eJA=="],
"minipass": ["minipass@7.1.3", "", {}, "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A=="],
"mkdirp": ["mkdirp@3.0.1", "", { "bin": { "mkdirp": "dist/cjs/src/bin.js" } }, "sha512-+NsyUUAZDmo6YVHzL/stxSu3t9YS1iljliy3BSDrXJ/dkn1KYdmtZODGGjLcc9XLgVVpH4KshHB8XmZgMhaBXg=="],
@@ -2576,12 +2427,8 @@
"natural-compare": ["natural-compare@1.4.0", "", {}, "sha512-OWND8ei3VtNC9h7V60qff3SVobHr996CTwgxubgyQYEpg290h9J0buyECNNJexkFm5sOajh5G116RYA1c8ZMSw=="],
"neo-async": ["neo-async@2.6.2", "", {}, "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw=="],
"next-themes": ["next-themes@0.4.6", "", { "peerDependencies": { "react": "^16.8 || ^17 || ^18 || ^19 || ^19.0.0-rc", "react-dom": "^16.8 || ^17 || ^18 || ^19 || ^19.0.0-rc" } }, "sha512-pZvgD5L0IEvX5/9GWyHMf3m8BKiVQwsCMHfoFosXtXBMnaS0ZnIJ9ST4b4NqLVKDEm8QBxoNNGNaBv2JNF6XNA=="],
"nexus-rpc": ["nexus-rpc@0.0.3", "", {}, "sha512-ksIBeYHDv6FHAR1h3Vwk0kQ6REhdo1STqDHXMzu/yn/VN/XzgOor2IUjewG6paoA7IgxlIPZoOU/PD78PC6HTg=="],
"node-domexception": ["node-domexception@1.0.0", "", {}, "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ=="],
"node-emoji": ["node-emoji@2.2.0", "", { "dependencies": { "@sindresorhus/is": "^4.6.0", "char-regex": "^1.0.2", "emojilib": "^2.4.0", "skin-tone": "^2.0.0" } }, "sha512-Z3lTE9pLaJF47NyMhd4ww1yFTAP8YhYI8SleJiHzM46Fgpm5cnNzSl9XfzFNqbaz+VlJrIj3fXQ4DeN1Rjm6cw=="],
@@ -2722,8 +2569,6 @@
"prettier": ["prettier@3.9.6", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g=="],
"prettier-linter-helpers": ["prettier-linter-helpers@1.0.1", "", { "dependencies": { "fast-diff": "^1.1.2" } }, "sha512-SxToR7P8Y2lWmv/kTzVLC1t/GDI2WGjMwNhLLE9qtH8Q13C+aEmuRlzDst4Up4s0Wc8sF2M+J57iB3cMLqftfg=="],
"pretty-format": ["pretty-format@27.5.1", "", { "dependencies": { "ansi-regex": "^5.0.1", "ansi-styles": "^5.0.0", "react-is": "^17.0.1" } }, "sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ=="],
"prisma": ["prisma@7.10.0", "", { "dependencies": { "@prisma/config": "7.10.0", "@prisma/dev": "0.24.17", "@prisma/engines": "7.10.0", "@prisma/studio-core": "0.33.0", "mysql2": "3.15.3", "postgres": "3.4.7" }, "peerDependencies": { "better-sqlite3": ">=9.0.0", "typescript": ">=5.4.0" }, "optionalPeers": ["better-sqlite3", "typescript"], "bin": { "prisma": "build/index.js" } }, "sha512-o0ornyJOWgygVAzGCpr8PdXV8EJLHyVGDDUr/voBQt8Azzw8cYTByzzPGcA/m4tCkPcnJA8raEOv2CslsKhPEw=="],
@@ -2832,8 +2677,6 @@
"rou3": ["rou3@0.7.12", "", {}, "sha512-iFE4hLDuloSWcD7mjdCDhx2bKcIsYbtOTpfH5MHHLSKMOUyjqQXTeZVa289uuwEGEKFoE/BAPbhaU4B774nceg=="],
"rxjs": ["rxjs@7.8.2", "", { "dependencies": { "tslib": "^2.1.0" } }, "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA=="],
"safe-array-concat": ["safe-array-concat@1.1.4", "", { "dependencies": { "call-bind": "^1.0.9", "call-bound": "^1.0.4", "get-intrinsic": "^1.3.0", "has-symbols": "^1.1.0", "isarray": "^2.0.5" } }, "sha512-wtZlHyOje6OZTGqAoaDKxFkgRtkF9CnHAVnCHKfuj200wAgL+bSJhdsCD2l0Qx/2ekEXjPWcyKkfGb5CPboslg=="],
"safe-buffer": ["safe-buffer@5.2.1", "", {}, "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ=="],
@@ -2850,14 +2693,14 @@
"scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="],
"schema-utils": ["schema-utils@4.3.3", "", { "dependencies": { "@types/json-schema": "^7.0.9", "ajv": "^8.9.0", "ajv-formats": "^2.1.1", "ajv-keywords": "^5.1.0" } }, "sha512-eflK8wEtyOE6+hsaRVPxvUKYCpRgzLqDTb8krvAsRIwOGlHoSgYLgBXoubGgLd2fT41/OUYdb48v4k4WWHQurA=="],
"secure-json-parse": ["secure-json-parse@4.1.0", "", {}, "sha512-l4KnYfEyqYJxDwlNVyRfO2E4NTHfMKAWdUuA8J0yve2Dz/E/PdBepY03RvyJpssIpRFwJoCD55wA+mEDs6ByWA=="],
"semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="],
"seq-queue": ["seq-queue@0.0.5", "", {}, "sha512-hr3Wtp/GZIc/6DAGPDcV4/9WoZhjrkXsi5B/07QgX8tsdc6ilr7BFM6PM6rbdAX1kFSDYeZGLipIZZKyQP0O5Q=="],
"serialize-error": ["serialize-error@8.1.0", "", { "dependencies": { "type-fest": "^0.20.2" } }, "sha512-3NnuWfM6vBYoy5gZFvHiYsVbafvI9vZv/+jlIigFn4oP4zjNPK3LhcY0xSCgeb1a5L8jO71Mit9LlNoi2UfDDQ=="],
"seroval": ["seroval@1.6.4", "", {}, "sha512-LErWMNS2RRFdu2RMA5u/PA59/IWs0XsikyEXGQ2/36iEWFrdG0ABmg17E17cikrv76891kOAMq3TkTFXpwAHXw=="],
"seroval-plugins": ["seroval-plugins@1.6.4", "", { "peerDependencies": { "seroval": "^1.0" } }, "sha512-R0f1U9hmn38+dFMz6b6ab8lwucmw4AtiY7St+JPWudy1dm+Bs3g884nyrsH9Cy6rKpZKLYayXuMda9GZ/fl8JQ=="],
@@ -2894,14 +2737,8 @@
"sonner": ["sonner@2.0.8", "", { "peerDependencies": { "@types/react": "^18.0.0 || ^19.0.0", "react": "^18.0.0 || ^19.0.0 || ^19.0.0-rc", "react-dom": "^18.0.0 || ^19.0.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-UM/ByIoFra8yzV75n1o0Puu0bw5U/9UNnDacrJNspekBewIfsQ3D6ez1nvlWpt7aTsO6rujQtifBpycwIivqlg=="],
"source-map": ["source-map@0.7.6", "", {}, "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
"source-map-loader": ["source-map-loader@5.0.0", "", { "dependencies": { "iconv-lite": "^0.6.3", "source-map-js": "^1.0.2" }, "peerDependencies": { "webpack": "^5.72.1" } }, "sha512-k2Dur7CbSLcAH73sBcIkV5xjPV4SzqO1NJ7+XaQl8if3VODDUj3FNchNGpqgJSKbvUfJuhVdv8K2Eu8/TNl2eA=="],
"source-map-support": ["source-map-support@0.5.21", "", { "dependencies": { "buffer-from": "^1.0.0", "source-map": "^0.6.0" } }, "sha512-uBHU3L3czsIyYXKX88fdrGovxdSCoTGDRZ6SYXtSRxLZUzHg5P/66Ht6uoUlHu9EZod+inXhKo3qQgwXUT/y1w=="],
"space-separated-tokens": ["space-separated-tokens@2.0.2", "", {}, "sha512-PEGlAwrG8yXGXRjW32fGbg66JAlOAwbObuqVoJpv/mRgoWDQfgH1wDPvtzWyUSNAXBGSk8h755YDbbcEy3SH2Q=="],
"split-ca": ["split-ca@1.0.1", "", {}, "sha512-Q5thBSxp5t8WPTTJQS59LrGqOZqOsrhDGDVm8azCqIBjSBd7nd9o2PM+mDulQQkh8h//4U6hFZnc/mul8t5pWQ=="],
@@ -2952,14 +2789,10 @@
"style-to-object": ["style-to-object@1.0.14", "", { "dependencies": { "inline-style-parser": "0.2.7" } }, "sha512-LIN7rULI0jBscWQYaSswptyderlarFkjQ+t79nzty8tcIAceVomEVlLzH5VP4Cmsv6MtKhs7qaAiwlcp+Mgaxw=="],
"supports-color": ["supports-color@8.1.1", "", { "dependencies": { "has-flag": "^4.0.0" } }, "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q=="],
"superjson": ["superjson@1.13.3", "", { "dependencies": { "copy-anything": "^3.0.2" } }, "sha512-mJiVjfd2vokfDxsQPOwJ/PtanO87LhpYY88ubI5dUB1Ab58Txbyje3+jpm+/83R/fevaq/107NNhtYBLuoTrFg=="],
"supports-preserve-symlinks-flag": ["supports-preserve-symlinks-flag@1.0.0", "", {}, "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w=="],
"swc-loader": ["swc-loader@0.2.7", "", { "dependencies": { "@swc/counter": "^0.1.3" }, "peerDependencies": { "@swc/core": "^1.2.147", "webpack": ">=2" } }, "sha512-nwYWw3Fh9ame3Rtm7StS9SBLpHRRnYcK7bnpF3UKZmesAK0gw2/ADvlURFAINmPvKtDLzp+GBiP9yLoEjg6S9w=="],
"synckit": ["synckit@0.11.13", "", { "dependencies": { "@pkgr/core": "^0.3.6" } }, "sha512-eNRKgb3z66Yp3D2CixVujOUvXLFUTij/zVnV8KRyvFdQwpz7I5DS8UfRkTeLzb64u+dkzDSdelE24izu+zSSUg=="],
"tabbable": ["tabbable@6.5.0", "", {}, "sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA=="],
"tailwind-merge": ["tailwind-merge@3.6.0", "", {}, "sha512-uxL7qAVQriqRQPAyK3pj66VqskWqoZ37PW94jwOTwNfq/z9oyu1V+eqrZqtR2+fCiXdYOZe/Modt8GtvqNzu+w=="],
@@ -2976,14 +2809,10 @@
"teex": ["teex@1.0.1", "", { "dependencies": { "streamx": "^2.12.5" } }, "sha512-eYE6iEI62Ni1H8oIa7KlDU6uQBtqr4Eajni3wX7rpfXD8ysFx8z0+dri+KWEPWpBsxXfxu58x/0jvTVT1ekOSg=="],
"terser": ["terser@5.51.2", "", { "dependencies": { "@jridgewell/source-map": "^0.3.3", "acorn": "^8.15.0", "commander": "^2.20.0", "source-map-support": "~0.5.20" }, "bin": { "terser": "bin/terser" } }, "sha512-bWnjSNscmuI+GJze6ZupnHP8G/cTcsJF+bXCeQknk2SHQsgbNJnLrqiH9jZ2W4STPVXH2mDKKRX3iwPhc9Cn/Q=="],
"testcontainers": ["testcontainers@12.1.0", "", { "dependencies": { "@balena/dockerignore": "^1.0.2", "@types/dockerode": "^4.0.1", "archiver": "^7.0.1", "async-lock": "^1.4.1", "byline": "^5.0.0", "debug": "^4.4.3", "docker-compose": "^1.4.2", "dockerode": "^5.0.1", "get-port": "^5.1.1", "proper-lockfile": "^4.1.2", "properties-reader": "^3.0.1", "ssh-remote-port-forward": "^1.0.4", "tar-fs": "^3.1.3", "tmp": "^0.2.7", "undici": "^8.9.0" } }, "sha512-YjDLqIITuhGLMnM10yhg3oV6lIG5IMpz1R1DPBZoOOks83q7i7IVpeSWRTiyl7roozjiyLmwIoLK/KY8OnZmIA=="],
"text-decoder": ["text-decoder@1.2.7", "", { "dependencies": { "b4a": "^1.6.4" } }, "sha512-vlLytXkeP4xvEq2otHeJfSQIRyWxo/oZGEbXrtEEF9Hnmrdly59sUbzZ/QgyWuLYHctCHxFF4tRQZNQ9k60ExQ=="],
"thingies": ["thingies@2.6.1", "", { "peerDependencies": { "tslib": "^2" } }, "sha512-cV/CMGTK3M4MlnJ/0At6ismOw/A0EEniDNScajjz/Br3c1sqE72YD01rGpPTKwd27wAxI5Pr+6+0w8yofzFRYw=="],
"thread-stream": ["thread-stream@4.2.0", "", { "dependencies": { "real-require": "^1.0.0" } }, "sha512-e2zZ96wSChazBsbENf/Pcm/4swHt2cEKQ92rhUjkL9GCKiTDJIaTBenjE/m9DXi0QBmTMDkFDdOomUy20A1tDQ=="],
"tiny-inflate": ["tiny-inflate@1.0.3", "", {}, "sha512-pkY1fj1cKHb2seWDy0B16HeWyczlJA9/WW3u3c4z/NiWDsO3DOU5D7nhTLE9CF0yXv/QZFY7sEJmj24dK+Rrqw=="],
@@ -3000,8 +2829,6 @@
"tmp": ["tmp@0.2.7", "", {}, "sha512-e0votIpp4Uo2AJYSzVHV6xCcawuiez3DzqDAbrTc3YxBkplN6e+dM13ZeIcZnDg/QpSuU2zfZ3rzwY8ukEnaXw=="],
"tree-dump": ["tree-dump@1.1.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-rMuvhU4MCDbcbnleZTFezWsaZXRFemSqAM+7jPnzUl1fo9w3YEKOxAeui0fz3OI4EU4hf23iyA7uQRVko+UaBA=="],
"trim-lines": ["trim-lines@3.0.1", "", {}, "sha512-kRj8B+YHZCc9kQYdWfJB2/oUl9rA99qbowYYBtr4ui4mZyAQ2JpvVBd/6U2YloATfqBhBTSMhTpgBHtU0Mf3Rg=="],
"trough": ["trough@2.2.0", "", {}, "sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw=="],
@@ -3022,6 +2849,8 @@
"type-check": ["type-check@0.4.0", "", { "dependencies": { "prelude-ls": "^1.2.1" } }, "sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew=="],
"type-fest": ["type-fest@0.20.2", "", {}, "sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ=="],
"typebox": ["typebox@1.3.22", "", {}, "sha512-YoX6QdtuJR/+YpP/qXf9gJXTxvSuiUlKYgW3AilsOSLlsZe/VH6ILR5vhoJM5QXY//uzIRE7NYij3h2+nG9vRQ=="],
"typed-array-buffer": ["typed-array-buffer@1.0.3", "", { "dependencies": { "call-bound": "^1.0.3", "es-errors": "^1.3.0", "is-typed-array": "^1.1.14" } }, "sha512-nAYYwfY3qnzX30IkA6AQZjVbtK6duGontcQm1WSG1MD94YLqK0515GNApXkoxKOWMusVssAHWLh9SeaoefYFGw=="],
@@ -3054,8 +2883,6 @@
"unified": ["unified@11.0.5", "", { "dependencies": { "@types/unist": "^3.0.0", "bail": "^2.0.0", "devlop": "^1.0.0", "extend": "^3.0.0", "is-plain-obj": "^4.0.0", "trough": "^2.0.0", "vfile": "^6.0.0" } }, "sha512-xKvGhPWw3k84Qjh8bI3ZeJjqnyadK+GEFtazSfZv/rKeTkTjOJho6mFqh2SM96iIcZokxiOpg78GazTSg8+KHA=="],
"unionfs": ["unionfs@4.6.0", "", { "dependencies": { "fs-monkey": "^1.0.0" } }, "sha512-fJAy3gTHjFi5S3TP5EGdjs/OUMFFvI/ady3T8qVuZfkv8Qi8prV/Q8BuFEgODJslhZTT2z2qdD2lGdee9qjEnA=="],
"unist-util-is": ["unist-util-is@6.0.1", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g=="],
"unist-util-position": ["unist-util-position@5.0.0", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-fucsC7HjXvkB5R3kTCO7kUjRdrS0BJt3M/FPxmHMBOm8JQi2BsHAHFsy27E0EolP8rp0NzXsJ+jNPyDWvOJZPA=="],
@@ -3084,7 +2911,7 @@
"util-deprecate": ["util-deprecate@1.0.2", "", {}, "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw=="],
"uuid": ["uuid@11.1.1", "", { "bin": { "uuid": "dist/esm/bin/uuid" } }, "sha512-vIYxrBCC/N/K+Js3qSN88go7kIfNPssr/hHCesKCQNAjmgvYS2oqr69kIufEG+O4+PfezOH4EbIeHCfFov8ZgQ=="],
"uuid": ["uuid@14.0.2", "", { "bin": { "uuid": "dist-node/bin/uuid" } }, "sha512-xZe/16rV4aa+HGSOCiY2YeLT1OybRLrrkL/Rqaq7p7GMVXjFh+6wN4oMYgjFmnSnhY8t6Xpdl2l9qmnHYuMHwQ=="],
"valibot": ["valibot@1.4.2", "", { "peerDependencies": { "typescript": ">=5" }, "optionalPeers": ["typescript"] }, "sha512-gjdCvJ6d3RyHAneqxMYMW9QMCwYMb3jpOO0IyHZV1bnRHFBHrX3VkIILt5XYR0WhwHiH7Mty8ovuPZ/O3gamrg=="],
@@ -3102,14 +2929,8 @@
"vitest": ["vitest@4.1.11", "", { "dependencies": { "@vitest/expect": "4.1.11", "@vitest/mocker": "4.1.11", "@vitest/pretty-format": "4.1.11", "@vitest/runner": "4.1.11", "@vitest/snapshot": "4.1.11", "@vitest/spy": "4.1.11", "@vitest/utils": "4.1.11", "es-module-lexer": "^2.0.0", "expect-type": "^1.3.0", "magic-string": "^0.30.21", "obug": "^2.1.1", "pathe": "^2.0.3", "picomatch": "^4.0.3", "std-env": "^4.0.0-rc.1", "tinybench": "^2.9.0", "tinyexec": "^1.0.2", "tinyglobby": "^0.2.15", "tinyrainbow": "^3.1.0", "vite": "^6.0.0 || ^7.0.0 || ^8.0.0", "why-is-node-running": "^2.3.0" }, "peerDependencies": { "@edge-runtime/vm": "*", "@opentelemetry/api": "^1.9.0", "@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0", "@vitest/browser-playwright": "4.1.11", "@vitest/browser-preview": "4.1.11", "@vitest/browser-webdriverio": "4.1.11", "@vitest/coverage-istanbul": "4.1.11", "@vitest/coverage-v8": "4.1.11", "@vitest/ui": "4.1.11", "happy-dom": "*", "jsdom": "*" }, "optionalPeers": ["@edge-runtime/vm", "@opentelemetry/api", "@types/node", "@vitest/browser-playwright", "@vitest/browser-preview", "@vitest/browser-webdriverio", "@vitest/coverage-istanbul", "@vitest/coverage-v8", "@vitest/ui", "happy-dom", "jsdom"], "bin": { "vitest": "./vitest.mjs" } }, "sha512-fhACrNXUidIbGSBr5FlbuBkO7VWC1ZyLl0DO4CU2DrQoAPxX84Ysxs+HeGQpii5lZWV1Q4gBZTTu49mF+A6Edw=="],
"watchpack": ["watchpack@2.5.2", "", { "dependencies": { "graceful-fs": "^4.1.2" } }, "sha512-6i/00NBjP4yGPs+caKSyRfpTF/8Torsu0MOW3mMzIbhgISFder8i7xbqgHlLMwJrdiN8ndBV3UA1/AfzPSr+jg=="],
"web-streams-polyfill": ["web-streams-polyfill@3.3.3", "", {}, "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw=="],
"webpack": ["webpack@5.110.1", "", { "dependencies": { "@types/estree": "^1.0.8", "@types/json-schema": "^7.0.15", "@webassemblyjs/ast": "^1.14.1", "@webassemblyjs/wasm-edit": "^1.14.1", "@webassemblyjs/wasm-parser": "^1.14.1", "acorn": "^8.16.0", "browserslist": "^4.28.1", "chrome-trace-event": "^1.0.2", "enhanced-resolve": "^5.24.4", "es-module-lexer": "^2.1.0", "events": "^3.2.0", "graceful-fs": "^4.2.11", "mime-db": "^1.54.0", "minimizer-webpack-plugin": "^5.7.0", "neo-async": "^2.6.2", "schema-utils": "^4.3.3", "tapable": "^2.3.0", "watchpack": "^2.5.2", "webpack-sources": "^3.5.1" }, "peerDependencies": { "webpack-cli": "*" }, "optionalPeers": ["webpack-cli"], "bin": { "webpack": "bin/webpack.js" } }, "sha512-gInQB+jxXxgnZyvPwuzT5NGQmECDqeu85oxcrjinrYHqPoBex0hCAN2SFTJVyPVrK0Pq9E44VFP+e89fAc10/w=="],
"webpack-sources": ["webpack-sources@3.5.1", "", {}, "sha512-jyuiGJdtvY434z5bUZrjz67v76/ePNvFZTp9Mdz29IlH4+GPsgyGjiv0fKI+M7BdkU6ADjulUcKAd3tUK3WlEw=="],
"webpack-virtual-modules": ["webpack-virtual-modules@0.6.2", "", {}, "sha512-66/V2i5hQanC51vBQKPH4aI8NMAcBW59FVBs+rC7eGHupMyfn34q7rZIE+ETlJ+XTevqfUhVVBgSUNSW2flEUQ=="],
"whatwg-mimetype": ["whatwg-mimetype@3.0.0", "", {}, "sha512-nt+N2dzIutVRxARx1nghPKGv1xHikU7HKdfafKkLNLindmPU/ch3U31NOCGGA/dmPcmb1VlofO0vnKAcsm0o/Q=="],
@@ -3190,6 +3011,8 @@
"@base-ui/utils/reselect": ["reselect@5.3.0", "", {}, "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg=="],
"@dbos-inc/dbos-sdk/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
"@earendil-works/pi-agent-core/ignore": ["ignore@7.0.5", "", {}, "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg=="],
"@earendil-works/pi-agent-core/typebox": ["typebox@1.1.38", "", {}, "sha512-pZ0aQPmMmXoUvSbeuWf/Hzsc+avNw/Zd6VeE8CFgkVGWyuHPJvqeJJDeJqLve+K70LvjYIoleGcoJHPT17cWoA=="],
@@ -3204,22 +3027,12 @@
"@inlang/sdk/@sinclair/typebox": ["@sinclair/typebox@0.31.30", "", {}, "sha512-MGsM7bVmHg3sUKCphlu3SGQ+T+5JTbygZWYArfKQCW5anJeACHeqLhcnf+R7XlCWZ+QR3C8JTBx3kCKJTN69ow=="],
"@inlang/sdk/uuid": ["uuid@14.0.2", "", { "bin": { "uuid": "dist-node/bin/uuid" } }, "sha512-xZe/16rV4aa+HGSOCiY2YeLT1OybRLrrkL/Rqaq7p7GMVXjFh+6wN4oMYgjFmnSnhY8t6Xpdl2l9qmnHYuMHwQ=="],
"@isaacs/cliui/string-width": ["string-width@5.1.2", "", { "dependencies": { "eastasianwidth": "^0.2.0", "emoji-regex": "^9.2.2", "strip-ansi": "^7.0.1" } }, "sha512-HnLOCR3vjcY8beoNLtcjZ5/nxn2afmME6lhrDrebokqMap+XbeW8n9TXpPDOqdGK5qcI3oT0GKTW6wC7EMiVqA=="],
"@isaacs/cliui/strip-ansi": ["strip-ansi@7.2.0", "", { "dependencies": { "ansi-regex": "^6.2.2" } }, "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w=="],
"@isaacs/cliui/wrap-ansi": ["wrap-ansi@8.1.0", "", { "dependencies": { "ansi-styles": "^6.1.0", "string-width": "^5.0.1", "strip-ansi": "^7.0.1" } }, "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/json-pack": ["@jsonjoy.com/json-pack@17.67.0", "", { "dependencies": { "@jsonjoy.com/base64": "17.67.0", "@jsonjoy.com/buffers": "17.67.0", "@jsonjoy.com/codegen": "17.67.0", "@jsonjoy.com/json-pointer": "17.67.0", "@jsonjoy.com/util": "17.67.0", "hyperdyperid": "^1.2.0", "thingies": "^2.5.0", "tree-dump": "^1.1.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-t0ejURcGaZsn1ClbJ/3kFqSOjlryd92eQY465IYrezsXmPcfHPE/av4twRSxf6WE+TkZgLY+71vCZbiIiFKA/w=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/util": ["@jsonjoy.com/util@17.67.0", "", { "dependencies": { "@jsonjoy.com/buffers": "17.67.0", "@jsonjoy.com/codegen": "17.67.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-6+8xBaz1rLSohlGh68D1pdw3AwDi9xydm8QNlAFkvnavCJYSze+pxoW2VKP8p308jtlMRLs5NTHfPlZLd4w7ew=="],
"@jsonjoy.com/json-pack/@jsonjoy.com/buffers": ["@jsonjoy.com/buffers@1.2.1", "", { "peerDependencies": { "tslib": "2" } }, "sha512-12cdlDwX4RUM3QxmUbVJWqZ/mrK6dFQH4Zxq6+r1YXKXYBNgZXndx2qbCJwh3+WWkCSn67IjnlG3XYTvmvYtgA=="],
"@jsonjoy.com/util/@jsonjoy.com/buffers": ["@jsonjoy.com/buffers@1.2.1", "", { "peerDependencies": { "tslib": "2" } }, "sha512-12cdlDwX4RUM3QxmUbVJWqZ/mrK6dFQH4Zxq6+r1YXKXYBNgZXndx2qbCJwh3+WWkCSn67IjnlG3XYTvmvYtgA=="],
"@prisma/dev/std-env": ["std-env@3.10.0", "", {}, "sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg=="],
"@prisma/engines/@prisma/get-platform": ["@prisma/get-platform@7.10.0", "", { "dependencies": { "@prisma/debug": "7.10.0" } }, "sha512-0bra1LFYi8xNw0yqV62bHJNQk4BKleOngZiqPWIQc7a3+9q6rqsnqJ15BuepP8943PFMvtmgnP52juZsyYkA6w=="],
@@ -3234,10 +3047,6 @@
"@reduxjs/toolkit/reselect": ["reselect@5.3.0", "", {}, "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg=="],
"@shumai/core/@temporalio/activity": ["@temporalio/activity@1.24.0", "", { "dependencies": { "@temporalio/client": "1.24.0", "@temporalio/common": "1.24.0" } }, "sha512-eBBho/r9XHe2KczYGzmaSp6e9+6c9ta/GGOBldIBr00D/cV34kJ76iAY3Ue1pQTaP7YaX3phMuOz07uUg/IKyA=="],
"@shumai/core/@temporalio/workflow": ["@temporalio/workflow@1.24.0", "", { "dependencies": { "@temporalio/common": "1.24.0", "@temporalio/proto": "1.24.0", "nexus-rpc": "^0.0.3" } }, "sha512-pIA9nxkDCCbBnNuMSffQ44HWHYe3zXFfrDpDLwzh/FUSx8uLbsBM8jwf5oUMf+5eV/qVW9PknZhOu9IFO4XnPA=="],
"@shumai/webui/@types/react-dom": ["@types/react-dom@19.2.5", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-fMPwH9v7r/pp43yUd2/Mbiex5KouJwwR3dzHkhLREUC6764VyDsqxhAxv6OFEYR1RhjOyD1naqba8ECDBe7ZQg=="],
"@spectrum-web-components/action-button/@spectrum-web-components/icon": ["@spectrum-web-components/icon@0.37.0", "", { "dependencies": { "@spectrum-web-components/base": "^0.37.0", "@spectrum-web-components/iconset": "^0.37.0" } }, "sha512-DZ1yOKHxNQygzIJ7u9akYComDnMmxuTzfmYxxt9sHdUXmc1NbMkCJqEaAG/LYsdb+hc4xWaDTT9SzArLKZ/btA=="],
@@ -3356,14 +3165,6 @@
"@tanstack/router-plugin/unplugin": ["unplugin@3.3.0", "", { "dependencies": { "@jridgewell/remapping": "^2.3.5", "picomatch": "^4.0.4", "webpack-virtual-modules": "^0.6.2" }, "peerDependencies": { "@farmfe/core": "*", "@rspack/core": "*", "bun-types-no-globals": "*", "esbuild": "*", "rolldown": "*", "rollup": "*", "unloader": "*", "vite": "*", "webpack": "*" }, "optionalPeers": ["@farmfe/core", "@rspack/core", "bun-types-no-globals", "esbuild", "rolldown", "rollup", "unloader", "vite", "webpack"] }, "sha512-qa66K+crbfyE6JK10GjvbJeRrOsuC/JpbnHctfyp/i4oBTxWOzJfRZyDiOk1PtErMFRu8JhsU/wPvOdBNWe5Rg=="],
"@temporalio/common/ms": ["ms@3.0.0-canary.1", "", {}, "sha512-kh8ARjh8rMN7Du2igDRO9QJnqCb2xYTJxyQYK7vJJS4TvLLmsbyhiKpSW+t+y26gyOyMd0riphX0GeWKU3ky5g=="],
"@temporalio/common/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@temporalio/proto/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@temporalio/worker/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@types/ssh2/@types/node": ["@types/node@18.19.130", "", { "dependencies": { "undici-types": "~5.26.4" } }, "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg=="],
"@typescript-eslint/eslint-plugin/ignore": ["ignore@7.0.6", "", {}, "sha512-BAg6QkE8W+TuQLrrw0Ugr7HegXduRuuj8/ti2kSOc+jz1dmx8/WNcjr6XGnq5YpDWxFwwaavqD0+jIUOKelTsw=="],
@@ -3390,10 +3191,6 @@
"@visx/vendor/d3-format": ["d3-format@3.1.0", "", {}, "sha512-YyUI6AEuY/Wpt8KWLgZHsIU86atmikuoOmCfommt0LYHiQSPjvX2AcFc38PX0CBpr2RCyZhjex+NS/LPOv6YqA=="],
"ajv-formats/ajv": ["ajv@8.20.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA=="],
"ajv-keywords/ajv": ["ajv@8.20.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA=="],
"bcrypt-pbkdf/tweetnacl": ["tweetnacl@0.14.5", "", {}, "sha512-KXXFFdAbFXY4geFIwoyNK+f5Z1b7swfXABfL7HXCmoIWMKU3dmS26672A4EeQtDzLKy7SXmfBu51JolvEKwtGA=="],
"better-call/@better-auth/utils": ["@better-auth/utils@0.4.2", "", { "dependencies": { "@noble/hashes": "^2.0.1" } }, "sha512-AUxrvu+HaaODsUyzDxFgwd/8RZ1yZaYo42LXKSrU2oGgR38pS1ij8nqQKNgtTWoYGpNevNXtCfgTy6loHveW9A=="],
@@ -3446,18 +3243,10 @@
"readdir-glob/minimatch": ["minimatch@5.1.9", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-7o1wEA2RyMP7Iu7GNba9vc0RWWGACJOCZBJX2GJWip0ikV+wcOsgVuY9uE8CPiyQhkGFSlhuSkZPavN7u1c2Fw=="],
"schema-utils/ajv": ["ajv@8.20.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA=="],
"sharp/semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="],
"source-map-loader/iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="],
"source-map-support/source-map": ["source-map@0.6.1", "", {}, "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g=="],
"ssh-remote-port-forward/@types/ssh2": ["@types/ssh2@0.5.52", "", { "dependencies": { "@types/node": "*", "@types/ssh2-streams": "*" } }, "sha512-lbLLlXxdCZOSJMCInKH2+9V/77ET2J6NPQHpFI0kda61Dd1KglJs+fPQBchizmzYSOJBgdTajhPqBO1xxLywvg=="],
"terser/commander": ["commander@2.20.3", "", {}, "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ=="],
"thread-stream/real-require": ["real-require@1.0.0", "", {}, "sha512-P4nbQYQfePJxRSmY+v/KINxVucm4NF3p3s7pJveMTtom52FR4YGltUQLB8idDXwDDWW+eYrWDFbuzUnjoWHF7g=="],
"wrap-ansi/ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="],
@@ -3472,14 +3261,6 @@
"@isaacs/cliui/wrap-ansi/ansi-styles": ["ansi-styles@6.2.3", "", {}, "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/json-pack/@jsonjoy.com/base64": ["@jsonjoy.com/base64@17.67.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-5SEsJGsm15aP8TQGkDfJvz9axgPwAEm98S5DxOuYe8e1EbfajcDmgeXXzccEjh+mLnjqEKrkBdjHWS5vFNwDdw=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/json-pack/@jsonjoy.com/codegen": ["@jsonjoy.com/codegen@17.67.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-idnkUplROpdBOV0HMcwhsCUS5TRUi9poagdGs70A6S4ux9+/aPuKbh8+UYRTLYQHtXvAdNfQWXDqZEx5k4Dj2Q=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/json-pack/@jsonjoy.com/json-pointer": ["@jsonjoy.com/json-pointer@17.67.0", "", { "dependencies": { "@jsonjoy.com/util": "17.67.0" }, "peerDependencies": { "tslib": "2" } }, "sha512-+iqOFInH+QZGmSuaybBUNdh7yvNrXvqR+h3wjXm0N/3JK1EyyFAeGJvqnmQL61d1ARLlk/wJdFKSL+LHJ1eaUA=="],
"@jsonjoy.com/fs-snapshot/@jsonjoy.com/util/@jsonjoy.com/codegen": ["@jsonjoy.com/codegen@17.67.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-idnkUplROpdBOV0HMcwhsCUS5TRUi9poagdGs70A6S4ux9+/aPuKbh8+UYRTLYQHtXvAdNfQWXDqZEx5k4Dj2Q=="],
"@prisma/streams-local/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/primitive": ["@radix-ui/primitive@1.1.3", "", {}, "sha512-JTF99U/6XIjCBo0wqkU5sK10glYe27MRRsfwoiq5zzOEZLHU3A3KCMa5X/azekYRCJ0HlwI0crAXS/5dEHTzDg=="],
@@ -3488,14 +3269,6 @@
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/react-use-controllable-state": ["@radix-ui/react-use-controllable-state@1.2.2", "", { "dependencies": { "@radix-ui/react-use-effect-event": "0.0.2", "@radix-ui/react-use-layout-effect": "1.1.1" }, "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-BjasUjixPFdS+NKkypcyyN5Pmg83Olst0+c6vGov0diwTEo6mgdqVR6hxcEgFuh4QrAs7Rc+9KuGJ9TVCj0Zzg=="],
"@shumai/core/@temporalio/activity/@temporalio/client": ["@temporalio/client@1.24.0", "", { "dependencies": { "@grpc/grpc-js": "^1.12.4", "@temporalio/common": "1.24.0", "@temporalio/proto": "1.24.0", "abort-controller": "^3.0.0", "long": "^5.3.2", "nexus-rpc": "^0.0.3", "uuid": "^11.1.0" } }, "sha512-/GGg6ZYl+pN/bQEvq414CyjDvyM+MM1WsMzi/7SCYb3UlivVXx9nOmH/6BGx8OfYFayp11fkmfSh6rt7vW/ePQ=="],
"@shumai/core/@temporalio/activity/@temporalio/common": ["@temporalio/common@1.24.0", "", { "dependencies": { "@temporalio/proto": "1.24.0", "long": "^5.3.2", "ms": "3.0.0-canary.1", "nexus-rpc": "^0.0.3", "protobufjs": "^8.7.1" } }, "sha512-jXp77fXwynBcVNc/CaGMT33TVYnhcN26S959HDv7Q71oc0rN6d8RHWR5z0nRCDAu5CQwZQRRhVK0TRdyAiEg7Q=="],
"@shumai/core/@temporalio/workflow/@temporalio/common": ["@temporalio/common@1.24.0", "", { "dependencies": { "@temporalio/proto": "1.24.0", "long": "^5.3.2", "ms": "3.0.0-canary.1", "nexus-rpc": "^0.0.3", "protobufjs": "^8.7.1" } }, "sha512-jXp77fXwynBcVNc/CaGMT33TVYnhcN26S959HDv7Q71oc0rN6d8RHWR5z0nRCDAu5CQwZQRRhVK0TRdyAiEg7Q=="],
"@shumai/core/@temporalio/workflow/@temporalio/proto": ["@temporalio/proto@1.24.0", "", { "dependencies": { "long": "^5.3.2", "protobufjs": "^8.7.1" } }, "sha512-vhlhB9GpoyJx+78r3lDahHywpdri6LKx7kt7djLIw6XzHyb8FQTIi4lX8DSJBqKp3+HVfL1CLwiqO26KFRzg/g=="],
"@spectrum-web-components/action-button/@spectrum-web-components/icon/@spectrum-web-components/iconset": ["@spectrum-web-components/iconset@0.37.0", "", { "dependencies": { "@spectrum-web-components/base": "^0.37.0" } }, "sha512-s1TBjWeQtGdGf6n+r+aG2glv77IgR0pNs0HLusGyYt6jzWFkP276fE7Neqq/2XvG9a9zvAwngffnWS3td/Gtmg=="],
"@spectrum-web-components/action-group/@spectrum-web-components/icons-workflow/@spectrum-web-components/icon": ["@spectrum-web-components/icon@0.37.0", "", { "dependencies": { "@spectrum-web-components/base": "^0.37.0", "@spectrum-web-components/iconset": "^0.37.0" } }, "sha512-DZ1yOKHxNQygzIJ7u9akYComDnMmxuTzfmYxxt9sHdUXmc1NbMkCJqEaAG/LYsdb+hc4xWaDTT9SzArLKZ/btA=="],
@@ -3580,10 +3353,6 @@
"@visx/vendor/@types/d3-scale/@types/d3-time": ["@types/d3-time@3.0.4", "", {}, "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g=="],
"ajv-formats/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"ajv-keywords/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"dockerode/tar-fs/tar-stream": ["tar-stream@2.2.0", "", { "dependencies": { "bl": "^4.0.3", "end-of-stream": "^1.4.1", "fs-constants": "^1.0.0", "inherits": "^2.0.3", "readable-stream": "^3.1.1" } }, "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ=="],
"eslint-plugin-react/minimatch/brace-expansion": ["brace-expansion@1.1.18", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw=="],
@@ -3598,28 +3367,12 @@
"readdir-glob/minimatch/brace-expansion": ["brace-expansion@2.1.4", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg=="],
"schema-utils/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/react-primitive/@radix-ui/react-slot": ["@radix-ui/react-slot@1.2.3", "", { "dependencies": { "@radix-ui/react-compose-refs": "1.1.2" }, "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-aeNmHnBxbi2St0au6VBVC7JXFlhLlOnvIIlePNniyUNAClzmtAUEY8/pBiK3iHjufOlwA+c20/8jngo7xcrg8A=="],
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/react-use-controllable-state/@radix-ui/react-use-effect-event": ["@radix-ui/react-use-effect-event@0.0.2", "", { "dependencies": { "@radix-ui/react-use-layout-effect": "1.1.1" }, "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-Qp8WbZOBe+blgpuUT+lw2xheLP8q0oatc9UpmiemEICxGvFLYmHm9QowVZGHtJlGbS6A6yJ3iViad/2cVjnOiA=="],
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/react-use-controllable-state/@radix-ui/react-use-layout-effect": ["@radix-ui/react-use-layout-effect@1.1.1", "", { "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-RbJRS4UWQFkzHTTwVymMTUv8EqYhOp8dOOviLj2ugtTiXRaRQS7GLGxZTLL1jWhMeoSCf5zmcZkqTl9IiYfXcQ=="],
"@shumai/core/@temporalio/activity/@temporalio/client/@temporalio/proto": ["@temporalio/proto@1.24.0", "", { "dependencies": { "long": "^5.3.2", "protobufjs": "^8.7.1" } }, "sha512-vhlhB9GpoyJx+78r3lDahHywpdri6LKx7kt7djLIw6XzHyb8FQTIi4lX8DSJBqKp3+HVfL1CLwiqO26KFRzg/g=="],
"@shumai/core/@temporalio/activity/@temporalio/common/@temporalio/proto": ["@temporalio/proto@1.24.0", "", { "dependencies": { "long": "^5.3.2", "protobufjs": "^8.7.1" } }, "sha512-vhlhB9GpoyJx+78r3lDahHywpdri6LKx7kt7djLIw6XzHyb8FQTIi4lX8DSJBqKp3+HVfL1CLwiqO26KFRzg/g=="],
"@shumai/core/@temporalio/activity/@temporalio/common/ms": ["ms@3.0.0-canary.1", "", {}, "sha512-kh8ARjh8rMN7Du2igDRO9QJnqCb2xYTJxyQYK7vJJS4TvLLmsbyhiKpSW+t+y26gyOyMd0riphX0GeWKU3ky5g=="],
"@shumai/core/@temporalio/activity/@temporalio/common/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@shumai/core/@temporalio/workflow/@temporalio/common/ms": ["ms@3.0.0-canary.1", "", {}, "sha512-kh8ARjh8rMN7Du2igDRO9QJnqCb2xYTJxyQYK7vJJS4TvLLmsbyhiKpSW+t+y26gyOyMd0riphX0GeWKU3ky5g=="],
"@shumai/core/@temporalio/workflow/@temporalio/common/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@shumai/core/@temporalio/workflow/@temporalio/proto/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
"@spectrum-web-components/action-group/@spectrum-web-components/icons-workflow/@spectrum-web-components/icon/@spectrum-web-components/iconset": ["@spectrum-web-components/iconset@0.37.0", "", { "dependencies": { "@spectrum-web-components/base": "^0.37.0" } }, "sha512-s1TBjWeQtGdGf6n+r+aG2glv77IgR0pNs0HLusGyYt6jzWFkP276fE7Neqq/2XvG9a9zvAwngffnWS3td/Gtmg=="],
"@spectrum-web-components/card/@spectrum-web-components/icons-workflow/@spectrum-web-components/icon/@spectrum-web-components/iconset": ["@spectrum-web-components/iconset@0.37.0", "", { "dependencies": { "@spectrum-web-components/base": "^0.37.0" } }, "sha512-s1TBjWeQtGdGf6n+r+aG2glv77IgR0pNs0HLusGyYt6jzWFkP276fE7Neqq/2XvG9a9zvAwngffnWS3td/Gtmg=="],
@@ -3641,7 +3394,5 @@
"readdir-glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="],
"@prisma/studio-core/@radix-ui/react-toggle/@radix-ui/react-primitive/@radix-ui/react-slot/@radix-ui/react-compose-refs": ["@radix-ui/react-compose-refs@1.1.2", "", { "peerDependencies": { "@types/react": "*", "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-z4eqJvfiNnFMHIIvXP3CY57y2WJs5g2v3X0zm9mEJkrkNv4rDxu+sg9Jh8EkXyeqBkB7SOcboo9dMVqhyrACIg=="],
"@shumai/core/@temporalio/activity/@temporalio/client/@temporalio/proto/protobufjs": ["protobufjs@8.8.0", "", { "dependencies": { "long": "^5.3.2" } }, "sha512-N3xhQ5yyBx3vQq4gubBfASzYhJGNzeDbjqBpu61g7UVylsN/qyffU96TKWD3GbbLOKF82VGNRNvv1+BFgE31Eg=="],
}
}
@@ -15,7 +15,7 @@ services:
retries: 5
embedding-server:
image: shumaione/embedding-server:latest
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/embedding-server:latest
container_name: shumai_embedding
restart: unless-stopped
@@ -15,12 +15,12 @@ services:
retries: 5
embedding-server:
image: shumaione/embedding-server:latest
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/embedding-server:latest
container_name: shumai_embedding
restart: unless-stopped
turntable-renderer:
image: shumaione/turntable-renderer:latest
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/turntable-renderer:latest
container_name: shumai_turntable
restart: unless-stopped
@@ -1,84 +0,0 @@
services:
# --- Shumai Application Services ---
postgres:
image: pgvector/pgvector:pg18
container_name: shumai_postgres
environment:
POSTGRES_USER: shumai
POSTGRES_PASSWORD: shumai_password
POSTGRES_DB: shumai_db
volumes:
- postgres_data:/var/lib/postgresql
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U shumai -d shumai_db']
interval: 5s
timeout: 5s
retries: 5
shumai:
image: shumaione/shumai:latest
container_name: shumai_app
ports:
- '3000:3000'
volumes:
- shumai_data:/app/data
environment:
DATABASE_URL: postgres://shumai:shumai_password@postgres:5432/shumai_db?search_path=public&sslmode=disable
STORAGE_BACKEND: s3
AWS_ENDPOINT_URL_S3: S3_ENDPOINT_URL
S3_REGION: S3_REGION
S3_BUCKET: BUCKET_NAME
S3_ACCESS_KEY_ID: ID
S3_SECRET_ACCESS_KEY: KEY
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
WORKFLOW_EXECUTOR: temporal
TEMPORAL_ADDRESS: temporal:7233
GEMINI_API_KEY: KEY
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
depends_on:
postgres:
condition: service_healthy
temporal:
condition: service_healthy
# --- Temporal Services ---
temporal:
image: temporalio/auto-setup:latest
container_name: temporal_server
environment:
- DB=postgres12
- DB_PORT=5432
- POSTGRES_USER=shumai
- POSTGRES_PWD=shumai_password
- POSTGRES_SEEDS=postgres
- BIND_ON_IP=0.0.0.0
ports:
- '7233:7233'
healthcheck:
test:
[
'CMD-SHELL',
'temporal operator cluster health --address localhost:7233 || temporal operator cluster health --address $$(hostname -i):7233',
]
interval: 5s
timeout: 5s
start_period: 30s
retries: 10
depends_on:
postgres:
condition: service_healthy
temporal-ui:
image: temporalio/ui:latest
container_name: temporal_ui
environment:
- TEMPORAL_ADDRESS=temporal:7233
ports:
- '8080:8080'
depends_on:
temporal:
condition: service_healthy
volumes:
postgres_data:
shumai_data:
+34
View File
@@ -3,6 +3,40 @@ title: "Changelog"
description: "Recent updates, new features, and improvements in Shumai"
---
<Update label="v0.6.1" description="October 24, 2026">
### Added
* **dashboard**: Add Workflow Tasks section to the Team Dashboard allowing team owners to monitor background workflow jobs, inspect execution payloads and outputs, and stop in-flight tasks
### Fixed
* **metadata**: Fix an issue where clicking to edit a metadata field on a card or list row inadvertently selected the file, and prevent expanded text overlays or date pickers from being clipped by container borders
### Changed
* **deps**: Upgrade Bun to 1.4.4
</Update>
<Update label="v0.6.0" description="October 20, 2026">
### Added
* **deploy**: Add dedicated `shumaione/shumai-transcode` Docker image and a complete AWS EKS deployment example (`eks/`) featuring KEDA queue-driven autoscaling, idle-timeout auto-drain worker mode, and Scale-to-Zero On-Demand Graviton5 instances for media processing
* **storage**: Support configuring a separate public endpoint (`AWS_PUBLIC_ENDPOINT_URL_S3`) for browser-facing presigned URLs while preserving internal cluster endpoints for backend workers
### Fixed
* **transcode**: Fix an issue where PDF, 3D model, and image background transcode workflows could fail with missing temporary file errors if a worker restarted or recovered midway through processing
* **webui**: Improve file list and detail loading with matching skeleton layouts, a persistent sidebar carousel to eliminate layout shift, and smoothed video poster previews
### Changed
* **workflow**: Migrate the background workflow engine to DBOS, unifying all-in-one and dedicated worker deployments using the existing PostgreSQL database without requiring a separate Temporal cluster
</Update>
<Update label="v0.5.3" description="October 16, 2026">
### Added
+173
View File
@@ -0,0 +1,173 @@
---
title: Deploy with Distributed Workers
description: Configure a high-scale production deployment using dedicated background workers, DBOS, and S3-compatible storage.
---
For production environments or workloads with high-volume video processing and heavy AI agent usage, running background tasks on a single server can cause bottlenecks. Shumai resolves this by decoupling the web dashboard from background processing through dedicated worker nodes orchestrated by **DBOS Transact**.
All background tasks (media transcoding, AI embeddings, agent tasks) are dispatched to durable task queues (`transcode_queue` and `agent_queue`) hosted directly in PostgreSQL. No separate external cluster (like Temporal) is required.
---
## How it Works
In a distributed deployment, the core Shumai web application handles user requests, database updates, and file uploads, while delegating durable background workflows to specialized worker instances via PostgreSQL task queues.
```mermaid
graph TD
UserUpload[User Uploads Video / Triggers Agent] --> WebApp[Shumai Web UI / API]
WebApp -->|Enqueue Task| DB[(PostgreSQL / DBOS Queues)]
DB -->|Queue: transcode_queue| TranscodeWorker[Transcode Worker Machine]
DB -->|Queue: agent_queue| AgentWorker[Agent Worker Machine]
```
- **Transcoding Workers**: Can be run on GPU-accelerated or high-CPU compute instances.
- **AI Agent Workers**: Can be run on isolated, sandboxed hosts to safely execute automation scripts.
> [!IMPORTANT]
> **Storage Requirement**: Because the core application and the background workers run on separate machines in a distributed setup, **you must configure an S3-compatible storage backend** (such as AWS S3, Cloudflare R2, or MinIO). Local storage (`STORAGE_BACKEND=local`) is not supported in this mode because the workers and the web application do not share a common filesystem.
---
## Step 1: Deploy the Core Web Application
On your primary host machine, run PostgreSQL and the main Shumai web application.
To instruct the main web application to handle only API and UI traffic without running background worker queues locally, set `SHUMAI_WORKERS=none`:
```env
DATABASE_URL=postgresql://shumai:shumai_password@localhost:5432/shumai_db?schema=public
STORAGE_BACKEND=s3
AWS_ENDPOINT_URL_S3=https://your-s3-endpoint.com
S3_BUCKET=shumai
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key-id
S3_SECRET_ACCESS_KEY=your-secret-access-key
SHUMAI_WORKERS=none
BETTER_AUTH_SECRET=your-random-32-character-secret-key
```
When started with `SHUMAI_WORKERS=none` (or `--workers=none`), the web server starts immediately without polling or consuming tasks from the transcode or agent queues.
---
## Step 2: Configure & Run Dedicated Workers
On your specialized worker machines, configure each worker with a `.env` file referencing the shared PostgreSQL database and S3 storage:
```env
DATABASE_URL=postgresql://shumai:shumai_password@<primary-host-ip>:5432/shumai_db?schema=public
STORAGE_BACKEND=s3
AWS_ENDPOINT_URL_S3=https://your-s3-endpoint.com
S3_BUCKET=shumai
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key-id
S3_SECRET_ACCESS_KEY=your-secret-access-key
CONCURRENCY_TRANSCODE=1
CONCURRENCY_AGENT=5
```
### Media Transcoding Worker
Run this on your media processing nodes (requires system installations of `ffmpeg` and `ffprobe`):
```bash
npm install -g @shumai-one/shumai-transcode
shumai-transcode
```
Alternatively, you can run the transcode worker using the dedicated lightweight Docker image:
```bash
docker run -d \
--name shumai-transcode-worker \
--env-file .env \
shumaione/shumai-transcode:latest
```
(You can also use the all-in-one image: `docker run -d --name shumai-transcode-worker --env-file .env shumaione/shumai:latest shumai-transcode`.)
### AI Agent Worker
Run this on your sandboxed agent execution nodes (requires sandbox dependencies like `bubblewrap` or `sandbox-exec`):
```bash
npm install -g @shumai-one/shumai-agent
shumai-agent
```
Alternatively, you can run the agent worker using the official Docker image:
```bash
docker run -d \
--name shumai-agent-worker \
--env-file .env \
--cap-add=SYS_ADMIN \
shumaione/shumai:latest shumai-agent
```
> [!IMPORTANT]
> **For Bun Users:** Since the worker package binaries contain a `#!/usr/bin/env node` shebang, running `shumai-transcode` or `shumai-agent` directly will run under Node.js. If you wish to run them using Bun, prefix the commands with `bun run --bun` (e.g., `bun run --bun shumai-transcode` or `bun run --bun shumai-agent`).
---
## Worker Concurrency & Multi-Instance Deployments
### Concurrency Tuning
Workers pull tasks from PostgreSQL queues based on configurable concurrency limits:
- `CONCURRENCY_TRANSCODE`: Maximum concurrent video transcoding jobs per worker (defaults to `1`).
- `CONCURRENCY_AGENT`: Maximum concurrent agent and embedding workflows per worker (defaults to `5`, recommend setting to `1` when paired with a local `embedding-server`).
### Replica Identity
When running multiple replicas of the transcode or agent worker across multiple machines or pods:
- `DbosExecutor` automatically derives a unique per-instance ID from the machine hostname and process ID to ensure startup recovery only re-enqueues tasks belonging to that specific instance.
- **Custom Worker ID**: Set `SHUMAI_WORKERS_ID` (or `SHUMAI_WORKER_ID`) if you wish to assign an explicit identifier.
- **Kubernetes**: In Kubernetes Pods, set `POD_NAME` (via Downward API `fieldRef: { fieldPath: metadata.name }`) to provide stable, unique per-pod identity across pod restarts.
---
## Scalable Deployment on Kubernetes & AWS EKS
For production cloud deployments requiring high-throughput media processing, Shumai can be deployed on Kubernetes or **AWS Elastic Kubernetes Service (AWS EKS)** with automated scale-to-zero queue-driven scaling via **KEDA (Kubernetes Event-driven Autoscaling)**.
```mermaid
graph TD
User([User / Web Browser]) -->|Port 80/3000| Server[Pod: shumai-server<br/>SHUMAI_WORKERS=agent]
Server -->|Enqueue to transcode_queue| DB[(PostgreSQL: pgvector18<br/>dbos.workflow_status)]
Server -->|Direct Uploads / Stream Proxies| Storage[(S3 / Garage Storage)]
KEDA[KEDA ScaledJob<br/>PostgreSQL Trigger<br/>min: 0, max: 10] -.->|Monitors Queue| DB
KEDA -->|Spawns Jobs for queue| Job1[Job: shumai-transcode-1<br/>Auto-drain worker]
KEDA -->|Spawns Jobs for queue| JobN[Job: shumai-transcode-N<br/>Auto-drain worker]
Job1 -->|Fetch Source / Store Proxies| Storage
JobN -->|Fetch Source / Store Proxies| Storage
Job1 -->|Mark Complete & Exit 0| DB
JobN -->|Mark Complete & Exit 0| DB
CA[Cluster Autoscaler / Karpenter] -.->|Scale-to-Zero On-Demand Nodes<br/>0 to 10 instances| JobN
```
### Key Architectural Highlights
- **Decoupled Stateless Workers**: The main web and API server, database, and storage run on reliable On-Demand compute (`c9g.large`, `SHUMAI_WORKERS=agent`), delegating media transcoding to background workers orchestrated via DBOS task queues in PostgreSQL.
- **Dedicated Graviton5 On-Demand Compute (`c9g.xlarge`)**: Media encoding runs on 4 vCPU / 8 GiB RAM AWS Graviton5 On-Demand instances (~$0.174/hr), utilizing ARM64 vector optimizations and high-speed DDR5 memory without involuntary Spot interruptions.
- **Queue-Driven Scale-to-Zero via KEDA**: KEDA monitors `dbos.workflow_status` for enqueued and pending transcode tasks and dynamically provisions Kubernetes `ScaledJob` resources (scaling 0 to 10). When the queue is empty, transcode nodes scale down to zero (`minSize: 0`), incurring **$0/hr** in worker costs.
- **Idle-Timeout Auto-Drain**: Worker pods continuously process queued tasks with concurrency `1`. When the queue is fully drained and a pod has been idle for 10 seconds (`SHUMAI_WORKER_IDLE_TIMEOUT_MS=10000`), it exits `0` cleanly so the node can scale down, processing batch uploads without cold-start pod churn.
- **Automated Stale Task Recovery**: The core web server runs an integrated recovery sweep that detects stale or orphaned tasks (e.g. from node crash) and resumes them automatically in DBOS, with a 3-attempt cap to safely handle poison-pill media.
### Complete Deployment Example & Source Code
For the full, production-ready manifests, declarative `eksctl` cluster configuration, and step-by-step walkthrough, see the **[EKS Deployment Example](https://github.com/shumaiOne/shumai/tree/main/eks)** in the Shumai repository:
👉 **[View the EKS Deployment Guide & Manifests on GitHub](https://github.com/shumaiOne/shumai/tree/main/eks)**
The repository includes:
- [`eks/eks-cluster.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/eks-cluster.yaml): Complete `eksctl` ClusterConfig with compute tiering.
- [`eks/kustomization.yaml`](https://github.com/shumaiOne/shumai/blob/main/eks/kustomization.yaml): Single-command Kustomize deployment for all services.
- Detailed instructions for setting up public endpoints, automated Cluster Autoscaler, and zero-downtime graceful shutdown.
+8 -7
View File
@@ -64,14 +64,15 @@ For example, if you deploy Shumai on a remote server at `http://123.456.7.8` and
## Workflow Config
Configures whether background transcoding and AI agents are executed locally or offloaded to a Temporal cluster.
Configures background worker execution, concurrency limits, and worker identity for DBOS Transact queues.
| Variable | Description | Example / Default |
| :---------------------- | :------------------------------------------------------------------------------------------------- | :------------------------------ |
| `WORKFLOW_EXECUTOR` | Engine choice. Use `local` for in-process database polling, or `temporal` for distributed workers. | `local` (default) or `temporal` |
| `TEMPORAL_ADDRESS` | Host and port of the Temporal cluster gRPC service. | `localhost:7233` (default) |
| `CONCURRENCY_AGENT` | Maximum concurrent AI tasks executed by workers. Set to `1` when running local `embedding-server`. | `5` (default) |
| `CONCURRENCY_TRANSCODE` | Maximum concurrent media transcoding tasks executed by workers. | `1` (default) |
| Variable | Description | Example / Default |
| :---------------------- | :-------------------------------------------------------------------------------------------------------------------------------- | :---------------- |
| `SHUMAI_WORKERS` | Queues listened to by the web server. Options: `all` (default), `none` (API/UI only), `agent` (or `ai`), `transcode`. | `all` (default) |
| `CONCURRENCY_TRANSCODE` | Maximum concurrent media transcoding tasks executed by workers. | `1` (default) |
| `CONCURRENCY_AGENT` | Maximum concurrent AI tasks executed by workers. Set to `1` when running local `embedding-server`. | `5` (default) |
| `SHUMAI_WORKER_ID` | Optional explicit instance ID for worker replicas in multi-node deployments (auto-derived from hostname and process ID if unset). | `worker-1` |
| `POD_NAME` | Kubernetes pod name, used to provide stable per-pod worker identity across StatefulSet restarts. | `transcode-pod-0` |
---
-85
View File
@@ -1,85 +0,0 @@
---
title: Deploy with Temporal
description: Configure a high-scale production deployment using Temporal and S3-compatible storage.
---
For production environments or workloads with high-volume video processing and heavy AI agent usage, running background tasks on a single server can cause bottlenecks. Shumai resolves this by offloading task queue management, retries, and background executions to **Temporal**.
This allows you to decouple workloads and run specialized worker nodes on separate, dedicated machines.
---
## How it Works
When configured for Temporal, the core Shumai web application does not process background jobs itself. Instead, it publishes tasks to a Temporal Cluster queue.
```mermaid
graph TD
UserUpload[User Uploads Video] --> WebApp[Shumai Web UI / API]
WebApp -->|Register Task| Temporal[Temporal Cluster]
Temporal -->|Queue: Transcoding| TranscodeWorker[Transcode Worker Machine]
Temporal -->|Queue: AI Agents| AgentWorker[Agent Worker Machine]
```
- **Transcoding Workers**: Can be run on GPU-accelerated or high-CPU compute instances.
- **AI Agent Workers**: Can be run on isolated, sandboxed hosts to safely execute automation scripts.
> [!IMPORTANT]
> **Storage Requirement**: Because the core application and the background workers run on separate machines in a distributed setup, **you must configure an S3-compatible storage backend** (such as AWS S3, Cloudflare R2, or MinIO). Local storage (`STORAGE_BACKEND=local`) is not supported in this mode because the workers and the web application do not share a common filesystem.
---
## Step 1: Deploy the Core Stack
On your primary host machine, spin up the database, Temporal cluster services, and the main Shumai web application:
1. **Download the Temporal Stack Configuration**:
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/shumaiOne/shumai/main/docker-compose/temporal/docker-compose.yaml
```
2. **Configure S3 Settings**:
Open the `docker-compose.yaml` file and set your S3 credentials and endpoint environment variables under the `shumai` service.
3. **Start the Services**:
```bash
docker compose up -d
```
This exposes the web application on port `3000`, the database on `5432`, the Temporal gRPC service on `7233`, and the Temporal management dashboard on `8080`.
---
## Step 2: Configure & Run Workers
On your specialized worker machines, you will run the dedicated background worker services. Each worker requires a `.env` file referencing the database, S3 storage, and the Temporal cluster:
```env
DATABASE_URL=postgresql://shumai:shumai_password@<primary-host-ip>:5432/shumai_db?schema=public
WORKFLOW_EXECUTOR=temporal
TEMPORAL_ADDRESS=<primary-host-ip>:7233
STORAGE_BACKEND=s3
AWS_ENDPOINT_URL_S3=https://your-s3-endpoint.com
S3_BUCKET=shumai
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-access-key-id
S3_SECRET_ACCESS_KEY=your-secret-access-key
```
### Media Transcoding Worker
Run this on your media processing nodes (requires system installations of `ffmpeg` and `ffprobe`):
```bash
npm install -g @shumai-one/shumai-transcode
shumai-transcode
```
### AI Agent Worker
Run this on your sandboxed agent execution nodes (requires sandbox dependencies like `bubblewrap` or `sandbox-exec`):
```bash
npm install -g @shumai-one/shumai-agent
shumai-agent
```
> [!IMPORTANT]
> **For Bun Users:** Since the worker package binaries contain a `#!/usr/bin/env node` shebang, running `shumai-transcode` or `shumai-agent` directly will run under Node.js. If you wish to run them using Bun, prefix the commands with `bun run --bun` (e.g., `bun run --bun shumai-transcode` or `bun run --bun shumai-agent`).
+2 -2
View File
@@ -76,8 +76,8 @@
"pages": ["configuration/env-variables", "configuration/hardware-transcoding"]
},
{
"group": "Temporal Deployment",
"pages": ["configuration/temporal"]
"group": "Distributed Deployment",
"pages": ["configuration/distributed"]
},
{
"group": "Releases",
+1 -1
View File
@@ -32,6 +32,6 @@ Ideal for small teams, local development, or low-volume workflows. In this setup
### 2. High-Scale Distributed Deployment
Recommended for production environments with high-volume video uploads. In this setup, background tasks are managed via a centralized queue (Temporal). This allows you to offload heavy workloads to specialized worker machines (for example, putting video transcoding tasks on high-CPU/GPU machines and AI agents on separate sandboxed environments).
Recommended for production environments with high-volume video uploads. In this setup, background tasks are managed via durable queues (orchestrated by DBOS directly in PostgreSQL). This allows you to offload heavy workloads to specialized worker machines (for example, putting video transcoding tasks on high-CPU/GPU machines and AI agents on separate sandboxed environments).
_Note: S3-compatible cloud storage is required for distributed deployments so that all workers can access the uploaded files._
+2 -2
View File
@@ -27,9 +27,9 @@ Docker Compose automatically configures and spins up all required services (the
3. **Access the Dashboard**:
Open your browser and navigate to **`http://localhost:3000`** to access the Shumai Web UI.
### Production Deployments (Temporal)
### Production Deployments (Distributed Workers)
If you are deploying Shumai for a production team with high-volume video uploads, we recommend setting up a distributed deployment using **Temporal** and an S3-compatible storage backend. This offloads resource-heavy video transcoding and AI agents to separate, dedicated worker nodes to keep your main dashboard fast and responsive. For setup instructions, see the [Deploy with Temporal](/configuration/temporal) guide.
If you are deploying Shumai for a production team with high-volume video uploads, we recommend setting up a distributed deployment using dedicated workers and an S3-compatible storage backend. This offloads resource-heavy video transcoding and AI agents to separate, dedicated worker nodes to keep your main dashboard fast and responsive. For setup instructions, see the [Deploy with Distributed Workers](/configuration/distributed) guide.
---
+2 -2
View File
@@ -77,6 +77,6 @@ The **FFmpeg Threads** slider controls the maximum number of CPU threads used by
## Processing Architecture
Transcoding is resource-intensive. Depending on your scaling needs, your administrator can run the transcode engine on the main server (Single-Server mode) or delegate tasks to specialized transcoding worker machines (Temporal Worker mode).
Transcoding is resource-intensive. Depending on your scaling needs, your administrator can run the transcode engine on the main server (Single-Server mode) or delegate tasks to specialized transcoding worker machines (Dedicated Worker mode).
In all modes, the processing machine must have system media utilities (`ffmpeg` and `ffprobe`) installed. See the [Temporal Deployment Guide](/configuration/temporal) for setting up distributed transcoding workers.
In all modes, the processing machine must have system media utilities (`ffmpeg` and `ffprobe`) installed. See the [Distributed Deployment Guide](/configuration/distributed) for setting up dedicated transcoding workers.
+245
View File
@@ -0,0 +1,245 @@
# Shumai on AWS EKS with KEDA & Scale-to-Zero On-Demand Workers
This directory contains a complete, production-ready Kubernetes deployment example for running Shumai on **Amazon Elastic Kubernetes Service (Amazon EKS)**, using **KEDA (Kubernetes Event-driven Autoscaling)** and **Scale-to-Zero On-Demand AWS Graviton5 instances** for compute-intensive media processing.
The example should also be straightforward to adapt to any standard Kubernetes cluster.
To demonstrate a fully self-hosted setup, it uses Garage as an S3-compatible object storage solution. However, if you're deploying on AWS, consider using **Amazon S3** instead of running your own S3-compatible storage server, as it is managed and highly scalable.
---
## Architecture Overview
```text
[ User / Internet ]
│
┌────────────────────┴────────────────────┐
│ (Port 80/3000) │ (Port 3900)
▼ ▼
┌───────────────────────┐ ┌───────────────────────┐
│ shumai-server (ELB) │ │ garage-public (ELB) │
└───────────┬───────────┘ └───────────┬───────────┘
│ │ Direct Browser Uploads
│ ▼ (Presigned URLs)
┌───────────────────┼─────────────────────────────────────────────────────────────────┐
│ General Workload Node (c9g.large On-Demand, ~$0.087/hr) │
│ │ │
│ ▼ │
│ ┌───────────────────────┐ Enqueue DBOS Tasks ┌───────────────────┐ │
│ │ shumai-server ├────────────────────────►│ shumai-postgres │ │
│ │(SHUMAI_WORKERS=agent) │ (transcode_queue) │ (pgvector:pg18) │ │
│ │ [Stale Recovery Loop] │ │ │ │
│ └───────────┬───────────┘ └─────────┬─────────┘ │
│ │ Internal S3 Traffic │ │
│ ▼ │ │
│ ┌───────────────────────┐ │ │
│ │ garage-s3 │ │ │
│ │ (dxflrs/garage) │ │ │
│ └───────────▲───────────┘ │ │
└───────────────────┼───────────────────────────────────────────────┼─────────────────┘
│ Read Source / Write Proxies │
│ │ KEDA monitors
│ ▼ dbos.workflow_status
│ ┌───────────────────────────────────────┐
│ │ KEDA Controller: ScaledJob │
│ │ Spawns Jobs based on pending tasks │
│ └───────────────────┬───────────────────┘
│ │
│ ▼ Creates K8s Jobs
┌───────────────────┴─────────────────────────────────────────────────────────────────┐
│ Transcode Compute Pool (c9g.xlarge Graviton5 On-Demand, minSize: 0, maxSize: 5) │
│ │
│ ┌──────────────────────────┐ ┌──────────────────────────┐ ┌────────────────────┐ │
│ │ Job: transcode-job-1 │ │ Job: transcode-job-2 │ │Job: transcode-job-N│ │
│ │ (Auto-Drain Worker) │ │ (Auto-Drain Worker) │ │ (Auto-Drain Worker)│ │
│ └────────────┬─────────────┘ └────────────┬─────────────┘ └──────────┬─────────┘ │
│ ▼ ▼ ▼ │
│ ┌──────────────────────────┐ ┌──────────────────────────┐ ┌────────────────────┐ │
│ │ Node 1: c9g.xlarge │ │ Node 2: c9g.xlarge │ │ Node N: c9g.xlarge │ │
│ │ (On-Demand) │ │ (On-Demand) │ │ (On-Demand) │ │
│ └──────────────────────────┘ └──────────────────────────┘ └────────────────────┘ │
│ │
│ Scale-to-Zero: When queue is empty, 0 Jobs run and 0 EC2 instances are billed! │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
---
## Core Design Principles
### 1. Scale-to-Zero On-Demand Instances (`minSize: 0`)
Media transcoding with FFmpeg is bursty: hours may pass with zero transcoding activity, followed by sudden batches of 4K video uploads.
- **Zero Idle Cost**: The transcode node group [`eks/eks-cluster.yaml`](./eks-cluster.yaml) is configured with `minSize: 0` and `desiredCapacity: 0`. When no media transcoding tasks are active, **zero transcode EC2 instances exist**, costing **$0/hr**.
- **No Spot Interruption Risk**: Transcoding jobs run on AWS Graviton5 **On-Demand** instances (`c9g.xlarge`, ~$0.174/hr). This eliminates the risk of Spot instances being reclaimed mid-encode by AWS, preventing wasted CPU cycles and avoiding unexpected transcoding delays.
### 2. KEDA `ScaledJob` vs. `Deployment` / `StatefulSet`
Standard Kubernetes Horizontal Pod Autoscaler (HPA) scales `Deployment` replicas based on metrics like CPU utilization. However, HPAs on Deployments suffer from severe limitations during video processing:
1. When a 1-minute video finishes on Pod A while a 30-minute video is still encoding on Pod B, CPU drops and HPA reduces replica count, which can arbitrarily terminate Pod B mid-encode.
2. StatefulSets scale down in strict ordinal order (`pod-4`, `pod-3`, etc.), which would terminate `pod-4` even if `pod-4` is the only pod currently transcoding!
**KEDA `ScaledJob` solves this natively**:
- KEDA queries PostgreSQL directly:
```sql
SELECT COUNT(*) FROM dbos.workflow_status WHERE queue_name = 'transcode_queue' AND status IN ('ENQUEUED', 'PENDING');
```
- KEDA spawns independent Kubernetes `Job` pods based on enqueued workload.
- Each Job runs the transcode worker with an idle timeout (`SHUMAI_WORKER_IDLE_TIMEOUT_MS=10000`).
- When the job drains its tasks and becomes idle, the pod exits `0` and Kubernetes marks the Job completed.
- Completing or terminating one job **never impacts any other job**!
### 3. Idle-Timeout Auto-Drain (10s Inactivity Shutdown)
In [`eks/shumai-transcode.yaml`](./shumai-transcode.yaml), the worker container runs with `SHUMAI_WORKER_IDLE_TIMEOUT_MS: "10000"`:
- **Concurrency Isolation**: Transcode concurrency is set to `1` per pod (`CONCURRENCY_TRANSCODE=1`). Each pod has exclusive access to the node's 4 vCPUs for FFmpeg.
- **Batch Processing Without Cold Starts**: Unlike single-task execution, a running worker continuously pulls tasks from `transcode_queue`. When a user uploads 100 images or multiple video files, a warm worker processes them back-to-back in seconds without restarting the container or control plane churn.
- **Automatic Scale-to-Zero**: When `activeWorkflows === 0` and no new task arrives within 10 consecutive seconds, the worker initiates graceful DBOS shutdown and exits with code `0`.
- The completed Job pod terminates, freeing the node so Cluster Autoscaler can terminate the EC2 instance down to zero.
---
## Step-by-Step Deployment Guide
### Prerequisites
- [AWS CLI](https://aws.amazon.com/cli/) installed and configured (`aws configure`).
- [`eksctl`](https://eksctl.io/) installed.
- [`kubectl`](https://kubernetes.io/docs/tasks/tools/) installed.
- [`helm`](https://helm.sh/) installed.
### Step 1: Provision the EKS Cluster
Create the cluster using the declarative [`eks/eks-cluster.yaml`](./eks-cluster.yaml) configuration:
```bash
eksctl create cluster -f eks/eks-cluster.yaml
```
This provisions:
- An EKS 1.31 control plane in `us-east-1` across multiple AZs. *(Note: AWS Graviton5 `c9g` instances are available in `us-east-1`. If deploying to a region where `c9g` is not yet offered, substitute `c7g.large` / `c7g.xlarge` or `c8g` in `eks/eks-cluster.yaml`.)*
- `shumai-general`: 1x `c9g.large` On-Demand instance (labeled `workload: general`) for API, database, and storage.
- `shumai-transcode`: Scale-to-zero `c9g.xlarge` On-Demand instance pool (`minSize: 0`, `maxSize: 5`, `desiredCapacity: 0`, tainted `dedicated=transcode:NoSchedule`).
- AWS EBS CSI Driver with IAM OIDC roles.
### Step 2: Install KEDA
Install KEDA using Helm:
```bash
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace
```
Wait for the KEDA operator to reach `Running` state:
```bash
kubectl wait --namespace keda --for=condition=ready pod -l app.kubernetes.io/name=keda-operator --timeout=120s
```
### Step 3: Deploy the Shumai Stack
Deploy all components with Kustomize:
```bash
kubectl apply -k eks/
```
This applies:
- [`eks/storageclass.yaml`](./storageclass.yaml): Configures `gp3` as the default StorageClass using `ebs.csi.aws.com` with `WaitForFirstConsumer` binding and dynamic EBS volume expansion.
- PostgreSQL (`pgvector:pg18`) with durable EBS storage.
- Garage S3 object store with automatic bucket and CORS initialization (`garage-init` job).
- Shumai API & Web server.
- KEDA `ScaledJob` for transcode worker auto-scaling.
Wait for the core database and storage services to initialize:
```bash
# Wait for PostgreSQL and Garage S3 pods to become Ready
kubectl wait --namespace shumai --for=condition=ready pod -l app=shumai-postgres --timeout=180s
kubectl wait --namespace shumai --for=condition=ready pod -l app=garage --timeout=180s
# Wait for the bucket initialization job to complete
kubectl wait --namespace shumai --for=condition=complete job/garage-init --timeout=180s
# Verify all pods in the shumai namespace
kubectl get pods -n shumai -o wide
```
*(Note: On first startup, `shumai-server` automatically runs database migrations against PostgreSQL; this may take 30–60 seconds before the server reaches `Running (1/1)`.)*
### Step 4: Configure Public Access & S3 Endpoint
AWS automatically provisions external Elastic Load Balancers for `shumai-server` and `garage-public`. Because AWS takes 30–60 seconds to allocate external hostnames, wait for the hostnames to become available:
```bash
# Wait for AWS to assign external hostnames to the LoadBalancers
echo "Waiting for LoadBalancer external hostnames..."
until [ -n "$(kubectl get svc garage-public -n shumai -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')" ]; do
sleep 5
done
GARAGE_ELB=$(kubectl get svc garage-public -n shumai -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
SERVER_ELB=$(kubectl get svc shumai-server -n shumai -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "Garage S3 Hostname: http://${GARAGE_ELB}:3900"
echo "Shumai Web UI: http://${SERVER_ELB}"
```
Inject the public Garage S3 hostname into `shumai-server` so that browsers generate correct presigned upload URLs:
```bash
kubectl set env deployment/shumai-server -n shumai AWS_PUBLIC_ENDPOINT_URL_S3="http://${GARAGE_ELB}:3900"
```
*(Note: In production with custom domains, point your DNS CNAME records to the load balancers and configure `AWS_PUBLIC_ENDPOINT_URL_S3: https://s3.mycompany.com` in [`eks/config.yaml`](./config.yaml).)*
### Step 5: Deploy Kubernetes Cluster Autoscaler
Deploy Cluster Autoscaler to dynamically provision `c9g.xlarge` EC2 nodes when KEDA jobs are created:
```bash
curl -s https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml \
| sed 's/<YOUR CLUSTER NAME>/shumai-eks/g' \
| kubectl apply -f -
# Pin autoscaler to the general workload node
kubectl patch deployment cluster-autoscaler -n kube-system -p '{"spec":{"template":{"spec":{"nodeSelector":{"workload":"general"}}}}}'
```
### Step 6: Test Scale-to-Zero Queue-Driven Autoscaling
1. Open `http://${SERVER_ELB}` in your browser and register an account via **Sign Up**. *(The first registered account automatically becomes the instance administrator).*
2. Create a project and upload a batch of 5–10 video files simultaneously.
3. Observe KEDA detecting pending DBOS tasks and spawning individual Kubernetes Jobs:
```bash
kubectl get scaledjobs -n shumai
kubectl get jobs -n shumai -w
kubectl get pods -n shumai -l app=shumai-transcode -o wide
kubectl get nodes -L role,workload
```
4. Watch Cluster Autoscaler dynamically provision dedicated `c9g.xlarge` Graviton5 EC2 instances for the Jobs.
5. As each video finishes transcoding, its worker pod exits `0` and completes immediately.
6. Once all jobs are done, KEDA holds 0 jobs, and Cluster Autoscaler terminates all transcode EC2 instances, returning the transcode pool to **0 nodes ($0/hr)**!
### Step 7: Cluster Teardown & Cost Cleanup
When you are finished testing, delete all resources to prevent ongoing cloud charges:
```bash
# 1. Delete Kubernetes namespace first to release AWS ELB load balancers and EBS volumes
# (Important: Wait for this to complete so AWS SecurityGroups can be deleted without dependency conflicts)
kubectl delete namespace shumai
# 2. Uninstall KEDA
helm uninstall keda -n keda
kubectl delete namespace keda
# 3. Delete the EKS cluster and CloudFormation stacks
eksctl delete cluster -f eks/eks-cluster.yaml --wait
# 4. Confirm all AWS resources are released
aws eks list-clusters --region us-east-1
```
+31
View File
@@ -0,0 +1,31 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: shumai-config
namespace: shumai
data:
# S3 Storage Configuration (Garage S3)
STORAGE_BACKEND: 's3'
AWS_ENDPOINT_URL_S3: 'http://garage.shumai.svc.cluster.local:3900'
S3_BUCKET: 'shumai'
S3_REGION: 'garage'
S3_FORCE_PATH_STYLE: 'true'
# Server & Worker Configuration
SHUMAI_SERVER_PORT: '3000'
CONCURRENCY_TRANSCODE: '1'
SHUMAI_WORKER_DRAIN_TIMEOUT_MS: '290000'
---
apiVersion: v1
kind: Secret
metadata:
name: shumai-secret
namespace: shumai
type: Opaque
stringData:
DATABASE_URL: 'postgresql://shumai:shumai_password@shumai-postgres.shumai.svc.cluster.local:5432/shumai_db?schema=public'
KEDA_DATABASE_URL: 'postgresql://shumai:shumai_password@shumai-postgres.shumai.svc.cluster.local:5432/shumai_db?sslmode=disable'
BETTER_AUTH_SECRET: 'qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw='
S3_ACCESS_KEY_ID: 'GKshumaiaccesskeyid001'
S3_SECRET_ACCESS_KEY: 'shumaisecretaccesskey00000000000000001'
GARAGE_ADMIN_TOKEN: 'admin-secret-token'
+57
View File
@@ -0,0 +1,57 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: shumai-eks
region: us-east-1
version: '1.31'
iam:
withOIDC: true
addons:
- name: aws-ebs-csi-driver
wellKnownPolicies:
ebsCSIController: true
managedNodeGroups:
# General system workload: Web UI, REST API, DBOS PostgreSQL, and Garage S3
# Runs on On-Demand instances to guarantee database & storage durability without spot interruptions
- name: shumai-general
instanceType: c9g.large # 2 vCPU, 4 GiB RAM (Graviton5 On-Demand, ~$0.087/hr)
spot: false
minSize: 1
maxSize: 2
desiredCapacity: 1
amiFamily: AmazonLinux2023
volumeSize: 50
volumeType: gp3
labels:
workload: general
iam:
withAddonPolicies:
autoScaler: true
ebs: true
# Transcoding compute pool: High-CPU media processing
# Uses On-Demand instances with scale-to-zero (minSize: 0) managed by Cluster Autoscaler / Karpenter
- name: shumai-transcode
instanceType: c9g.xlarge # 4 vCPU, 8 GiB RAM (Graviton5 On-Demand, ~$0.174/hr)
spot: false
minSize: 0
maxSize: 5
desiredCapacity: 0
amiFamily: AmazonLinux2023
volumeSize: 50
volumeType: gp3
labels:
workload: transcode
role: transcode
taints:
- key: dedicated
value: transcode
effect: NoSchedule
iam:
withAddonPolicies:
autoScaler: true
ebs: true
+238
View File
@@ -0,0 +1,238 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: garage-config
namespace: shumai
data:
garage.toml: |
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1
rpc_bind_addr = "0.0.0.0:3901"
rpc_secret = "c7d7e35b7501a91e5cfb3874026315579930f367fa3b7e4a1e94cf67484a86cf"
[s3_api]
s3_region = "garage"
api_bind_addr = "0.0.0.0:3900"
root_domain = ".s3.garage"
[admin]
api_bind_addr = "0.0.0.0:3903"
admin_token = "admin-secret-token"
---
apiVersion: v1
kind: Service
metadata:
name: garage
namespace: shumai
labels:
app: garage
spec:
ports:
- port: 3900
name: s3-api
- port: 3901
name: rpc
- port: 3903
name: admin
selector:
app: garage
---
apiVersion: v1
kind: Service
metadata:
name: garage-public
namespace: shumai
labels:
app: garage
spec:
type: LoadBalancer
ports:
- port: 3900
targetPort: 3900
name: s3-api
selector:
app: garage
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: garage
namespace: shumai
spec:
serviceName: garage
replicas: 1
selector:
matchLabels:
app: garage
template:
metadata:
labels:
app: garage
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: eks.amazonaws.com/nodegroup
operator: In
values:
- shumai-general
- shumai-general-spot
- matchExpressions:
- key: workload
operator: In
values:
- general
containers:
- name: garage
image: dxflrs/garage:v2.4.1
imagePullPolicy: IfNotPresent
command: ['/garage', 'server']
ports:
- containerPort: 3900
name: s3-api
- containerPort: 3901
name: rpc
- containerPort: 3903
name: admin
volumeMounts:
- name: config
mountPath: /etc/garage.toml
subPath: garage.toml
- name: garage-meta
mountPath: /var/lib/garage/meta
- name: garage-data
mountPath: /var/lib/garage/data
volumes:
- name: config
configMap:
name: garage-config
volumeClaimTemplates:
- metadata:
name: garage-meta
spec:
accessModes: ['ReadWriteOnce']
resources:
requests:
storage: 1Gi
- metadata:
name: garage-data
spec:
accessModes: ['ReadWriteOnce']
resources:
requests:
storage: 10Gi
---
apiVersion: batch/v1
kind: Job
metadata:
name: garage-init
namespace: shumai
spec:
backoffLimit: 10
template:
metadata:
labels:
app: garage-init
spec:
restartPolicy: OnFailure
containers:
- name: init
image: alpine:3.20
env:
- name: S3_BUCKET
valueFrom:
configMapKeyRef:
name: shumai-config
key: S3_BUCKET
- name: S3_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: shumai-secret
key: S3_ACCESS_KEY_ID
- name: S3_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: shumai-secret
key: S3_SECRET_ACCESS_KEY
- name: GARAGE_ADMIN_TOKEN
valueFrom:
secretKeyRef:
name: shumai-secret
key: GARAGE_ADMIN_TOKEN
command:
- /bin/sh
- -c
- |
set -e
apk add --no-cache curl jq aws-cli >/dev/null 2>&1
echo "Waiting for Garage Admin API to respond..."
until curl -s -f -H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" http://garage:3903/v2/GetClusterStatus >/dev/null 2>&1; do
sleep 2
done
echo "Garage Admin API is online. Retrieving cluster status..."
STATUS=$(curl -s -H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" http://garage:3903/v2/GetClusterStatus)
NODE_ID=$(echo "$STATUS" | jq -r '.nodes[0].id // empty')
if [ -z "$NODE_ID" ]; then
echo "Error: Node ID not found"
exit 1
fi
echo "Discovered Node ID: $NODE_ID"
ROLE=$(echo "$STATUS" | jq -r '.nodes[0].role // empty')
if [ -z "$ROLE" ] || [ "$ROLE" = "null" ]; then
echo "Assigning layout role to node..."
curl -s -f -X POST http://garage:3903/v2/UpdateClusterLayout \
-H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"roles\": [{\"id\": \"$NODE_ID\", \"zone\": \"dc1\", \"capacity\": 10737418240, \"tags\": []}]}" >/dev/null
echo "Applying cluster layout (version 1)..."
curl -s -f -X POST http://garage:3903/v2/ApplyClusterLayout \
-H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"version": 1}' >/dev/null || true
fi
echo "Importing S3 Access Key..."
curl -s -f -X POST http://garage:3903/v2/ImportKey \
-H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"name\": \"shumai-key\", \"accessKeyId\": \"${S3_ACCESS_KEY_ID}\", \"secretAccessKey\": \"${S3_SECRET_ACCESS_KEY}\"}" >/dev/null || true
echo "Creating S3 bucket '${S3_BUCKET}'..."
BUCKET_RESP=$(curl -s -X POST http://garage:3903/v2/CreateBucket \
-H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"globalAlias\": \"${S3_BUCKET}\"}")
echo "$BUCKET_RESP"
BUCKET_ID=$(echo "$BUCKET_RESP" | jq -r '.id // empty')
if [ -z "$BUCKET_ID" ]; then
# Fetch existing bucket ID if already created
BUCKET_ID=$(curl -s -H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" "http://garage:3903/v2/GetBucketInfo?globalAlias=${S3_BUCKET}" | jq -r '.id // empty')
fi
if [ -n "$BUCKET_ID" ]; then
echo "Granting bucket permissions for key on bucket ID $BUCKET_ID..."
curl -s -f -X POST http://garage:3903/v2/AllowBucketKey \
-H "Authorization: Bearer ${GARAGE_ADMIN_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"bucketId\": \"${BUCKET_ID}\", \"accessKeyId\": \"${S3_ACCESS_KEY_ID}\", \"permissions\": {\"read\": true, \"write\": true, \"owner\": true}}" >/dev/null || true
fi
echo "Configuring S3 bucket CORS for web uploads..."
AWS_ACCESS_KEY_ID=${S3_ACCESS_KEY_ID} \
AWS_SECRET_ACCESS_KEY=${S3_SECRET_ACCESS_KEY} \
AWS_DEFAULT_REGION=garage \
aws --endpoint-url http://garage:3900 s3api put-bucket-cors \
--bucket "${S3_BUCKET}" \
--cors-configuration '{"CORSRules":[{"AllowedHeaders":["*"],"AllowedMethods":["GET","PUT","POST","DELETE","HEAD"],"AllowedOrigins":["*"],"ExposeHeaders":["ETag"],"MaxAgeSeconds":3000}]}' || true
echo "✅ Garage S3 storage cluster initialized successfully!"
+11
View File
@@ -0,0 +1,11 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- storageclass.yaml
- config.yaml
- postgres.yaml
- garage.yaml
- shumai-server.yaml
- shumai-transcode.yaml
+4
View File
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: shumai
+79
View File
@@ -0,0 +1,79 @@
apiVersion: v1
kind: Service
metadata:
name: shumai-postgres
namespace: shumai
labels:
app: shumai-postgres
spec:
ports:
- port: 5432
name: postgres
selector:
app: shumai-postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: shumai-postgres
namespace: shumai
spec:
serviceName: shumai-postgres
replicas: 1
selector:
matchLabels:
app: shumai-postgres
template:
metadata:
labels:
app: shumai-postgres
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: eks.amazonaws.com/nodegroup
operator: In
values:
- shumai-general
- shumai-general-spot
- matchExpressions:
- key: workload
operator: In
values:
- general
containers:
- name: postgres
image: pgvector/pgvector:pg18
ports:
- containerPort: 5432
name: postgres
env:
- name: POSTGRES_USER
value: 'shumai'
- name: POSTGRES_PASSWORD
value: 'shumai_password'
- name: POSTGRES_DB
value: 'shumai_db'
- name: PGDATA
value: '/var/lib/postgresql/18/docker/pgdata'
readinessProbe:
exec:
command:
- sh
- -c
- pg_isready -U shumai -d shumai_db
initialDelaySeconds: 5
periodSeconds: 5
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/18/docker
volumeClaimTemplates:
- metadata:
name: postgres-data
spec:
accessModes: ['ReadWriteOnce']
resources:
requests:
storage: 5Gi
+85
View File
@@ -0,0 +1,85 @@
apiVersion: v1
kind: Service
metadata:
name: shumai-server
namespace: shumai
labels:
app: shumai-server
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 3000
name: http
- port: 3000
targetPort: 3000
name: http-alt
selector:
app: shumai-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: shumai-server
namespace: shumai
labels:
app: shumai-server
spec:
replicas: 1
selector:
matchLabels:
app: shumai-server
template:
metadata:
labels:
app: shumai-server
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: eks.amazonaws.com/nodegroup
operator: In
values:
- shumai-general
- shumai-general-spot
- matchExpressions:
- key: workload
operator: In
values:
- general
containers:
- name: server
image: shumaione/shumai:latest
imagePullPolicy: Always
env:
- name: SHUMAI_WORKERS
value: 'agent'
envFrom:
- configMapRef:
name: shumai-config
- secretRef:
name: shumai-secret
ports:
- containerPort: 3000
name: http
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 2000m
memory: 2Gi
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 10
periodSeconds: 5
livenessProbe:
tcpSocket:
port: 3000
initialDelaySeconds: 15
periodSeconds: 10
+90
View File
@@ -0,0 +1,90 @@
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-postgres-auth
namespace: shumai
spec:
secretTargetRef:
- parameter: connection
name: shumai-secret
key: KEDA_DATABASE_URL
---
apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
name: shumai-transcode-scaler
namespace: shumai
labels:
app: shumai-transcode
spec:
jobTargetRef:
parallelism: 1
completions: 1
backoffLimit: 0
template:
metadata:
labels:
app: shumai-transcode
spec:
restartPolicy: Never
terminationGracePeriodSeconds: 300
tolerations:
- key: dedicated
operator: Equal
value: transcode
effect: NoSchedule
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: role
operator: In
values:
- transcode
- matchExpressions:
- key: workload
operator: In
values:
- transcode
- matchExpressions:
- key: eks.amazonaws.com/nodegroup
operator: In
values:
- shumai-transcode
containers:
- name: worker
image: shumaione/shumai-transcode:latest
imagePullPolicy: Always
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: SHUMAI_WORKER_IDLE_TIMEOUT_MS
value: '10000'
envFrom:
- configMapRef:
name: shumai-config
- secretRef:
name: shumai-secret
resources:
requests:
cpu: 3000m
memory: 4Gi
limits:
cpu: 3900m
memory: 6Gi
pollingInterval: 10
successfulJobsHistoryLimit: 5
failedJobsHistoryLimit: 10
maxReplicaCount: 5
scalingStrategy:
strategy: default
triggers:
- type: postgresql
metadata:
query: "SELECT COUNT(*) FROM dbos.workflow_status WHERE queue_name = 'transcode_queue' AND status IN ('ENQUEUED', 'PENDING');"
targetQueryValue: '1'
authenticationRef:
name: keda-postgres-auth
+12
View File
@@ -0,0 +1,12 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gp3
annotations:
storageclass.kubernetes.io/is-default-class: 'true'
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
type: gp3
fsType: ext4
+4 -5
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"repository": {
@@ -24,7 +24,7 @@
"test:e2e:workflow": "bun --bun vitest run -c packages/e2e/vitest.config.ts",
"test:e2e:install": "playwright install --with-deps",
"harness:dev": "bun run packages/webui/e2e/harness/serve.ts",
"format": "prettier --write .",
"format": "bun fmt",
"db:migrate": "bun run prisma migrate dev",
"db:deploy": "bun run prisma migrate deploy",
"db:generate": "cd packages/db && bunx prisma generate",
@@ -50,14 +50,13 @@
"@testcontainers/postgresql": "12.1.0",
"@testing-library/dom": "10.4.1",
"@testing-library/react": "16.3.3",
"@types/bun": "1.4.0",
"@types/bun": "1.4.3",
"@types/node": "25.9.1",
"bun-plugin-pino": "1.5.0",
"bun-plugin-tailwind": "0.1.2",
"bun-types": "1.4.0",
"bun-types": "1.4.3",
"eslint": "10.9.1",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-prettier": "5.5.6",
"eslint-plugin-react": "7.37.5",
"globals": "17.11.0",
"happy-dom": "20.12.0",
+1 -4
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -14,9 +14,6 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@temporalio/activity": "1.23.0",
"@temporalio/worker": "1.23.0",
"@temporalio/workflow": "1.23.0",
"adm-zip": "0.6.0",
"jittered-fractional-indexing": "1.0.1",
"sharp": "0.35.4",
+30 -46
View File
@@ -23,8 +23,7 @@ import {
import * as piAgent from '../index'
import { type AgentHarness, type Session } from '@earendil-works/pi-agent-core'
import { DatabaseSessionStorage, type DatabaseSessionMetadata } from '../database-session-storage'
import { triggerLocalCancel } from '@shumai/workflow-core'
import { Context } from '@temporalio/activity'
import { triggerLocalCancel, runInLocalTaskContext } from '@shumai/workflow-core'
import { logger } from '@shumai/core/src/logger'
import {
@@ -50,9 +49,6 @@ vi.mock('../index', async () => {
describe('Agent Activities', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.spyOn(Context, 'current').mockImplementation(() => {
throw new Error('Not in Temporal')
})
})
it('should call agentChatActivity and prompt the harness', async () => {
@@ -240,13 +236,7 @@ describe('Agent Activities', () => {
await executionPromise
})
it('should listen to Temporal cancellation signal and call harness.abort when cancelled in Temporal mode', async () => {
const mockTemporalContext = {
cancellationSignal: new EventTarget(),
heartbeat: vi.fn(),
} as unknown as Context
vi.spyOn(Context, 'current').mockReturnValue(mockTemporalContext)
it('should listen to abort signal and call harness.abort when cancelled', async () => {
let resolvePrompt!: (value: {
content: Array<{ type: string; text: string } | { type: string }>
usage: { input: number; output: number }
@@ -280,42 +270,36 @@ describe('Agent Activities', () => {
allowedDomains: [],
} as unknown as AgentExecutionContext
const addEventListenerSpy = vi.spyOn(mockTemporalContext.cancellationSignal, 'addEventListener')
const removeEventListenerSpy = vi.spyOn(
mockTemporalContext.cancellationSignal,
'removeEventListener',
)
const taskId = 'test-abort-task'
const abortController = new AbortController()
const executionPromise = agentChatActivity({
teamId: 't1',
agentId: 'b1',
message: 'Hi',
imageUrls: [],
projectId: 'p1',
folderId: 'f1',
sessionId: 'mock-session-id',
context,
await runInLocalTaskContext({ taskId, abortController }, async () => {
const executionPromise = agentChatActivity({
taskId,
teamId: 't1',
agentId: 'b1',
message: 'Hi',
imageUrls: [],
projectId: 'p1',
folderId: 'f1',
sessionId: 'mock-session-id',
context,
})
// Yield to let the asynchronous setup add the abort event listener
await new Promise((resolve) => setTimeout(resolve, 50))
abortController.abort()
expect(mockHarness.abort).toHaveBeenCalled()
// Resolve the prompt promise to allow the activity to finish clean
resolvePrompt({
content: [{ type: 'text', text: 'AI response' }],
usage: { input: 10, output: 20 },
})
await executionPromise
})
// Yield to let the asynchronous setup add the abort event listener
await new Promise((resolve) => setTimeout(resolve, 50))
expect(addEventListenerSpy).toHaveBeenCalledWith('abort', expect.any(Function), { once: true })
// Simulate Temporal cancellation event
mockTemporalContext.cancellationSignal.dispatchEvent(new Event('abort'))
expect(mockHarness.abort).toHaveBeenCalled()
// Resolve the prompt promise to allow the activity to finish clean
resolvePrompt({
content: [{ type: 'text', text: 'AI response' }],
usage: { input: 10, output: 20 },
})
await executionPromise
expect(removeEventListenerSpy).toHaveBeenCalledWith('abort', expect.any(Function))
addEventListenerSpy.mockRestore()
removeEventListenerSpy.mockRestore()
})
it('should include error message in text when stopReason is error', async () => {
File diff suppressed because it is too large Load Diff
+465 -276
View File
@@ -2,11 +2,17 @@ import { GoogleGenAI } from '@google/genai'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService } from '@shumai/core'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import { ApplicationFailure } from '@temporalio/activity'
import { execFile } from 'child_process'
import { promisify } from 'util'
import { ulid } from 'ulid'
import { ApplicationFailure, defineStep } from '@shumai/workflow-core'
import { prisma } from '@shumai/db'
import sharp from 'sharp'
const execFileAsync = promisify(execFile)
export interface Usage {
inputTokens: number
outputTokens: number
@@ -112,42 +118,44 @@ export interface GenerateTextEmbeddingParams {
teamId: string
}
export async function generateTextEmbeddingActivity(params: GenerateTextEmbeddingParams): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
if (provider !== 'gemma-embedding-2-openai' && !process.env.GEMINI_API_KEY) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
nonRetryable: true,
})
}
try {
const result = await textEmbeddingService.generateEmbedding(params.text, {
teamId: params.teamId,
skipCache: true,
})
return {
embedding: result.embedding,
usage: {
model: result.usage?.model || provider,
inputTokens: result.usage?.inputTokens ?? 0,
outputTokens: result.usage?.outputTokens ?? 0,
},
export const generateTextEmbeddingActivity = defineStep(
async function generateTextEmbeddingActivity(params: GenerateTextEmbeddingParams): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
if (provider !== 'gemma-embedding-2-openai' && !process.env.GEMINI_API_KEY) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
nonRetryable: true,
})
}
} catch (err) {
if (err instanceof ApplicationFailure) {
throw err
try {
const result = await textEmbeddingService.generateEmbedding(params.text, {
teamId: params.teamId,
skipCache: true,
})
return {
embedding: result.embedding,
usage: {
model: result.usage?.model || provider,
inputTokens: result.usage?.inputTokens ?? 0,
outputTokens: result.usage?.outputTokens ?? 0,
},
}
} catch (err) {
if (err instanceof ApplicationFailure) {
throw err
}
throw ApplicationFailure.create({
message: err instanceof Error ? err.message : String(err),
nonRetryable: false,
})
}
throw ApplicationFailure.create({
message: err instanceof Error ? err.message : String(err),
nonRetryable: false,
})
}
}
},
)
export interface ExtractAiMetadataParams {
assetKey: string
@@ -156,7 +164,7 @@ export interface ExtractAiMetadataParams {
isImage: boolean
}
export async function extractAiMetadataActivity(
export const extractAiMetadataActivity = defineStep(async function extractAiMetadataActivity(
params: ExtractAiMetadataParams,
): Promise<string[]> {
const bucket = process.env.S3_BUCKET || 'shumai'
@@ -207,50 +215,51 @@ export async function extractAiMetadataActivity(
}
return generatedFiles
}
})
export async function getEmbeddingContextActivity(params: { teamId: string; assetId: string }) {
const team = await prisma.team.findUnique({
where: { id: params.teamId },
})
if (!team) {
throw ApplicationFailure.create({ message: 'failed to get team', nonRetryable: true })
}
const agent = await prisma.agent.findFirst({
where: {
type: 'embedding',
enabled: true,
user: { teamMembers: { some: { teamId: params.teamId } } },
},
})
if (!agent || process.env.ENABLE_EMBEDDING_AGENT === 'false') {
throw ApplicationFailure.create({
message: 'embedding feature is disabled or agent not found',
nonRetryable: true,
export const getEmbeddingContextActivity = defineStep(
async function getEmbeddingContextActivity(params: { teamId: string; assetId: string }) {
const team = await prisma.team.findUnique({
where: { id: params.teamId },
})
}
if (!team) {
throw ApplicationFailure.create({ message: 'failed to get team', nonRetryable: true })
}
const asset = await prisma.asset.findUnique({
where: { id: params.assetId },
include: { storageKey: true },
})
if (!asset) {
throw ApplicationFailure.create({ message: 'failed to get asset', nonRetryable: true })
}
if (!asset.mediaType) {
throw ApplicationFailure.create({ message: 'asset has no media type', nonRetryable: true })
}
const agent = await prisma.agent.findFirst({
where: {
type: 'embedding',
enabled: true,
user: { teamMembers: { some: { teamId: params.teamId } } },
},
})
if (!agent || process.env.ENABLE_EMBEDDING_AGENT === 'false') {
throw ApplicationFailure.create({
message: 'embedding feature is disabled or agent not found',
nonRetryable: true,
})
}
const parsedDuration = parseFloat(process.env.EMBEDDING_CHUNK_DURATION || '')
const chunkDuration =
Number.isFinite(parsedDuration) && parsedDuration > 0 ? parsedDuration : 60.0
const asset = await prisma.asset.findUnique({
where: { id: params.assetId },
include: { storageKey: true },
})
if (!asset) {
throw ApplicationFailure.create({ message: 'failed to get asset', nonRetryable: true })
}
if (!asset.mediaType) {
throw ApplicationFailure.create({ message: 'asset has no media type', nonRetryable: true })
}
const parsedOverlap = parseFloat(process.env.EMBEDDING_CHUNK_OVERLAP || '')
const chunkOverlap = Number.isFinite(parsedOverlap) && parsedOverlap >= 0 ? parsedOverlap : 5.0
const embeddingProvider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const parsedDuration = parseFloat(process.env.EMBEDDING_CHUNK_DURATION || '')
const chunkDuration =
Number.isFinite(parsedDuration) && parsedDuration > 0 ? parsedDuration : 60.0
const existingEmbeddings = await prisma.$queryRaw<ExistingAssetEmbeddingInfo[]>`
const parsedOverlap = parseFloat(process.env.EMBEDDING_CHUNK_OVERLAP || '')
const chunkOverlap = Number.isFinite(parsedOverlap) && parsedOverlap >= 0 ? parsedOverlap : 5.0
const embeddingProvider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const existingEmbeddings = await prisma.$queryRaw<ExistingAssetEmbeddingInfo[]>`
SELECT
id,
start_time as "startTime",
@@ -261,15 +270,16 @@ export async function getEmbeddingContextActivity(params: { teamId: string; asse
WHERE asset_id = ${params.assetId}
`
return {
agent,
asset,
chunkDuration,
chunkOverlap,
embeddingProvider,
existingEmbeddings,
}
}
return {
agent,
asset,
chunkDuration,
chunkOverlap,
embeddingProvider,
existingEmbeddings,
}
},
)
export interface ExistingAssetEmbeddingInfo {
id: string
@@ -279,62 +289,66 @@ export interface ExistingAssetEmbeddingInfo {
has768: boolean
}
export async function deleteAssetEmbeddingsActivity(params: { assetId: string; ids?: string[] }) {
if (params.ids && params.ids.length > 0) {
await prisma.assetEmbedding.deleteMany({
where: {
assetId: params.assetId,
id: { in: params.ids },
},
})
} else {
await prisma.assetEmbedding.deleteMany({
where: {
assetId: params.assetId,
},
})
}
}
export async function saveAssetEmbeddingsActivity(params: {
assetId: string
embeddings: Array<{
embedding: number[]
startTime?: number
endTime?: number
}>
}) {
for (const item of params.embeddings) {
const embVec = JSON.stringify(item.embedding)
const is768 = item.embedding.length === 768
if (item.startTime !== undefined && item.endTime !== undefined) {
if (is768) {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding_768, start_time, end_time, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, ${item.startTime}, ${item.endTime}, NOW())
`
} else {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding, start_time, end_time, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, ${item.startTime}, ${item.endTime}, NOW())
`
}
export const deleteAssetEmbeddingsActivity = defineStep(
async function deleteAssetEmbeddingsActivity(params: { assetId: string; ids?: string[] }) {
if (params.ids && params.ids.length > 0) {
await prisma.assetEmbedding.deleteMany({
where: {
assetId: params.assetId,
id: { in: params.ids },
},
})
} else {
if (is768) {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding_768, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, NOW())
`
await prisma.assetEmbedding.deleteMany({
where: {
assetId: params.assetId,
},
})
}
},
)
export const saveAssetEmbeddingsActivity = defineStep(
async function saveAssetEmbeddingsActivity(params: {
assetId: string
embeddings: Array<{
embedding: number[]
startTime?: number
endTime?: number
}>
}) {
for (const item of params.embeddings) {
const embVec = JSON.stringify(item.embedding)
const is768 = item.embedding.length === 768
if (item.startTime !== undefined && item.endTime !== undefined) {
if (is768) {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding_768, start_time, end_time, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, ${item.startTime}, ${item.endTime}, NOW())
`
} else {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding, start_time, end_time, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, ${item.startTime}, ${item.endTime}, NOW())
`
}
} else {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, NOW())
`
if (is768) {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding_768, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, NOW())
`
} else {
await prisma.$executeRaw`
INSERT INTO asset_embeddings (id, asset_id, embedding, updated_at)
VALUES (gen_random_uuid()::text, ${params.assetId}, ${embVec}::vector, NOW())
`
}
}
}
}
}
},
)
export interface GenerateImageEmbeddingParams {
teamId: string
@@ -342,60 +356,60 @@ export interface GenerateImageEmbeddingParams {
mediaType: string
}
export async function generateImageEmbeddingActivity(
params: GenerateImageEmbeddingParams,
): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.assetKey,
)
export const generateImageEmbeddingActivity = defineStep(
async function generateImageEmbeddingActivity(params: GenerateImageEmbeddingParams): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.assetKey,
)
if (provider === 'gemma-embedding-2-openai') {
let imgData = data
let mimeType = params.mediaType || 'image/jpeg'
if (mimeType !== 'image/jpeg' && mimeType !== 'image/png') {
imgData = await sharp(data).jpeg().toBuffer()
mimeType = 'image/jpeg'
}
return callGemmaEmbeddingServer([
{
type: 'image_url',
// eslint-disable-next-line @typescript-eslint/naming-convention
image_url: {
url: `data:${mimeType};base64,${imgData.toString('base64')}`,
if (provider === 'gemma-embedding-2-openai') {
let imgData = data
let mimeType = params.mediaType || 'image/jpeg'
if (mimeType !== 'image/jpeg' && mimeType !== 'image/png') {
imgData = await sharp(data).jpeg().toBuffer()
mimeType = 'image/jpeg'
}
return callGemmaEmbeddingServer([
{
type: 'image_url',
// eslint-disable-next-line @typescript-eslint/naming-convention
image_url: {
url: `data:${mimeType};base64,${imgData.toString('base64')}`,
},
},
},
])
}
])
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
nonRetryable: true,
})
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
nonRetryable: true,
})
}
const resolvedApiKey = process.env[apiKey] || apiKey
const ai = new GoogleGenAI({ apiKey: resolvedApiKey })
const resolvedApiKey = process.env[apiKey] || apiKey
const ai = new GoogleGenAI({ apiKey: resolvedApiKey })
const usage: Usage = {
model: 'gemini-embedding-2',
inputTokens: 0,
outputTokens: 0,
}
const usage: Usage = {
model: 'gemini-embedding-2',
inputTokens: 0,
outputTokens: 0,
}
const embVec = await generateMultimodalEmbedding(ai, data, params.mediaType)
const embVec = await generateMultimodalEmbedding(ai, data, params.mediaType)
return {
embedding: embVec,
usage,
}
}
return {
embedding: embVec,
usage,
}
},
)
/**
* Standard aspect ratios for video embedding:
@@ -434,121 +448,296 @@ export interface GenerateVideoChunkEmbeddingParams {
chunkKey: string
}
export async function generateVideoChunkEmbeddingActivity(
params: GenerateVideoChunkEmbeddingParams,
): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const isGemma = provider === 'gemma-embedding-2-openai' || params.chunkKey.endsWith('.tar')
export const generateVideoChunkEmbeddingActivity = defineStep(
async function generateVideoChunkEmbeddingActivity(
params: GenerateVideoChunkEmbeddingParams,
): Promise<{
embedding: number[]
usage: Usage
}> {
const provider = process.env.EMBEDDING_PROVIDER || 'gemini-embedding-2'
const isGemma = provider === 'gemma-embedding-2-openai' || params.chunkKey.endsWith('.tar')
if (isGemma) {
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.chunkKey,
)
if (isGemma) {
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.chunkKey,
)
let filesMap: Map<string, File>
try {
const archive = new Bun.Archive(data)
filesMap = await archive.files()
} catch (err) {
throw ApplicationFailure.create({
message: `Failed to read chunk archive ${params.chunkKey}: ${err instanceof Error ? err.message : String(err)}`,
nonRetryable: true,
})
}
const frameFiles = Array.from(filesMap.keys())
.filter((f) => /\.(jpe?g|png|webp)$/i.test(f))
.sort()
if (frameFiles.length === 0) {
throw ApplicationFailure.create({
message: `No extracted frames found in chunk archive ${params.chunkKey}`,
nonRetryable: true,
})
}
let targetFrameCount = 14
try {
const firstFile = filesMap.get(frameFiles[0])
if (firstFile) {
const firstFileBytes = await firstFile.bytes()
if (firstFileBytes.byteLength > 0) {
const meta = await sharp(Buffer.from(firstFileBytes)).metadata()
targetFrameCount = getGemmaEmbeddingTargetFrameCount(meta.width, meta.height)
}
}
} catch {
targetFrameCount = 14
}
let selectedFrames = frameFiles
if (selectedFrames.length > targetFrameCount) {
const step = (selectedFrames.length - 1) / (targetFrameCount - 1)
selectedFrames = Array.from(
{ length: targetFrameCount },
(_, i) => selectedFrames[Math.round(i * step)],
)
}
const input: OpenAiEmbeddingItem[] = await Promise.all(
selectedFrames.map(async (f) => {
const file = filesMap.get(f)!
const fileBytes = await file.bytes()
if (fileBytes.byteLength === 0) {
throw ApplicationFailure.create({
message: `Extracted frame ${f} in chunk archive ${params.chunkKey} is empty (0 bytes)`,
nonRetryable: true,
})
}
return {
type: 'image_url',
// eslint-disable-next-line @typescript-eslint/naming-convention
image_url: {
url: `data:image/jpeg;base64,${Buffer.from(fileBytes).toString('base64')}`,
},
}
}),
)
return await callGemmaEmbeddingServer(input)
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
nonRetryable: true,
})
}
const resolvedApiKey = process.env[apiKey] || apiKey
const ai = new GoogleGenAI({ apiKey: resolvedApiKey })
const usage: Usage = {
model: 'gemini-embedding-2',
inputTokens: 0,
outputTokens: 0,
}
let filesMap: Map<string, File>
try {
const archive = new Bun.Archive(data)
filesMap = await archive.files()
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.chunkKey,
)
const embVec = await generateMultimodalEmbedding(ai, data, 'video/mp4')
return {
embedding: embVec,
usage,
}
} catch (err) {
throw ApplicationFailure.create({
message: `Failed to read chunk archive ${params.chunkKey}: ${err instanceof Error ? err.message : String(err)}`,
nonRetryable: true,
message: `Failed to generate video chunk embedding for key ${params.chunkKey}: ${err instanceof Error ? err.message : String(err)}`,
nonRetryable: false,
})
}
},
)
const frameFiles = Array.from(filesMap.keys())
.filter((f) => /\.(jpe?g|png|webp)$/i.test(f))
.sort()
export interface TranscodeVideoChunkParams {
assetId: string
assetKey?: string
filePath?: string
startTime: number
endTime: number
format?: 'video' | 'frames'
}
if (frameFiles.length === 0) {
throw ApplicationFailure.create({
message: `No extracted frames found in chunk archive ${params.chunkKey}`,
nonRetryable: true,
})
}
let targetFrameCount = 14
try {
const firstFile = filesMap.get(frameFiles[0])
if (firstFile) {
const firstFileBytes = await firstFile.bytes()
if (firstFileBytes.byteLength > 0) {
const meta = await sharp(Buffer.from(firstFileBytes)).metadata()
targetFrameCount = getGemmaEmbeddingTargetFrameCount(meta.width, meta.height)
}
}
} catch {
targetFrameCount = 14
}
let selectedFrames = frameFiles
if (selectedFrames.length > targetFrameCount) {
const step = (selectedFrames.length - 1) / (targetFrameCount - 1)
selectedFrames = Array.from(
{ length: targetFrameCount },
(_, i) => selectedFrames[Math.round(i * step)],
)
}
const input: OpenAiEmbeddingItem[] = await Promise.all(
selectedFrames.map(async (f) => {
const file = filesMap.get(f)!
const fileBytes = await file.bytes()
if (fileBytes.byteLength === 0) {
throw ApplicationFailure.create({
message: `Extracted frame ${f} in chunk archive ${params.chunkKey} is empty (0 bytes)`,
nonRetryable: true,
})
}
return {
type: 'image_url',
// eslint-disable-next-line @typescript-eslint/naming-convention
image_url: {
url: `data:image/jpeg;base64,${Buffer.from(fileBytes).toString('base64')}`,
},
}
}),
)
return await callGemmaEmbeddingServer(input)
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
export const transcodeVideoChunkActivity = defineStep(async function transcodeVideoChunkActivity(
params: TranscodeVideoChunkParams,
): Promise<{ chunkKey: string }> {
if (!params.filePath && !params.assetKey) {
throw ApplicationFailure.create({
message: 'GEMINI_API_KEY environment variable is not configured',
message: 'Neither filePath nor assetKey was provided for video chunk transcoding',
nonRetryable: true,
})
}
const resolvedApiKey = process.env[apiKey] || apiKey
const ai = new GoogleGenAI({ apiKey: resolvedApiKey })
if (params.format === 'frames') {
const tmpDir = transcodeService.createTempDir('video-chunk-frames-')
try {
const bucket = process.env.S3_BUCKET || 'shumai'
const inputSource =
params.filePath ||
(params.assetKey ? await s3Service.resolveInput(bucket, params.assetKey) : '')
if (!inputSource) {
throw ApplicationFailure.create({
message: 'Neither filePath nor assetKey was provided for video chunk transcoding',
nonRetryable: true,
})
}
const isRemote = inputSource.startsWith('http://') || inputSource.startsWith('https://')
const usage: Usage = {
model: 'gemini-embedding-2',
inputTokens: 0,
outputTokens: 0,
const duration = Math.max(0.1, params.endTime - params.startTime)
const fpsFilter = duration > 16 ? `fps=${16 / duration}` : 'fps=1'
const vframesArgs = duration > 16 ? ['-vframes', '16'] : []
await execFileAsync('ffmpeg', [
'-y',
'-loglevel',
'warning',
...(isRemote
? ['-reconnect', '1', '-reconnect_streamed', '1', '-reconnect_delay_max', '5']
: []),
'-ss',
params.startTime.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-t',
duration.toString(),
'-vf',
fpsFilter,
...vframesArgs,
'-q:v',
'2',
path.join(tmpDir, 'frame_%04d.jpg'),
])
const frameFiles = fs
.readdirSync(tmpDir)
.filter((f) => /\.(jpe?g|png|webp)$/i.test(f))
.sort()
const archiveEntries: Record<string, Buffer> = {}
for (const file of frameFiles) {
archiveEntries[file] = fs.readFileSync(path.join(tmpDir, file))
}
const archive = new Bun.Archive(archiveEntries)
const archiveBytes = await archive.bytes()
const archiveData = Buffer.from(
archiveBytes.buffer,
archiveBytes.byteOffset,
archiveBytes.byteLength,
)
const key = `files/${params.assetId}/tmp-embedding-chunks/frames-${params.startTime}-${params.endTime}-${ulid()}.tar`
await s3Service.putObject(bucket, key, archiveData, archiveData.length, 'application/x-tar')
return { chunkKey: key }
} catch (err) {
if (err instanceof ApplicationFailure) {
throw err
}
throw ApplicationFailure.create({
message: `Failed to extract video chunk frames for ${params.startTime}-${params.endTime}: ${err instanceof Error ? err.message : String(err)}`,
nonRetryable: false,
})
} finally {
transcodeService.removeDir(tmpDir)
}
}
const chunkTmp = path.join(os.tmpdir(), `video-chunk-${Date.now()}-${ulid()}.mp4`)
try {
const { buffer: data } = await s3Service.getObject(
process.env.S3_BUCKET || 'shumai',
params.chunkKey,
)
const embVec = await generateMultimodalEmbedding(ai, data, 'video/mp4')
return {
embedding: embVec,
usage,
const bucket = process.env.S3_BUCKET || 'shumai'
const inputSource =
params.filePath ||
(params.assetKey ? await s3Service.resolveInput(bucket, params.assetKey) : '')
if (!inputSource) {
throw ApplicationFailure.create({
message: 'Neither filePath nor assetKey was provided for video chunk transcoding',
nonRetryable: true,
})
}
const isRemote = inputSource.startsWith('http://') || inputSource.startsWith('https://')
// Slice video segment using ffmpeg, force 1fps and remove audio for gemini-embedding-2 optimization
// Note: -ss placed before -i for fast input seeking via demuxer Range requests
await execFileAsync('ffmpeg', [
'-y',
'-loglevel',
'warning',
...(isRemote
? ['-reconnect', '1', '-reconnect_streamed', '1', '-reconnect_delay_max', '5']
: []),
'-ss',
params.startTime.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-t',
(params.endTime - params.startTime).toString(),
'-vf',
'fps=1',
'-c:v',
'libx264',
'-preset',
'ultrafast',
'-crf',
'28',
'-an',
chunkTmp,
])
const chunkData = fs.readFileSync(chunkTmp)
// Upload to S3
const key = `files/${params.assetId}/tmp-embedding-chunks/chunk-${params.startTime}-${params.endTime}-${ulid()}.mp4`
await s3Service.putObject(bucket, key, chunkData, chunkData.length, 'video/mp4')
return { chunkKey: key }
} catch (err) {
if (err instanceof ApplicationFailure) {
throw err
}
throw ApplicationFailure.create({
message: `Failed to generate video chunk embedding for key ${params.chunkKey}: ${err instanceof Error ? err.message : String(err)}`,
message: `Failed to transcode video chunk for ${params.startTime}-${params.endTime}: ${err instanceof Error ? err.message : String(err)}`,
nonRetryable: false,
})
} finally {
if (fs.existsSync(chunkTmp)) {
try {
fs.unlinkSync(chunkTmp)
} catch (e) {
console.error('Failed to cleanup local chunk file:', e)
}
}
}
}
})
export const deleteS3ObjectActivity = defineStep(
async function deleteS3ObjectActivity(params: { key: string }): Promise<void> {
const bucket = process.env.S3_BUCKET || 'shumai'
await s3Service.deleteObject(bucket, params.key)
},
{ runOnCancel: true },
)
+1 -6
View File
@@ -604,9 +604,7 @@ export function formatSkillsForPrompt(
}
import { WorkflowTaskType } from '@shumai/db'
import { registerActivities, registerWorkflow } from '@shumai/workflow-core'
import * as agentActivities from './activities/agent'
import * as aiActivities from './activities/ai'
import { registerWorkflow } from '@shumai/workflow-core'
import { agentAutofillMedia } from './workflows/agent-autofill'
import { agentChat } from './workflows/agent-chat'
import { agentEmbeddingMedia } from './workflows/agent-embedding'
@@ -615,9 +613,6 @@ export function initAgentWorkflows() {
registerWorkflow(WorkflowTaskType.ai_embedding, agentEmbeddingMedia)
registerWorkflow(WorkflowTaskType.ai_metadata_autofill, agentAutofillMedia)
registerWorkflow(WorkflowTaskType.chat, agentChat)
registerActivities(agentActivities)
registerActivities(aiActivities)
}
export * from './activities/agent'
@@ -34,8 +34,7 @@ export async function resolveAnnotationsById(
})
if (entry) {
const details = (entry.data as Record<string, unknown> | null)?.details as
| ShumaiMessageContext
| undefined
ShumaiMessageContext | undefined
const entryAssetId = entry.assetId || details?.currentAsset?.id
if (entryAssetId && entryAssetId !== assetId) {
throw new Error(
+2 -4
View File
@@ -355,8 +355,7 @@ export function createReadAssetTool(
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode screenshot output
const output = completedTask.output as any
const screenshots = output?.screenshots as
| Array<{ key: string; timestamp: number }>
| undefined
Array<{ key: string; timestamp: number }> | undefined
if (!screenshots || screenshots.length === 0) {
throw new Error(
@@ -620,8 +619,7 @@ export function createReadAssetTool(
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode screenshot output
const output = completedTask.output as any
const screenshots = output?.screenshots as
| Array<{ key: string; timestamp: number }>
| undefined
Array<{ key: string; timestamp: number }> | undefined
if (!screenshots || screenshots.length === 0) {
throw new Error('No screenshots were generated.')
@@ -2,14 +2,29 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'
import { agentAutofillMedia } from './agent-autofill'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import * as workflowUtils from '@shumai/workflow-core'
import * as workflowCore from '@shumai/workflow-core'
import * as agentActivities from '../activities/agent'
vi.mock('@shumai/workflow-core', async () => {
const actual = await vi.importActual('@shumai/workflow-core')
return {
...actual,
getActivities: vi.fn(),
executeActivity: vi.fn(),
updateTaskStatusActivity: vi.fn(),
updateTaskUsageActivity: vi.fn(),
}
})
vi.mock('../activities/agent', async () => {
const actual = await vi.importActual('../activities/agent')
return {
...actual,
getAssetActivity: vi.fn(),
getProjectAutofillFieldsActivity: vi.fn(),
getAgentAutofillContextActivity: vi.fn(),
autofillAiActivity: vi.fn(),
updateAssetMetadataActivity: vi.fn(),
createCommentActivity: vi.fn(),
updateCommentActivity: vi.fn(),
}
})
@@ -23,32 +38,17 @@ describe('Agent Autofill Workflow', () => {
vi.clearAllMocks()
mockActivities = {
updateTaskStatusActivity: Object.assign(vi.fn(), {
_activityName: 'updateTaskStatusActivity',
}),
getAssetActivity: Object.assign(vi.fn(), { _activityName: 'getAssetActivity' }),
getProjectAutofillFieldsActivity: Object.assign(vi.fn(), {
_activityName: 'getProjectAutofillFieldsActivity',
}),
getAgentAutofillContextActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentAutofillContextActivity',
}),
getAssetAutofillContextActivity: Object.assign(vi.fn(), {
_activityName: 'getAssetAutofillContextActivity',
}),
autofillAiActivity: Object.assign(vi.fn(), { _activityName: 'autofillAiActivity' }),
updateTaskUsageActivity: Object.assign(vi.fn(), { _activityName: 'updateTaskUsageActivity' }),
updateAssetMetadataActivity: Object.assign(vi.fn(), {
_activityName: 'updateAssetMetadataActivity',
}),
createCommentActivity: Object.assign(vi.fn(), { _activityName: 'createCommentActivity' }),
updateCommentActivity: Object.assign(vi.fn(), { _activityName: 'updateCommentActivity' }),
getAgentWorkerQueueActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentWorkerQueueActivity',
}),
updateTaskStatusActivity: vi.mocked(workflowCore.updateTaskStatusActivity),
getAssetActivity: vi.mocked(agentActivities.getAssetActivity),
getProjectAutofillFieldsActivity: vi.mocked(agentActivities.getProjectAutofillFieldsActivity),
getAgentAutofillContextActivity: vi.mocked(agentActivities.getAgentAutofillContextActivity),
autofillAiActivity: vi.mocked(agentActivities.autofillAiActivity),
updateTaskUsageActivity: vi.mocked(workflowCore.updateTaskUsageActivity),
updateAssetMetadataActivity: vi.mocked(agentActivities.updateAssetMetadataActivity),
createCommentActivity: vi.mocked(agentActivities.createCommentActivity),
updateCommentActivity: vi.mocked(agentActivities.updateCommentActivity),
}
mockActivities.getAgentWorkerQueueActivity.mockResolvedValue('agent_queue')
mockActivities.updateTaskStatusActivity.mockResolvedValue({})
mockActivities.createCommentActivity.mockResolvedValue({ id: 'comment-placeholder-id' })
mockActivities.getAssetActivity.mockResolvedValue({
@@ -69,13 +69,6 @@ describe('Agent Autofill Workflow', () => {
sessionId: 'session-123',
usage: { inputTokens: 10, outputTokens: 15, model: 'gemini-model' },
})
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mockActivities contains vi.fn mock functions which are cast to expected activity proxy types
vi.mocked(workflowUtils.getActivities).mockReturnValue(mockActivities as any)
vi.mocked(workflowUtils.executeActivity).mockImplementation(async (_queue, act, ...args) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- act is one of the mocked activities
return (act as any)(...args)
})
})
it('should run agent autofill workflow successfully', async () => {
@@ -93,9 +86,6 @@ describe('Agent Autofill Workflow', () => {
await agentAutofillMedia(task)
// Verify queue discovery
expect(mockActivities.getAgentWorkerQueueActivity).toHaveBeenCalled()
// Verify task processing status
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
+38 -51
View File
@@ -1,38 +1,33 @@
import type { WorkflowTask } from '@shumai/db'
import { executeActivity, getActivities, TaskQueueAgent } from '@shumai/workflow-core'
import { ApplicationFailure } from '@temporalio/workflow'
import {
ApplicationFailure,
updateTaskStatusActivity,
updateTaskUsageActivity,
} from '@shumai/workflow-core'
import { getProxyType } from '@shumai/core/src/utils/mime'
import {
autofillAiActivity,
createCommentActivity,
getAgentAutofillContextActivity,
getAssetActivity,
getProjectAutofillFieldsActivity,
updateAssetMetadataActivity,
updateCommentActivity,
} from '../activities/agent'
export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
const {
updateTaskStatusActivity,
getAssetActivity,
getProjectAutofillFieldsActivity,
getAgentAutofillContextActivity,
autofillAiActivity,
updateTaskUsageActivity,
updateAssetMetadataActivity,
createCommentActivity,
updateCommentActivity,
getAgentWorkerQueueActivity,
} = getActivities()
let placeholderCommentId: string | undefined
let agentWorkerQueue = ''
try {
// 0. Discover queues
agentWorkerQueue = await executeActivity(TaskQueueAgent, getAgentWorkerQueueActivity)
// Update status to processing
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'processing',
})
// 0. Create Placeholder Comment
const payload = task.payload
const placeholder = await executeActivity(agentWorkerQueue, createCommentActivity, {
const placeholder = await createCommentActivity({
assetId: task.assetId,
message: '__AUTOFILL__',
sessionId: payload?.agent?.sessionId || task.id,
@@ -41,7 +36,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
placeholderCommentId = placeholder.id
// 1. Get Asset
const asset = await executeActivity(agentWorkerQueue, getAssetActivity, task.assetId)
const asset = await getAssetActivity(task.assetId)
if (!asset || !asset.project) {
throw ApplicationFailure.create({ message: 'Asset or project not found', nonRetryable: true })
}
@@ -49,19 +44,15 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
const projectId = asset.project.id
// 2. Get Project Autofill Fields
const fields = await executeActivity(
agentWorkerQueue,
getProjectAutofillFieldsActivity,
projectId,
)
const fields = await getProjectAutofillFieldsActivity(projectId)
if (fields.length === 0) {
if (placeholderCommentId) {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: 'Autofill completed: No autofill fields defined in project.',
})
}
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'completed',
})
@@ -69,7 +60,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
}
// 3. Fetch Agent Context
const context = await executeActivity(agentWorkerQueue, getAgentAutofillContextActivity, {
const context = await getAgentAutofillContextActivity({
teamId,
})
@@ -91,7 +82,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
: undefined
: undefined
const aiResult = await executeActivity(agentWorkerQueue, autofillAiActivity, {
const aiResult = await autofillAiActivity({
teamId,
assetId: asset.id,
assetName: asset.name,
@@ -99,19 +90,17 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
duration,
pageCount,
projectId: asset.projectId ?? undefined,
fields: fields.map(
(f: { key: string; config: Record<string, unknown>; description?: string | null }) => ({
id: f.key,
config: f.config as unknown as PrismaJson.FieldConfig,
description: f.description,
}),
),
fields: fields.map((f) => ({
id: f.key,
config: (f.config as unknown as PrismaJson.FieldConfig) ?? {},
description: f.description,
})),
context,
})
if (aiResult.usage) {
// Update Usage
await executeActivity(agentWorkerQueue, updateTaskUsageActivity, {
await updateTaskUsageActivity({
taskId: task.id,
inputTokens: aiResult.usage.inputTokens,
outputTokens: aiResult.usage.outputTokens,
@@ -129,7 +118,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
}))
if (metadataUpdates.length > 0) {
await executeActivity(agentWorkerQueue, updateAssetMetadataActivity, {
await updateAssetMetadataActivity({
assetId: asset.id,
metadata: metadataUpdates,
})
@@ -137,7 +126,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
// 7. Update Placeholder Comment
if (placeholderCommentId) {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: 'Autofill completed successfully.',
sessionId: aiResult.sessionId,
@@ -145,7 +134,7 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
}
// Update status to completed
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'completed',
})
@@ -153,9 +142,9 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
console.error(`AgentAutofillMedia failed for task ${task.id}:`, err)
// Update placeholder comment with error message
if (placeholderCommentId && agentWorkerQueue) {
if (placeholderCommentId) {
try {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: `Autofill failed: ${err instanceof Error ? err.message : String(err)}`,
})
@@ -165,13 +154,11 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
}
// Update status to failed
if (agentWorkerQueue) {
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
}
await updateTaskStatusActivity({
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
throw err
}
}
+37 -45
View File
@@ -2,14 +2,34 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'
import { agentChat } from './agent-chat'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import * as workflowUtils from '@shumai/workflow-core'
import * as workflowCore from '@shumai/workflow-core'
import * as agentActivities from '../activities/agent'
vi.mock('@shumai/workflow-core', async () => {
const actual = await vi.importActual('@shumai/workflow-core')
return {
...actual,
getActivities: vi.fn(),
executeActivity: vi.fn(),
updateTaskStatusActivity: vi.fn(),
updateTaskUsageActivity: vi.fn(),
}
})
vi.mock('../activities/agent', async () => {
const actual = await vi.importActual('../activities/agent')
return {
...actual,
getCommentActivity: vi.fn(),
createCommentActivity: vi.fn(),
getAssetActivity: vi.fn(),
initializeAgentSessionActivity: vi.fn(),
getAgentChatContextActivity: vi.fn(),
agentChatActivity: vi.fn(),
updateCommentActivity: vi.fn(),
deleteCommentActivity: vi.fn(),
getAssetPathHierarchyActivity: vi.fn(),
getAssetPathContextActivity: vi.fn(),
generateSessionNameActivity: vi.fn(),
getUserTeamInfoActivity: vi.fn(),
}
})
@@ -23,40 +43,22 @@ describe('Agent Chat Workflow', () => {
vi.clearAllMocks()
mockActivities = {
updateTaskStatusActivity: Object.assign(vi.fn(), {
_activityName: 'updateTaskStatusActivity',
}),
getCommentActivity: Object.assign(vi.fn(), { _activityName: 'getCommentActivity' }),
createCommentActivity: Object.assign(vi.fn(), { _activityName: 'createCommentActivity' }),
getAssetActivity: Object.assign(vi.fn(), { _activityName: 'getAssetActivity' }),
initializeAgentSessionActivity: Object.assign(vi.fn(), {
_activityName: 'initializeAgentSessionActivity',
}),
getAgentChatContextActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentChatContextActivity',
}),
getAgentWorkerQueueActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentWorkerQueueActivity',
}),
agentChatActivity: Object.assign(vi.fn(), { _activityName: 'agentChatActivity' }),
updateCommentActivity: Object.assign(vi.fn(), { _activityName: 'updateCommentActivity' }),
updateTaskUsageActivity: Object.assign(vi.fn(), { _activityName: 'updateTaskUsageActivity' }),
deleteCommentActivity: Object.assign(vi.fn(), { _activityName: 'deleteCommentActivity' }),
getAssetPathHierarchyActivity: Object.assign(vi.fn(), {
_activityName: 'getAssetPathHierarchyActivity',
}),
getAssetPathContextActivity: Object.assign(vi.fn(), {
_activityName: 'getAssetPathContextActivity',
}),
generateSessionNameActivity: Object.assign(vi.fn(), {
_activityName: 'generateSessionNameActivity',
}),
getUserTeamInfoActivity: Object.assign(vi.fn(), {
_activityName: 'getUserTeamInfoActivity',
}),
updateTaskStatusActivity: vi.mocked(workflowCore.updateTaskStatusActivity),
getCommentActivity: vi.mocked(agentActivities.getCommentActivity),
createCommentActivity: vi.mocked(agentActivities.createCommentActivity),
getAssetActivity: vi.mocked(agentActivities.getAssetActivity),
initializeAgentSessionActivity: vi.mocked(agentActivities.initializeAgentSessionActivity),
getAgentChatContextActivity: vi.mocked(agentActivities.getAgentChatContextActivity),
agentChatActivity: vi.mocked(agentActivities.agentChatActivity),
updateCommentActivity: vi.mocked(agentActivities.updateCommentActivity),
updateTaskUsageActivity: vi.mocked(workflowCore.updateTaskUsageActivity),
deleteCommentActivity: vi.mocked(agentActivities.deleteCommentActivity),
getAssetPathHierarchyActivity: vi.mocked(agentActivities.getAssetPathHierarchyActivity),
getAssetPathContextActivity: vi.mocked(agentActivities.getAssetPathContextActivity),
generateSessionNameActivity: vi.mocked(agentActivities.generateSessionNameActivity),
getUserTeamInfoActivity: vi.mocked(agentActivities.getUserTeamInfoActivity),
}
mockActivities.getAgentWorkerQueueActivity.mockResolvedValue('agent_queue')
mockActivities.updateTaskStatusActivity.mockResolvedValue({})
mockActivities.createCommentActivity.mockResolvedValue({ id: 'comment-placeholder-id' })
mockActivities.generateSessionNameActivity.mockResolvedValue(undefined)
@@ -92,13 +94,6 @@ describe('Agent Chat Workflow', () => {
sessionId: 'session-123',
usage: { inputTokens: 10, outputTokens: 20, model: 'gemini-model' },
})
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mockActivities contains vi.fn mock functions which are cast to expected activity proxy types
vi.mocked(workflowUtils.getActivities).mockReturnValue(mockActivities as any)
vi.mocked(workflowUtils.executeActivity).mockImplementation(async (_queue, act, ...args) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- act is one of the mocked activities
return (act as any)(...args)
})
})
it('should run agent chat workflow successfully with default configurations', async () => {
@@ -116,9 +111,6 @@ describe('Agent Chat Workflow', () => {
await agentChat(task)
// Verify queue discovery
expect(mockActivities.getAgentWorkerQueueActivity).toHaveBeenCalled()
// Verify task status processing
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
+43 -66
View File
@@ -1,6 +1,22 @@
import { ApplicationFailure } from '@temporalio/workflow'
import type { WorkflowTask } from '@shumai/db'
import { executeActivity, getActivities, TaskQueueAgent } from '@shumai/workflow-core'
import {
ApplicationFailure,
updateTaskStatusActivity,
updateTaskUsageActivity,
} from '@shumai/workflow-core'
import {
agentChatActivity,
createCommentActivity,
generateSessionNameActivity,
getAgentChatContextActivity,
getAssetActivity,
getAssetPathContextActivity,
getAssetPathHierarchyActivity,
getCommentActivity,
getUserTeamInfoActivity,
initializeAgentSessionActivity,
updateCommentActivity,
} from '../activities/agent'
import type {
ShumaiMessageContext,
ShumaiAssetContext,
@@ -11,32 +27,11 @@ import { getProxyType } from '@shumai/core/src/utils/mime'
import { secondToDegree } from '@shumai/core/src/utils/model3d'
export async function agentChat(task: WorkflowTask): Promise<void> {
const {
updateTaskStatusActivity,
getAssetActivity,
getCommentActivity,
getAgentChatContextActivity,
agentChatActivity,
createCommentActivity,
updateCommentActivity,
updateTaskUsageActivity,
getAgentWorkerQueueActivity,
initializeAgentSessionActivity,
getAssetPathHierarchyActivity,
getAssetPathContextActivity,
generateSessionNameActivity,
getUserTeamInfoActivity,
} = getActivities()
let placeholderCommentId: string | undefined
let agentWorkerQueue = ''
try {
// 0. Discover queue
agentWorkerQueue = await executeActivity(TaskQueueAgent, getAgentWorkerQueueActivity)
// Update status to processing
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'processing',
})
@@ -65,7 +60,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
let commentAnnotations: unknown[] | undefined
if (userCommentId) {
userComment = await executeActivity(agentWorkerQueue, getCommentActivity, userCommentId)
userComment = await getCommentActivity(userCommentId)
if (!userComment) {
throw ApplicationFailure.create({ message: 'User comment not found', nonRetryable: true })
}
@@ -84,7 +79,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
// 2. Create Placeholder Comment
const placeholder = await executeActivity(agentWorkerQueue, createCommentActivity, {
const placeholder = await createCommentActivity({
assetId: task.assetId,
message: '__CHAT__',
sessionId: sessionId || 'pending',
@@ -119,7 +114,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
// 3. Get Asset
const asset = await executeActivity(agentWorkerQueue, getAssetActivity, task.assetId)
const asset = await getAssetActivity(task.assetId)
if (!asset || !asset.project) {
throw ApplicationFailure.create({ message: 'Asset or project not found', nonRetryable: true })
}
@@ -134,7 +129,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
nonRetryable: true,
})
}
sessionId = await executeActivity(agentWorkerQueue, initializeAgentSessionActivity, {
sessionId = await initializeAgentSessionActivity({
teamId,
agentId: agentId,
userCommentId,
@@ -144,18 +139,14 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
// 4b. Fetch Agent Context
const context = await executeActivity(agentWorkerQueue, getAgentChatContextActivity, {
const context = await getAgentChatContextActivity({
teamId,
agentId: agentId,
userId: payload.agent?.userId,
projectId: payload.projectId,
})
const pathHierarchy = await executeActivity(
agentWorkerQueue,
getAssetPathHierarchyActivity,
task.assetId,
)
const pathHierarchy = await getAssetPathHierarchyActivity(task.assetId)
const assetPath = pathHierarchy?.path || ''
const ancestors = pathHierarchy?.ancestors || []
@@ -180,11 +171,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
if (userCommentId && userComment?.attachments) {
for (const att of userComment.attachments) {
if (att.asset) {
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
att.asset.id,
)
const filePath = await getAssetPathContextActivity(att.asset.id)
const attProxyType =
(att.asset.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(att.asset.mediaType, att.asset.name) ||
@@ -203,13 +190,9 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
} else if (payload.agent?.attachedFiles) {
for (const fileId of payload.agent.attachedFiles) {
const file = await executeActivity(agentWorkerQueue, getAssetActivity, fileId)
const file = await getAssetActivity(fileId)
if (file) {
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
fileId,
)
const filePath = await getAssetPathContextActivity(fileId)
const fileProxyType =
(file.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(file.mediaType, file.name) ||
@@ -232,13 +215,9 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
const referencedAssets: ShumaiAttachedFileContext[] = []
if (payload.agent?.assetIds) {
for (const assetId of payload.agent.assetIds) {
const referencedAsset = await executeActivity(agentWorkerQueue, getAssetActivity, assetId)
const referencedAsset = await getAssetActivity(assetId)
if (referencedAsset) {
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
assetId,
)
const filePath = await getAssetPathContextActivity(assetId)
const refProxyType =
(referencedAsset.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(referencedAsset.mediaType, referencedAsset.name) ||
@@ -265,7 +244,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
const targetUserId = userCommentId ? commentCreatorId : payload.agent?.userId
let userInfo: { name: string; role: string } | null = null
if (targetUserId) {
userInfo = await executeActivity(agentWorkerQueue, getUserTeamInfoActivity, {
userInfo = await getUserTeamInfoActivity({
userId: targetUserId,
teamId,
})
@@ -325,7 +304,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
folderId = asset.parentId
}
const aiResult = await executeActivity(agentWorkerQueue, agentChatActivity, {
const aiResult = await agentChatActivity({
taskId: task.id,
teamId,
agentId: agentId,
@@ -344,7 +323,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
// 7. Update Placeholder Comment
if (placeholderCommentId) {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: aiResult.text,
sessionId: aiResult.sessionId,
@@ -353,7 +332,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
// 8. Update Usage
if (aiResult.usage) {
await executeActivity(agentWorkerQueue, updateTaskUsageActivity, {
await updateTaskUsageActivity({
taskId: task.id,
inputTokens: aiResult.usage.inputTokens,
outputTokens: aiResult.usage.outputTokens,
@@ -363,7 +342,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
// 9. Generate Session Name for new chats
if (isNewChat) {
await executeActivity(agentWorkerQueue, generateSessionNameActivity, {
await generateSessionNameActivity({
teamId,
agentId,
prompt,
@@ -375,7 +354,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
// Update status to completed
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'completed',
output: { sessionId: aiResult.sessionId },
@@ -384,7 +363,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
console.error(`AgentChat failed for task ${task.id}:`, err)
// Update placeholder comment with error message
if (placeholderCommentId && agentWorkerQueue) {
if (placeholderCommentId) {
try {
let errorMessage = 'Sorry, I encountered an error while processing your request.'
if (err instanceof Error) {
@@ -406,7 +385,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
errorMessage = 'Agent execution stopped.'
}
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: errorMessage,
})
@@ -416,13 +395,11 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
// Update status to failed
if (agentWorkerQueue) {
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
}
await updateTaskStatusActivity({
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
throw err
}
}
@@ -2,14 +2,39 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'
import { agentEmbeddingMedia } from './agent-embedding'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import * as workflowUtils from '@shumai/workflow-core'
import * as workflowCore from '@shumai/workflow-core'
import * as agentActivities from '../activities/agent'
import * as aiActivities from '../activities/ai'
vi.mock('@shumai/workflow-core', async () => {
const actual = await vi.importActual('@shumai/workflow-core')
return {
...actual,
getActivities: vi.fn(),
executeActivity: vi.fn(),
updateTaskStatusActivity: vi.fn(),
updateTaskUsageActivity: vi.fn(),
}
})
vi.mock('../activities/agent', async () => {
const actual = await vi.importActual('../activities/agent')
return {
...actual,
createCommentActivity: vi.fn(),
updateCommentActivity: vi.fn(),
}
})
vi.mock('../activities/ai', async () => {
const actual = await vi.importActual('../activities/ai')
return {
...actual,
getEmbeddingContextActivity: vi.fn(),
generateImageEmbeddingActivity: vi.fn(),
generateVideoChunkEmbeddingActivity: vi.fn(),
saveAssetEmbeddingsActivity: vi.fn(),
deleteAssetEmbeddingsActivity: vi.fn(),
transcodeVideoChunkActivity: vi.fn(),
deleteS3ObjectActivity: vi.fn(),
}
})
@@ -23,49 +48,23 @@ describe('Agent Embedding Workflow', () => {
vi.clearAllMocks()
mockActivities = {
updateTaskStatusActivity: Object.assign(vi.fn(), {
_activityName: 'updateTaskStatusActivity',
}),
getEmbeddingContextActivity: Object.assign(vi.fn(), {
_activityName: 'getEmbeddingContextActivity',
}),
generateImageEmbeddingActivity: Object.assign(vi.fn(), {
_activityName: 'generateImageEmbeddingActivity',
}),
generateVideoChunkEmbeddingActivity: Object.assign(vi.fn(), {
_activityName: 'generateVideoChunkEmbeddingActivity',
}),
saveAssetEmbeddingsActivity: Object.assign(vi.fn(), {
_activityName: 'saveAssetEmbeddingsActivity',
}),
deleteAssetEmbeddingsActivity: Object.assign(vi.fn(), {
_activityName: 'deleteAssetEmbeddingsActivity',
}),
updateTaskUsageActivity: Object.assign(vi.fn(), { _activityName: 'updateTaskUsageActivity' }),
createCommentActivity: Object.assign(vi.fn(), { _activityName: 'createCommentActivity' }),
updateCommentActivity: Object.assign(vi.fn(), { _activityName: 'updateCommentActivity' }),
getAgentWorkerQueueActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentWorkerQueueActivity',
}),
getTranscodeWorkerQueueActivity: Object.assign(vi.fn(), {
_activityName: 'getTranscodeWorkerQueueActivity',
}),
downloadMediaToTmpActivity: Object.assign(vi.fn(), {
_activityName: 'downloadMediaToTmpActivity',
}),
cleanupTmpDirActivity: Object.assign(vi.fn(), {
_activityName: 'cleanupTmpDirActivity',
}),
transcodeVideoChunkActivity: Object.assign(vi.fn(), {
_activityName: 'transcodeVideoChunkActivity',
}),
deleteS3ObjectActivity: Object.assign(vi.fn(), {
_activityName: 'deleteS3ObjectActivity',
}),
updateTaskStatusActivity: vi.mocked(workflowCore.updateTaskStatusActivity),
getEmbeddingContextActivity: vi.mocked(aiActivities.getEmbeddingContextActivity),
generateImageEmbeddingActivity: vi.mocked(aiActivities.generateImageEmbeddingActivity),
generateVideoChunkEmbeddingActivity: vi.mocked(
aiActivities.generateVideoChunkEmbeddingActivity,
),
saveAssetEmbeddingsActivity: vi.mocked(aiActivities.saveAssetEmbeddingsActivity),
deleteAssetEmbeddingsActivity: vi.mocked(aiActivities.deleteAssetEmbeddingsActivity),
updateTaskUsageActivity: vi.mocked(workflowCore.updateTaskUsageActivity),
createCommentActivity: vi.mocked(agentActivities.createCommentActivity),
updateCommentActivity: vi.mocked(agentActivities.updateCommentActivity),
downloadMediaToTmpActivity: vi.fn(),
cleanupTmpDirActivity: vi.fn(),
transcodeVideoChunkActivity: vi.mocked(aiActivities.transcodeVideoChunkActivity),
deleteS3ObjectActivity: vi.mocked(aiActivities.deleteS3ObjectActivity),
}
mockActivities.getAgentWorkerQueueActivity.mockResolvedValue('agent_queue')
mockActivities.getTranscodeWorkerQueueActivity.mockResolvedValue('transcode_queue')
mockActivities.updateTaskStatusActivity.mockResolvedValue({})
mockActivities.createCommentActivity.mockResolvedValue({ id: 'comment-placeholder-id' })
mockActivities.getEmbeddingContextActivity.mockResolvedValue({
@@ -99,13 +98,6 @@ describe('Agent Embedding Workflow', () => {
)
mockActivities.deleteS3ObjectActivity.mockResolvedValue(undefined)
mockActivities.cleanupTmpDirActivity.mockResolvedValue(undefined)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mockActivities contains vi.fn mock functions which are cast to expected activity proxy types
vi.mocked(workflowUtils.getActivities).mockReturnValue(mockActivities as any)
vi.mocked(workflowUtils.executeActivity).mockImplementation(async (_queue, act, ...args) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- act is one of the mocked activities
return (act as any)(...args)
})
})
it('should run agent embedding workflow successfully for image', async () => {
@@ -124,9 +116,6 @@ describe('Agent Embedding Workflow', () => {
await agentEmbeddingMedia(task)
// Verify queue discovery
expect(mockActivities.getAgentWorkerQueueActivity).toHaveBeenCalled()
// Verify task processing status
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
@@ -307,9 +296,6 @@ describe('Agent Embedding Workflow', () => {
await agentEmbeddingMedia(task)
// Verify video transcode queue discovery
expect(mockActivities.getTranscodeWorkerQueueActivity).toHaveBeenCalled()
// Verify video chunk slicing activity called 3 times on transcode queue
expect(mockActivities.transcodeVideoChunkActivity).toHaveBeenCalledTimes(3)
expect(mockActivities.transcodeVideoChunkActivity).toHaveBeenNthCalledWith(1, {
+39 -67
View File
@@ -1,11 +1,20 @@
import { ApplicationFailure } from '@temporalio/workflow'
import type { WorkflowTask } from '@shumai/db'
import {
getActivities,
executeActivity,
TaskQueueAgent,
TaskQueueTranscode,
ApplicationFailure,
updateTaskStatusActivity,
updateTaskUsageActivity,
} from '@shumai/workflow-core'
import { createCommentActivity, updateCommentActivity } from '../activities/agent'
import {
deleteAssetEmbeddingsActivity,
deleteS3ObjectActivity,
generateImageEmbeddingActivity,
generateVideoChunkEmbeddingActivity,
getEmbeddingContextActivity,
saveAssetEmbeddingsActivity,
transcodeVideoChunkActivity,
type ExistingAssetEmbeddingInfo,
} from '../activities/ai'
export interface GeneratedEmbedding {
embedding: number[]
@@ -13,52 +22,21 @@ export interface GeneratedEmbedding {
endTime?: number
}
export interface ExistingAssetEmbeddingInfo {
id: string
startTime: number | null
endTime: number | null
has1536: boolean
has768: boolean
}
export type { ExistingAssetEmbeddingInfo }
export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
const {
updateTaskStatusActivity,
getEmbeddingContextActivity,
generateImageEmbeddingActivity,
generateVideoChunkEmbeddingActivity,
saveAssetEmbeddingsActivity,
deleteAssetEmbeddingsActivity,
updateTaskUsageActivity,
createCommentActivity,
updateCommentActivity,
getAgentWorkerQueueActivity,
getTranscodeWorkerQueueActivity,
transcodeVideoChunkActivity,
deleteS3ObjectActivity,
} = getActivities()
let placeholderCommentId: string | undefined
let agentWorkerQueue = ''
let transcodeWorkerQueue: string
try {
// 0. Discover queues
agentWorkerQueue = await executeActivity(TaskQueueAgent, getAgentWorkerQueueActivity)
transcodeWorkerQueue = await executeActivity(
TaskQueueTranscode,
getTranscodeWorkerQueueActivity,
)
// Update status to processing
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'processing',
})
// 0. Create Placeholder Comment
const payload = task.payload
const placeholder = await executeActivity(agentWorkerQueue, createCommentActivity, {
const placeholder = await createCommentActivity({
assetId: task.assetId,
message: '__EMBEDDING__',
sessionId: payload?.agent?.sessionId || task.id,
@@ -71,7 +49,7 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
}
// 1. Fetch Agent Context
const context = (await executeActivity(agentWorkerQueue, getEmbeddingContextActivity, {
const context = (await getEmbeddingContextActivity({
teamId: task.teamId,
assetId: task.assetId,
})) as unknown as {
@@ -98,7 +76,7 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
.map((e) => e.id)
if (incompatibleIds.length > 0) {
await executeActivity(agentWorkerQueue, deleteAssetEmbeddingsActivity, {
await deleteAssetEmbeddingsActivity({
assetId: task.assetId,
ids: incompatibleIds,
})
@@ -160,12 +138,12 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
(e) => e.startTime === null && e.endTime === null,
)
if (!alreadyEmbedded) {
const result = await executeActivity(agentWorkerQueue, generateImageEmbeddingActivity, {
const result = await generateImageEmbeddingActivity({
teamId: task.teamId,
assetKey: embeddingKey,
mediaType: embeddingMediaType,
mediaType: embeddingMediaType || 'image/jpeg',
})
await executeActivity(agentWorkerQueue, saveAssetEmbeddingsActivity, {
await saveAssetEmbeddingsActivity({
assetId: task.assetId,
embeddings: [{ embedding: result.embedding }],
})
@@ -206,7 +184,7 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
}
// Step A: Slice chunk on transcode worker and upload to S3
const chunkRes = await executeActivity(transcodeWorkerQueue, transcodeVideoChunkActivity, {
const chunkRes = await transcodeVideoChunkActivity({
assetId: asset.id,
assetKey: embeddingKey,
startTime: start,
@@ -217,17 +195,13 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
try {
// Step B: Generate embedding on agent worker
const chunkResult = await executeActivity(
agentWorkerQueue,
generateVideoChunkEmbeddingActivity,
{
teamId: task.teamId,
chunkKey,
},
)
const chunkResult = await generateVideoChunkEmbeddingActivity({
teamId: task.teamId,
chunkKey,
})
// Step C: Save chunk embedding to database immediately
await executeActivity(agentWorkerQueue, saveAssetEmbeddingsActivity, {
await saveAssetEmbeddingsActivity({
assetId: task.assetId,
embeddings: [
{
@@ -244,7 +218,7 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
}
} finally {
// Step D: Delete temporary chunk from S3
await executeActivity(transcodeWorkerQueue, deleteS3ObjectActivity, {
await deleteS3ObjectActivity({
key: chunkKey,
})
}
@@ -257,7 +231,7 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
}
// Update Usage
await executeActivity(agentWorkerQueue, updateTaskUsageActivity, {
await updateTaskUsageActivity({
taskId: task.id,
inputTokens: usage.inputTokens,
outputTokens: usage.outputTokens,
@@ -266,14 +240,14 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
// 7. Update Placeholder Comment
if (placeholderCommentId) {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: 'Embedding completed successfully.',
})
}
// Update status to completed
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
await updateTaskStatusActivity({
taskId: task.id,
status: 'completed',
})
@@ -281,9 +255,9 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
console.error(`AgentEmbeddingMedia failed for task ${task.id}:`, err)
// Update placeholder comment with error message
if (placeholderCommentId && agentWorkerQueue) {
if (placeholderCommentId) {
try {
await executeActivity(agentWorkerQueue, updateCommentActivity, {
await updateCommentActivity({
commentId: placeholderCommentId,
message: `Embedding failed: ${err instanceof Error ? err.message : String(err)}`,
})
@@ -293,13 +267,11 @@ export async function agentEmbeddingMedia(task: WorkflowTask): Promise<void> {
}
// Update status to failed
if (agentWorkerQueue) {
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
}
await updateTaskStatusActivity({
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
throw err
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/api",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+131
View File
@@ -68,6 +68,14 @@ vi.mock('@shumai/core/src/gotenberg/gotenberg', () => ({
},
}))
vi.mock('@shumai/core/src/workflowTask/workflowTask', () => ({
workflowTaskService: {
listWorkflowTasks: vi.fn(),
getBatchWorkflowTaskStatus: vi.fn(),
stopWorkflowTask: vi.fn(),
},
}))
describe('team api', () => {
const app = new Hono().use('*', authMiddleware).route('/', teamRoute)
let mockCreateTeam: any // eslint-disable-line @typescript-eslint/no-explicit-any
@@ -655,6 +663,129 @@ describe('team api', () => {
)
})
it('GET /teams/:teamId/workflow-tasks returns list of workflow tasks for team owner', async () => {
const { workflowTaskService } = await import('@shumai/core/src/workflowTask/workflowTask')
vi.mocked(workflowTaskService.listWorkflowTasks).mockResolvedValue({
nodes: [
{
id: 'task1',
uid: 'uid1',
teamId: 't1',
projectId: 'p1',
assetId: 'a1',
type: 'transcode_video' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
status: 'processing' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
payload: {},
output: null,
heartbeat: null,
inputTokens: 0,
outputTokens: 0,
model: null,
sessionId: null,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
],
pageInfo: {
hasNextPage: false,
endCursor: null,
},
total: 1,
})
const res = await app.request('/teams/t1/workflow-tasks')
expect(res.status).toBe(200)
const data = await res.json()
expect(data.nodes).toHaveLength(1)
expect(data.nodes[0].id).toBe('task1')
expect(authzService.hasPermission).toHaveBeenCalledWith(
expect.objectContaining({
permission: Permission.Admin,
type: ResourceType.Team,
id: 't1',
}),
)
})
it('POST /teams/:teamId/workflow-tasks/batch-status returns batch task statuses', async () => {
const { workflowTaskService } = await import('@shumai/core/src/workflowTask/workflowTask')
vi.mocked(workflowTaskService.getBatchWorkflowTaskStatus).mockResolvedValue({
tasks: [
{
id: 'task1',
uid: 'uid1',
teamId: 't1',
projectId: 'p1',
assetId: 'a1',
type: 'transcode_video' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
status: 'completed' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
payload: {},
output: null,
heartbeat: null,
inputTokens: 0,
outputTokens: 0,
model: null,
sessionId: null,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
],
})
const res = await app.request('/teams/t1/workflow-tasks/batch-status', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ taskIds: ['task1'] }),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.tasks).toHaveLength(1)
expect(data.tasks[0].status).toBe('completed')
})
it('POST /teams/:teamId/workflow-tasks/:taskId/stop stops task for team owner', async () => {
const { workflowTaskService } = await import('@shumai/core/src/workflowTask/workflowTask')
vi.mocked(workflowTaskService.stopWorkflowTask).mockResolvedValue({
success: true,
task: {
id: 'task1',
uid: 'uid1',
teamId: 't1',
projectId: 'p1',
assetId: 'a1',
type: 'transcode_video' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
status: 'failed' as any, // eslint-disable-line @typescript-eslint/no-explicit-any
payload: {},
output: { error: 'Task was stopped by user' },
heartbeat: null,
inputTokens: 0,
outputTokens: 0,
model: null,
sessionId: null,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
})
const res = await app.request('/teams/t1/workflow-tasks/task1/stop', {
method: 'POST',
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.success).toBe(true)
expect(data.task.status).toBe('failed')
expect(authzService.hasPermission).toHaveBeenCalledWith(
expect.objectContaining({
permission: Permission.Admin,
type: ResourceType.Team,
id: 't1',
}),
)
})
it('GET /teams/:teamId/turntable-settings returns turntable settings', async () => {
const { turntableService } = await import('@shumai/core/src/turntable/turntable')
vi.mocked(turntableService.getSettings).mockResolvedValue({
+71
View File
@@ -16,6 +16,8 @@ import {
updateTeamMemberRoleRequestSchema,
createApiTokenRequestSchema,
listAuditLogsQuerySchema,
listWorkflowTasksQuerySchema,
batchWorkflowTaskStatusSchema,
} from '@shumai/dtos'
import {
updateUserMetadataRequestSchema,
@@ -25,6 +27,7 @@ import {
import { apiTokenService } from '@shumai/core/src/user/api-token'
import { aiUsageService } from '@shumai/core/src/ai-usage/ai-usage'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import { workflowTaskService } from '@shumai/core/src/workflowTask/workflowTask'
import { turntableService } from '@shumai/core/src/turntable/turntable'
import { gotenbergService } from '@shumai/core/src/gotenberg/gotenberg'
import { NotificationType } from '@shumai/db'
@@ -517,5 +520,73 @@ const route = new Hono<{ Variables: { user: User } }>()
return c.json(logs)
})
.get(
'/teams/:teamId/workflow-tasks',
zValidator('query', listWorkflowTasksQuerySchema),
async (c) => {
const user = c.get('user')
const teamId = c.req.param('teamId')
const query = c.req.valid('query')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.Team,
id: teamId,
})
const tasks = await workflowTaskService.listWorkflowTasks({
teamId,
status: query.status,
types: query.types,
first: query.first,
after: query.after,
})
return c.json(tasks)
},
)
.post(
'/teams/:teamId/workflow-tasks/batch-status',
zValidator('json', batchWorkflowTaskStatusSchema),
async (c) => {
const user = c.get('user')
const teamId = c.req.param('teamId')
const req = c.req.valid('json')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.Team,
id: teamId,
})
const result = await workflowTaskService.getBatchWorkflowTaskStatus({
teamId,
taskIds: req.taskIds,
})
return c.json(result)
},
)
.post('/teams/:teamId/workflow-tasks/:taskId/stop', async (c) => {
const user = c.get('user')
const teamId = c.req.param('teamId')
const taskId = c.req.param('taskId')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.Team,
id: teamId,
})
const result = await workflowTaskService.stopWorkflowTask({
teamId,
taskId,
})
return c.json(result)
})
export default route
+1 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/core",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -34,8 +34,6 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@sinclair/typebox": "0.34.52",
"@temporalio/activity": "1.24.0",
"@temporalio/workflow": "1.24.0",
"adm-zip": "0.6.0",
"ai": "7.0.84",
"better-auth": "1.6.11",
+6 -3
View File
@@ -3240,7 +3240,7 @@ describe('AssetService — natural sort by name', () => {
await expect(assetService.emptyTrash(project.id)).resolves.not.toThrow()
})
it('should purge trashed assets whose status was processed, cancel their in-flight tasks, and delete WorkflowTasks', async () => {
it('should purge trashed assets whose status was processed, cancel their in-flight tasks, and retain WorkflowTasks as failed', async () => {
const cancelSpy = vi.spyOn(workflowService, 'cancel').mockResolvedValue(undefined)
const team = await prisma.team.create({ data: { name: 'Purge Task Team' } })
@@ -3292,9 +3292,12 @@ describe('AssetService — natural sort by name', () => {
// Task should be cancelled
expect(cancelSpy).toHaveBeenCalledWith(task.id)
// WorkflowTask should be deleted from DB
// WorkflowTask should be retained in DB and marked as failed
const taskInDb = await prisma.workflowTask.findUnique({ where: { id: task.id } })
expect(taskInDb).toBeNull()
expect(taskInDb).not.toBeNull()
expect(taskInDb?.status).toBe(WorkflowTaskStatus.failed)
const output = taskInDb?.output as { error?: string }
expect(output?.error).toBe('Asset was purged')
cancelSpy.mockRestore()
})
+23 -19
View File
@@ -1419,6 +1419,17 @@ export class AssetService {
logger.warn({ taskId: task.id, err }, 'Failed to cancel workflow task during purge')
}
}
await this.prismaClient.workflowTask.updateMany({
where: {
assetId: { in: affectedIds },
status: { in: [WorkflowTaskStatus.pending, WorkflowTaskStatus.processing] },
},
data: {
status: WorkflowTaskStatus.failed,
output: { error: 'Asset was purged' },
},
})
}
}
@@ -1518,8 +1529,15 @@ export class AssetService {
}
}
await this.prismaClient.workflowTask.deleteMany({
where: { assetId: { in: pendingAssetIds } },
await this.prismaClient.workflowTask.updateMany({
where: {
assetId: { in: pendingAssetIds },
status: { in: [WorkflowTaskStatus.pending, WorkflowTaskStatus.processing] },
},
data: {
status: WorkflowTaskStatus.failed,
output: { error: 'Asset was purged' },
},
})
}
@@ -2408,12 +2426,7 @@ export class AssetService {
}
const proxyType = (media?.proxyType || null) as
| 'image'
| 'video'
| 'audio'
| 'pdf'
| '3d'
| null
'image' | 'video' | 'audio' | 'pdf' | '3d' | null
result.push({
id: a.id,
@@ -2780,12 +2793,7 @@ export class AssetService {
if (!asset.media) return null
const proxyType = (asset.media?.proxyType || null) as
| 'image'
| 'video'
| 'audio'
| 'pdf'
| '3d'
| null
'image' | 'video' | 'audio' | 'pdf' | '3d' | null
let rawThumbKey: string | undefined
if (proxyType === 'image' && asset.media.thumbnail?.key) {
@@ -2864,11 +2872,7 @@ export class AssetService {
'GET',
)
const attachmentProxyType = (a.asset.media?.proxyType || null) as
| 'image'
| 'video'
| 'audio'
| 'pdf'
| null
'image' | 'video' | 'audio' | 'pdf' | null
attachments.push({
id: a.id,
assetId: a.asset.id,
+1 -3
View File
@@ -90,9 +90,7 @@ export class GotenbergService {
})
if (versionRes.ok) {
const vData = (await versionRes.json().catch(() => null)) as
| { version?: string }
| string
| null
{ version?: string } | string | null
if (vData && typeof vData === 'object' && 'version' in vData) {
version = String(vData.version)
} else if (typeof vData === 'string') {
+1 -2
View File
@@ -68,8 +68,7 @@ export async function resolveIntegrationConfig<TKey extends keyof PrismaJson.Set
})
const teamSettings = team?.settings as PrismaJson.Settings | null
const serviceSettings = teamSettings?.[def.settingsKey] as
| { url?: string; username?: string; password?: string }
| undefined
{ url?: string; username?: string; password?: string } | undefined
if (serviceSettings?.url?.trim()) {
return {
+1 -3
View File
@@ -41,9 +41,7 @@ export interface ProxyServerInfo {
}
export type ConnectionOutcome =
| { status: 'connected' }
| { status: 'needs-auth' }
| { status: 'error'; message: string }
{ status: 'connected' } | { status: 'needs-auth' } | { status: 'error'; message: string }
export interface CallToolOutcome {
ok: boolean
+1 -2
View File
@@ -351,8 +351,7 @@ export class McpService {
instructions?: string | null
} = {}
const impl = connection.serverVersion as
| { name?: string; title?: string; description?: string }
| undefined
{ name?: string; title?: string; description?: string } | undefined
if (impl && typeof impl === 'object') {
const reportedName = impl.title ?? impl.name
if (typeof reportedName === 'string' && reportedName.trim() !== '') {
+33 -21
View File
@@ -43,24 +43,30 @@ export function getNextOptionColor(existingOptions: PrismaJson.SelectOption[] =
export class MetadataService {
constructor(private client: typeof prisma = prisma) {}
async syncSystemFields(): Promise<void> {
await this.client.$transaction(async (tx) => {
for (const field of systemFields) {
await tx.metadataField.upsert({
where: { key: field.key },
update: {
scope: field.scope,
readOnly: field.readOnly,
config: field.config,
},
create: {
key: field.key,
scope: field.scope,
readOnly: field.readOnly ?? false,
config: field.config,
},
})
}
})
await this.client.$transaction(
async (tx) => {
for (const field of systemFields) {
await tx.metadataField.upsert({
where: { key: field.key },
update: {
scope: field.scope,
readOnly: field.readOnly,
config: field.config,
},
create: {
key: field.key,
scope: field.scope,
readOnly: field.readOnly ?? false,
config: field.config,
},
})
}
},
{
maxWait: 10000,
timeout: 30000,
},
)
}
async createTeamField(
@@ -312,9 +318,15 @@ export class MetadataService {
reqs: UpdateAssetMetadataRequest[],
allowReadOnly = false,
): Promise<void> {
await this.client.$transaction(async (tx) => {
await this.updateAssetMetadataInternal(tx, assetId, reqs, allowReadOnly)
})
await this.client.$transaction(
async (tx) => {
await this.updateAssetMetadataInternal(tx, assetId, reqs, allowReadOnly)
},
{
maxWait: 10000,
timeout: 30000,
},
)
}
/**
@@ -0,0 +1,54 @@
{
"anthropic": {
"models": {
"claude-3-5-sonnet-20241022": {
"id": "claude-3-5-sonnet-20241022",
"name": "Claude 3.5 Sonnet",
"tool_call": true,
"reasoning": false,
"limit": {
"context": 200000,
"output": 8192
},
"cost": {
"input": 3,
"output": 15,
"cache_read": 0.3,
"cache_write": 3.75
}
}
}
},
"openai": {
"models": {
"gpt-4o": {
"id": "gpt-4o",
"name": "GPT-4o",
"tool_call": true,
"reasoning": false,
"limit": {
"context": 128000,
"output": 4096
},
"cost": {
"input": 2.5,
"output": 10
}
},
"gpt-4o-extra-new": {
"id": "gpt-4o-extra-new",
"name": "GPT-4o Extra New",
"tool_call": true,
"reasoning": false,
"limit": {
"context": 128000,
"output": 4096
},
"cost": {
"input": 2.5,
"output": 10
}
}
}
}
}
@@ -0,0 +1,7 @@
{
"data": [
{
"id": "meta/llama-3.1-70b-instruct"
}
]
}
@@ -0,0 +1,17 @@
{
"data": [
{
"id": "meta-llama/llama-3.1-8b-instruct",
"name": "Meta: Llama 3.1 8B Instruct",
"context_length": 131072,
"pricing": {
"prompt": "0.00000005",
"completion": "0.00000008"
},
"top_provider": {
"max_completion_tokens": 8192
},
"supported_parameters": ["tools"]
}
]
}
@@ -1141,8 +1141,7 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
baseUrl: getBedrockBaseUrl(id),
reasoning: m.reasoning === true,
input: (m.modalities?.input?.includes('image') ? ['text', 'image'] : ['text']) as (
| 'text'
| 'image'
'text' | 'image'
)[],
cost: {
input: m.cost?.input || 0,
+44 -6
View File
@@ -8,6 +8,42 @@ import * as generateModelsModule from './generator/generate-models'
import { ProviderConfigSerializable, providerModelSchema } from '@shumai/dtos'
import { getBuiltinProvidersMap } from '@shumai/core/src/provider/builtin'
import { z } from 'zod'
import modelsDevFixture from './fixtures/models-dev.json'
import openRouterFixture from './fixtures/openrouter.json'
import nvidiaFixture from './fixtures/nvidia.json'
const originalFetch = globalThis.fetch
function setupCatalogFetchMock() {
return vi.spyOn(globalThis, 'fetch').mockImplementation(async (input: RequestInfo | URL) => {
const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url
if (url.includes('models.dev')) {
return new Response(JSON.stringify(modelsDevFixture), {
status: 200,
headers: { 'Content-Type': 'application/json' },
})
}
if (url.includes('openrouter.ai')) {
return new Response(JSON.stringify(openRouterFixture), {
status: 200,
headers: { 'Content-Type': 'application/json' },
})
}
if (url.includes('nvidia')) {
return new Response(JSON.stringify(nvidiaFixture), {
status: 200,
headers: { 'Content-Type': 'application/json' },
})
}
if (url.includes('ai-gateway')) {
return new Response(JSON.stringify({ data: [] }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
})
}
return originalFetch(input)
})
}
type ProviderModel = z.infer<typeof providerModelSchema>
@@ -349,10 +385,9 @@ describe('ProviderService', () => {
expect((modelsAfterUpdate[1].config as { api: string }).api).toBe('anthropic-messages')
})
it(
'should check for updates and correctly segregate new vs existing providers',
{ timeout: 30000 },
async () => {
it('should check for updates and correctly segregate new vs existing providers', async () => {
const fetchSpy = setupCatalogFetchMock()
try {
// 1. Check updates with empty team: all discovered providers are new
const emptyTeam = await prisma.team.create({
data: { name: 'Empty Provider Team' },
@@ -368,6 +403,7 @@ describe('ProviderService', () => {
// 3. Check updates after init: existing providers should have isNewProvider: false
const afterInitCheck = await providerService.checkUpdates(teamWithBuiltins.id)
const existing = afterInitCheck.providers.find((p) => !p.isNewProvider)
expect(existing).toBeDefined()
if (existing) {
expect(existing.isNewProvider).toBe(false)
const dbModels = await providerService.listModelsByProvider(
@@ -381,8 +417,10 @@ describe('ProviderService', () => {
expect(dbModelIds.has(m.modelId)).toBe(false)
}
}
},
)
} finally {
fetchSpy.mockRestore()
}
})
it('should throw an error when upstream catalog fetch fails in checkUpdates', async () => {
const team = await teamService.ensureDefaultTeam()
+39 -1
View File
@@ -351,7 +351,35 @@ describe('S3Service implementations', () => {
const input = await s3.resolveInput('test-bucket', 'video.mp4')
expect(input).toBe('https://presigned.s3.url')
expect(presignSpy).toHaveBeenCalledWith('test-bucket', 'video.mp4', 'GET')
expect(presignSpy).toHaveBeenCalledWith(
'test-bucket',
'video.mp4',
'GET',
false,
undefined,
true,
)
})
it('should support separate publicEndpoint for presigning while resolveInput uses internal client', async () => {
const s3 = new S3StorageService(
'http://internal-s3:9000',
'key',
'secret',
'test-bucket',
'us-east-1',
'https://public.example.com',
)
getSignedUrlSpy.mockClear()
await s3.presign('test-bucket', 'asset.mp4', 'GET')
const presignClientCall = getSignedUrlSpy.mock.calls[0][0]
getSignedUrlSpy.mockClear()
await s3.resolveInput('test-bucket', 'asset.mp4')
const resolveClientCall = getSignedUrlSpy.mock.calls[0][0]
expect(presignClientCall).not.toBe(resolveClientCall)
})
it('should stream in uploadFileToKey', async () => {
@@ -642,6 +670,16 @@ describe('S3Service implementations', () => {
expect(url).toContain('http://123.456.7.8:12345')
expect(url).not.toContain('4567')
})
it('should initialize S3StorageService with AWS_PUBLIC_ENDPOINT_URL_S3 when set', async () => {
process.env.STORAGE_BACKEND = 's3'
process.env.AWS_ENDPOINT_URL_S3 = 'http://garage.shumai.svc.cluster.local:3900'
process.env.AWS_PUBLIC_ENDPOINT_URL_S3 = 'http://localhost:3900'
vi.resetModules()
const { s3Service } = await import('./s3')
expect(s3Service).toBeDefined()
})
})
describe('buildContentDisposition', () => {
+33 -11
View File
@@ -111,6 +111,7 @@ export interface S3Service {
method: string,
download?: boolean,
filename?: string,
internal?: boolean,
) => Promise<string>
presignMultipart: (
bucket: string,
@@ -123,6 +124,7 @@ export interface S3Service {
export class S3StorageService implements S3Service {
private client: S3Client
private presignClient: S3Client
private bucket: string
private presignCache = new LruTtlCache<string, string>(50000)
@@ -132,6 +134,7 @@ export class S3StorageService implements S3Service {
secretAccessKey: string,
bucket: string,
region: string = 'auto',
publicEndpoint?: string,
) {
this.client = new S3Client({
region,
@@ -144,6 +147,20 @@ export class S3StorageService implements S3Service {
requestChecksumCalculation: 'WHEN_REQUIRED',
responseChecksumValidation: 'WHEN_REQUIRED',
})
this.presignClient =
publicEndpoint && publicEndpoint !== endpoint
? new S3Client({
region,
endpoint: publicEndpoint,
credentials: {
accessKeyId,
secretAccessKey,
},
forcePathStyle: true,
requestChecksumCalculation: 'WHEN_REQUIRED',
responseChecksumValidation: 'WHEN_REQUIRED',
})
: this.client
this.bucket = bucket
}
@@ -342,12 +359,13 @@ export class S3StorageService implements S3Service {
method: string,
download?: boolean,
filename?: string,
internal?: boolean,
): Promise<string> {
const expireHours = parseInt(process.env.PRESIGNED_URL_EXPIRES_IN || '5', 10)
const expiresInSeconds = expireHours * 3600
const cacheTtlMs = expiresInSeconds * 1000
const cacheKey = `${bucket}/${key}?download=${download || false}&filename=${filename || ''}`
const cacheKey = `${bucket}/${key}?download=${download || false}&filename=${filename || ''}&internal=${internal || false}`
if (method === 'GET' && !download) {
const cached = this.presignCache.get(cacheKey)
if (cached) {
@@ -355,6 +373,7 @@ export class S3StorageService implements S3Service {
}
}
const signingClient = internal ? this.client : this.presignClient
let url: string
if (method === 'GET') {
@@ -363,19 +382,19 @@ export class S3StorageService implements S3Service {
Key: key,
ResponseContentDisposition: download ? buildContentDisposition(filename) : undefined,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(signingClient, command, { expiresIn: expiresInSeconds })
} else if (method === 'PUT') {
const command = new PutObjectCommand({
Bucket: bucket,
Key: key,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(signingClient, command, { expiresIn: expiresInSeconds })
} else if (method === 'DELETE') {
const command = new DeleteObjectCommand({
Bucket: bucket,
Key: key,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(signingClient, command, { expiresIn: expiresInSeconds })
} else {
throw new Error(`Unsupported method for presign: ${method}`)
}
@@ -405,13 +424,13 @@ export class S3StorageService implements S3Service {
UploadId: request.uploadId,
PartNumber: request.partNumber,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
} else {
const command = new PutObjectCommand({
Bucket: bucket,
Key: key,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
}
} else if (request.method === 'POST') {
if (request.uploadId) {
@@ -420,13 +439,13 @@ export class S3StorageService implements S3Service {
Key: key,
UploadId: request.uploadId,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
} else {
const command = new CreateMultipartUploadCommand({
Bucket: bucket,
Key: key,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
}
} else if (request.method === 'GET') {
if (!request.uploadId) {
@@ -437,7 +456,7 @@ export class S3StorageService implements S3Service {
Key: key,
UploadId: request.uploadId,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
} else if (request.method === 'DELETE') {
if (!request.uploadId) {
throw new Error('Abort multipart upload requires uploadId')
@@ -447,7 +466,7 @@ export class S3StorageService implements S3Service {
Key: key,
UploadId: request.uploadId,
})
url = await getSignedUrl(this.client, command, { expiresIn: expiresInSeconds })
url = await getSignedUrl(this.presignClient, command, { expiresIn: expiresInSeconds })
} else {
throw new Error(`Unsupported method for presignMultipart: ${request.method}`)
}
@@ -466,7 +485,7 @@ export class S3StorageService implements S3Service {
}
async resolveInput(bucket: string, key: string): Promise<string> {
return this.presign(bucket, key, 'GET')
return this.presign(bucket, key, 'GET', false, undefined, true)
}
}
@@ -743,6 +762,8 @@ export function getStorageBackend(): 's3' | 'local' {
const storageBackend = getStorageBackend()
const port = process.env.SHUMAI_SERVER_PORT || '3000'
const s3Endpoint = process.env.AWS_ENDPOINT_URL_S3 || `http://localhost:${port}`
const s3PublicEndpoint =
process.env.AWS_PUBLIC_ENDPOINT_URL_S3 || process.env.S3_PUBLIC_ENDPOINT || undefined
export const s3Service: S3Service =
storageBackend === 's3'
@@ -752,5 +773,6 @@ export const s3Service: S3Service =
process.env.S3_SECRET_ACCESS_KEY || 'minioadmin',
process.env.S3_BUCKET || 'shumai',
process.env.S3_REGION || 'auto',
s3PublicEndpoint,
)
: new LocalStorageService(s3Endpoint)
@@ -0,0 +1,550 @@
import { describe, it, expect } from 'vitest'
import {
prisma,
WorkflowTaskStatus,
WorkflowTaskType,
AssetStatus,
registerWorkflowCancel,
} from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { HTTPException } from 'hono/http-exception'
import { workflowTaskService } from './workflowTask'
describe('WorkflowTaskService', () => {
setupTestDbHooks()
it('lists workflow tasks for a team with pagination and filters', async () => {
const team1 = await prisma.team.create({
data: { name: 'Workflow Team 1' },
})
const team2 = await prisma.team.create({
data: { name: 'Workflow Team 2' },
})
const project1 = await prisma.project.create({
data: { name: 'Project 1', teamId: team1.id },
})
const asset1 = await prisma.asset.create({
data: {
name: 'video1.mp4',
type: 'file',
sizeByte: 2048,
status: 'uploaded',
projectId: project1.id,
},
})
// Task 1: Team 1 direct
const task1 = await prisma.workflowTask.create({
data: {
assetId: asset1.id,
teamId: team1.id,
projectId: project1.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.completed,
},
})
// Task 2: Team 1 via project (teamId is null)
const task2 = await prisma.workflowTask.create({
data: {
assetId: asset1.id,
projectId: project1.id,
type: WorkflowTaskType.chat,
status: WorkflowTaskStatus.pending,
},
})
// Task 3: Team 2
await prisma.workflowTask.create({
data: {
assetId: 'other-asset',
teamId: team2.id,
type: WorkflowTaskType.transcode_image,
status: WorkflowTaskStatus.processing,
},
})
// 1. List all for Team 1
const listAll = await workflowTaskService.listWorkflowTasks({
teamId: team1.id,
})
expect(listAll.total).toBe(2)
expect(listAll.nodes).toHaveLength(2)
expect(listAll.nodes.map((n) => n.id)).toEqual([task2.id, task1.id]) // ordered by id desc
expect(listAll.nodes[0].fileName).toBe('video1.mp4')
expect(listAll.nodes[0].fileSize).toBe(2048)
expect(listAll.nodes[0].projectName).toBe('Project 1')
expect(listAll.nodes[0].projectId).toBe(project1.id)
expect(listAll.nodes[1].fileName).toBe('video1.mp4')
expect(listAll.nodes[1].fileSize).toBe(2048)
expect(listAll.nodes[1].projectName).toBe('Project 1')
// 2. Filter by status
const listPending = await workflowTaskService.listWorkflowTasks({
teamId: team1.id,
status: [WorkflowTaskStatus.pending],
})
expect(listPending.total).toBe(1)
expect(listPending.nodes[0].id).toBe(task2.id)
// 3. Filter by type
const listChat = await workflowTaskService.listWorkflowTasks({
teamId: team1.id,
types: [WorkflowTaskType.chat],
})
expect(listChat.total).toBe(1)
expect(listChat.nodes[0].id).toBe(task2.id)
// 4. Pagination
const page1 = await workflowTaskService.listWorkflowTasks({
teamId: team1.id,
first: 1,
})
expect(page1.nodes).toHaveLength(1)
expect(page1.nodes[0].id).toBe(task2.id)
expect(page1.pageInfo.hasNextPage).toBe(true)
const page2 = await workflowTaskService.listWorkflowTasks({
teamId: team1.id,
first: 1,
after: page1.pageInfo.endCursor!,
})
expect(page2.nodes).toHaveLength(1)
expect(page2.nodes[0].id).toBe(task1.id)
})
it('gets batch status for specific task IDs with team security', async () => {
const team1 = await prisma.team.create({
data: { name: 'Batch Team 1' },
})
const team2 = await prisma.team.create({
data: { name: 'Batch Team 2' },
})
const task1 = await prisma.workflowTask.create({
data: {
assetId: 'a1',
teamId: team1.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.processing,
},
})
const task2 = await prisma.workflowTask.create({
data: {
assetId: 'a2',
teamId: team2.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.processing,
},
})
// Query empty
const empty = await workflowTaskService.getBatchWorkflowTaskStatus({
teamId: team1.id,
taskIds: [],
})
expect(empty.tasks).toEqual([])
// Query both task1 and task2 under team1
const res = await workflowTaskService.getBatchWorkflowTaskStatus({
teamId: team1.id,
taskIds: [task1.id, task2.id],
})
// task2 should be excluded because it belongs to team2
expect(res.tasks).toHaveLength(1)
expect(res.tasks[0].id).toBe(task1.id)
expect(res.tasks[0].status).toBe(WorkflowTaskStatus.processing)
})
it('stops a pending or processing task successfully', async () => {
const team = await prisma.team.create({
data: { name: 'Stop Team' },
})
const pendingTask = await prisma.workflowTask.create({
data: {
assetId: 'asset-pending',
teamId: team.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
payload: { projectId: 'p1' },
},
})
const processingTask = await prisma.workflowTask.create({
data: {
assetId: 'asset-processing',
teamId: team.id,
type: WorkflowTaskType.ai_metadata_autofill,
status: WorkflowTaskStatus.processing,
},
})
// Stop pending task
const stopPendingRes = await workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: pendingTask.id,
})
expect(stopPendingRes.success).toBe(true)
expect(stopPendingRes.task.status).toBe(WorkflowTaskStatus.failed)
const pendingOutput = stopPendingRes.task.output as { error: string }
expect(pendingOutput.error).toBe('Task was stopped by user')
// Stop processing task
const stopProcessingRes = await workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: processingTask.id,
})
expect(stopProcessingRes.success).toBe(true)
expect(stopProcessingRes.task.status).toBe(WorkflowTaskStatus.failed)
})
it('updates associated asset status to failed atomically when task is stopped', async () => {
const team = await prisma.team.create({
data: { name: 'Asset Stop Team' },
})
const project = await prisma.project.create({
data: { name: 'Asset Stop Project', teamId: team.id },
})
// Asset 1: currently uploaded (e.g. pending transcode)
const asset1 = await prisma.asset.create({
data: {
name: 'pending-video.mp4',
type: 'file',
status: AssetStatus.uploaded,
projectId: project.id,
media: {
duration: 120,
filesize: 1000,
frames: 3000,
imageTranscodes: [],
videoTranscodes: [],
finishedAt: '',
metadata: null,
original: null,
},
},
})
const task1 = await prisma.workflowTask.create({
data: {
assetId: asset1.id,
teamId: team.id,
projectId: project.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
},
})
// Asset 2: already processed (media finished)
const asset2 = await prisma.asset.create({
data: {
name: 'completed-video.mp4',
type: 'file',
status: AssetStatus.processed,
projectId: project.id,
media: {
duration: 60,
filesize: 500,
frames: 1500,
imageTranscodes: [],
videoTranscodes: [],
finishedAt: '2026-01-01',
metadata: null,
original: null,
},
},
})
const task2 = await prisma.workflowTask.create({
data: {
assetId: asset2.id,
teamId: team.id,
projectId: project.id,
type: WorkflowTaskType.ai_metadata_autofill,
status: WorkflowTaskStatus.processing,
},
})
// Stop task 1 (pending transcode)
await workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: task1.id,
})
const updatedAsset1 = await prisma.asset.findUniqueOrThrow({
where: { id: asset1.id },
})
expect(updatedAsset1.status).toBe(AssetStatus.failed)
const media1 = updatedAsset1.media as PrismaJson.MediaInfo
expect(media1.duration).toBe(120) // preserved other json properties
expect(media1.filesize).toBe(1000)
expect(media1.error).toBe('Task was stopped by user')
// Stop task 2 (asset already processed)
await workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: task2.id,
})
const updatedAsset2 = await prisma.asset.findUniqueOrThrow({
where: { id: asset2.id },
})
// Must NOT be overwritten to failed
expect(updatedAsset2.status).toBe(AssetStatus.processed)
const media2 = updatedAsset2.media as PrismaJson.MediaInfo
expect(media2.duration).toBe(60)
expect(media2.error).toBeUndefined()
})
it('throws 400 when attempting to stop completed or failed task', async () => {
const team = await prisma.team.create({
data: { name: 'Stop Error Team' },
})
const completedTask = await prisma.workflowTask.create({
data: {
assetId: 'asset-completed',
teamId: team.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.completed,
},
})
await expect(
workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: completedTask.id,
}),
).rejects.toThrow(HTTPException)
})
it('throws 404 when stopping non-existent task and 403 when task belongs to other team', async () => {
const team1 = await prisma.team.create({
data: { name: 'Team 1' },
})
const team2 = await prisma.team.create({
data: { name: 'Team 2' },
})
const task2 = await prisma.workflowTask.create({
data: {
assetId: 'asset-team2',
teamId: team2.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
},
})
// Non-existent task
await expect(
workflowTaskService.stopWorkflowTask({
teamId: team1.id,
taskId: 'non-existent-task-id',
}),
).rejects.toThrow(HTTPException)
// Task belongs to team 2
await expect(
workflowTaskService.stopWorkflowTask({
teamId: team1.id,
taskId: task2.id,
}),
).rejects.toThrow(HTTPException)
})
it('throws 502 and does not modify task or asset when workflow cancellation fails', async () => {
const team = await prisma.team.create({
data: { name: 'Cancel Fail Team' },
})
const project = await prisma.project.create({
data: { name: 'Cancel Fail Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'fail-cancel.mp4',
type: 'file',
status: AssetStatus.processing,
projectId: project.id,
},
})
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
teamId: team.id,
projectId: project.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.processing,
},
})
registerWorkflowCancel(async () => {
throw new Error('DBOS connection failure')
})
try {
await expect(
workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: task.id,
}),
).rejects.toMatchObject({ status: 502 })
// Verify task status was NOT changed to failed
const currentTask = await prisma.workflowTask.findUniqueOrThrow({
where: { id: task.id },
})
expect(currentTask.status).toBe(WorkflowTaskStatus.processing)
// Verify asset status was NOT changed to failed
const currentAsset = await prisma.asset.findUniqueOrThrow({
where: { id: asset.id },
})
expect(currentAsset.status).toBe(AssetStatus.processing)
} finally {
registerWorkflowCancel(async () => {})
}
})
it('throws 409 and does not overwrite status when task completes during cancellation', async () => {
const team = await prisma.team.create({
data: { name: 'Race Team' },
})
const project = await prisma.project.create({
data: { name: 'Race Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'race.mp4',
type: 'file',
status: AssetStatus.processing,
projectId: project.id,
},
})
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
teamId: team.id,
projectId: project.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.processing,
},
})
// Simulate task completing in the background while cancelWorkflowTask is awaiting
registerWorkflowCancel(async () => {
await prisma.workflowTask.update({
where: { id: task.id },
data: { status: WorkflowTaskStatus.completed },
})
await prisma.asset.update({
where: { id: asset.id },
data: { status: AssetStatus.processed },
})
})
try {
await expect(
workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: task.id,
}),
).rejects.toMatchObject({ status: 409 })
// Verify task remains completed, not overwritten to failed
const currentTask = await prisma.workflowTask.findUniqueOrThrow({
where: { id: task.id },
})
expect(currentTask.status).toBe(WorkflowTaskStatus.completed)
// Verify asset remains processed, not overwritten to failed
const currentAsset = await prisma.asset.findUniqueOrThrow({
where: { id: asset.id },
})
expect(currentAsset.status).toBe(AssetStatus.processed)
} finally {
registerWorkflowCancel(async () => {})
}
})
it('enriches tasks with fileName, fileSize, and projectName, and handles purged assets gracefully', async () => {
const team = await prisma.team.create({
data: { name: 'Enrich Team' },
})
const project = await prisma.project.create({
data: { name: 'Enrich Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'sample_audio.mp3',
type: 'file',
sizeByte: 5242880,
status: 'uploaded',
projectId: project.id,
},
})
const taskWithAsset = await prisma.workflowTask.create({
data: {
assetId: asset.id,
teamId: team.id,
projectId: project.id,
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
},
})
const taskWithPurgedAsset = await prisma.workflowTask.create({
data: {
assetId: 'purged-asset-id',
teamId: team.id,
type: WorkflowTaskType.transcode_image,
status: WorkflowTaskStatus.completed,
},
})
const listRes = await workflowTaskService.listWorkflowTasks({ teamId: team.id })
expect(listRes.total).toBe(2)
const foundActive = listRes.nodes.find((n) => n.id === taskWithAsset.id)
expect(foundActive).toBeDefined()
expect(foundActive?.fileName).toBe('sample_audio.mp3')
expect(foundActive?.fileSize).toBe(5242880)
expect(foundActive?.projectName).toBe('Enrich Project')
expect(foundActive?.projectId).toBe(project.id)
const foundPurged = listRes.nodes.find((n) => n.id === taskWithPurgedAsset.id)
expect(foundPurged).toBeDefined()
expect(foundPurged?.fileName).toBeNull()
expect(foundPurged?.fileSize).toBeNull()
expect(foundPurged?.projectName).toBeNull()
// Test batch status enrichment
const batchRes = await workflowTaskService.getBatchWorkflowTaskStatus({
teamId: team.id,
taskIds: [taskWithAsset.id, taskWithPurgedAsset.id],
})
expect(batchRes.tasks).toHaveLength(2)
const batchActive = batchRes.tasks.find((t) => t.id === taskWithAsset.id)
expect(batchActive?.fileName).toBe('sample_audio.mp3')
expect(batchActive?.fileSize).toBe(5242880)
expect(batchActive?.projectName).toBe('Enrich Project')
// Test stopWorkflowTask enrichment
const stopRes = await workflowTaskService.stopWorkflowTask({
teamId: team.id,
taskId: taskWithAsset.id,
})
expect(stopRes.success).toBe(true)
expect(stopRes.task.fileName).toBe('sample_audio.mp3')
expect(stopRes.task.fileSize).toBe(5242880)
expect(stopRes.task.projectName).toBe('Enrich Project')
})
})
@@ -0,0 +1,330 @@
import {
prisma,
cancelWorkflowTask,
WorkflowTaskStatus,
type WorkflowTaskType,
type Prisma,
} from '@shumai/db'
import type { WorkflowTaskItem, ListWorkflowTasksResponse } from '@shumai/dtos'
import { paginateQuery } from '@shumai/core/src/pagination'
import { logger } from '@shumai/core/src/logger'
import { HTTPException } from 'hono/http-exception'
export interface ListWorkflowTasksParams {
teamId: string
status?: WorkflowTaskStatus[]
types?: WorkflowTaskType[]
first?: number
after?: string
}
export interface StopWorkflowTaskParams {
teamId: string
taskId: string
}
export interface GetBatchWorkflowTaskStatusParams {
teamId: string
taskIds: string[]
}
interface EnrichedMetadata {
fileName?: string | null
fileSize?: number | null
projectId?: string | null
projectName?: string | null
}
function mapToWorkflowTaskItem(
task: {
id: string
uid: string
teamId: string | null
projectId: string | null
assetId: string
type: WorkflowTaskType | null
status: WorkflowTaskStatus | null
payload: unknown
output: unknown
heartbeat: Date | null
inputTokens: number
outputTokens: number
model: string | null
sessionId: string | null
createdAt: Date
updatedAt: Date
},
extra?: EnrichedMetadata,
): WorkflowTaskItem {
return {
id: task.id,
uid: task.uid,
teamId: task.teamId,
projectId: extra?.projectId !== undefined ? extra.projectId : task.projectId,
projectName: extra?.projectName ?? null,
assetId: task.assetId,
fileName: extra?.fileName ?? null,
fileSize: extra?.fileSize ?? null,
type: task.type,
status: task.status,
payload: task.payload,
output: task.output,
heartbeat: task.heartbeat ? task.heartbeat.toISOString() : null,
inputTokens: task.inputTokens,
outputTokens: task.outputTokens,
model: task.model,
sessionId: task.sessionId,
createdAt: task.createdAt.toISOString(),
updatedAt: task.updatedAt.toISOString(),
}
}
export class WorkflowTaskService {
private async enrichWorkflowTasks<
T extends {
id: string
uid: string
teamId: string | null
projectId: string | null
assetId: string
type: WorkflowTaskType | null
status: WorkflowTaskStatus | null
payload: unknown
output: unknown
heartbeat: Date | null
inputTokens: number
outputTokens: number
model: string | null
sessionId: string | null
createdAt: Date
updatedAt: Date
},
>(tasks: T[]): Promise<WorkflowTaskItem[]> {
if (tasks.length === 0) {
return []
}
const assetIds = [...new Set(tasks.map((t) => t.assetId).filter(Boolean))]
const assets =
assetIds.length > 0
? await prisma.asset.findMany({
where: { id: { in: assetIds } },
select: { id: true, name: true, sizeByte: true, projectId: true },
})
: []
const assetMap = new Map(assets.map((a) => [a.id, a]))
const allProjectIds = [
...new Set(
[...tasks.map((t) => t.projectId), ...assets.map((a) => a.projectId)].filter(
(id): id is string => Boolean(id),
),
),
]
const projects =
allProjectIds.length > 0
? await prisma.project.findMany({
where: { id: { in: allProjectIds } },
select: { id: true, name: true },
})
: []
const projectMap = new Map(projects.map((p) => [p.id, p.name]))
return tasks.map((task) => {
const asset = assetMap.get(task.assetId)
const effectiveProjectId = task.projectId || asset?.projectId || null
const projectName = effectiveProjectId ? (projectMap.get(effectiveProjectId) ?? null) : null
const fileName = asset?.name ?? null
const fileSize = asset ? Number(asset.sizeByte) : null
return mapToWorkflowTaskItem(task, {
fileName,
fileSize,
projectId: effectiveProjectId,
projectName,
})
})
}
async listWorkflowTasks(params: ListWorkflowTasksParams): Promise<ListWorkflowTasksResponse> {
const teamProjects = await prisma.project.findMany({
where: { teamId: params.teamId },
select: { id: true },
})
const projectIds = teamProjects.map((p) => p.id)
const where: Prisma.WorkflowTaskWhereInput = {
OR: [
{ teamId: params.teamId },
...(projectIds.length > 0 ? [{ projectId: { in: projectIds } }] : []),
],
}
if (params.status && params.status.length > 0) {
where.status = { in: params.status }
}
if (params.types && params.types.length > 0) {
where.type = { in: params.types }
}
const result = await paginateQuery(
(skip, take) =>
prisma.workflowTask.findMany({
where,
orderBy: { id: 'desc' },
skip,
take,
}),
() => prisma.workflowTask.count({ where }),
{ first: params.first, after: params.after },
)
const nodes = await this.enrichWorkflowTasks(result.data)
return {
nodes,
pageInfo: {
hasNextPage: !!result.pageInfo.cursor,
endCursor: result.pageInfo.cursor || null,
},
total: result.pageInfo.total ?? 0,
}
}
async getBatchWorkflowTaskStatus(
params: GetBatchWorkflowTaskStatusParams,
): Promise<{ tasks: WorkflowTaskItem[] }> {
if (!params.taskIds || params.taskIds.length === 0) {
return { tasks: [] }
}
const teamProjects = await prisma.project.findMany({
where: { teamId: params.teamId },
select: { id: true },
})
const projectIds = teamProjects.map((p) => p.id)
const tasks = await prisma.workflowTask.findMany({
where: {
id: { in: params.taskIds },
OR: [
{ teamId: params.teamId },
...(projectIds.length > 0 ? [{ projectId: { in: projectIds } }] : []),
],
},
orderBy: { id: 'desc' },
})
const enriched = await this.enrichWorkflowTasks(tasks)
return { tasks: enriched }
}
async stopWorkflowTask(
params: StopWorkflowTaskParams,
): Promise<{ success: boolean; task: WorkflowTaskItem }> {
const task = await prisma.workflowTask.findUnique({
where: { id: params.taskId },
})
if (!task) {
throw new HTTPException(404, { message: 'Workflow task not found' })
}
// Verify task belongs to this team (either directly or via project)
let belongsToTeam = task.teamId === params.teamId
if (!belongsToTeam && task.projectId) {
const project = await prisma.project.findUnique({
where: { id: task.projectId },
select: { teamId: true },
})
if (project?.teamId === params.teamId) {
belongsToTeam = true
}
}
if (!belongsToTeam) {
throw new HTTPException(403, { message: 'Permission denied: Task does not belong to team' })
}
if (
task.status !== WorkflowTaskStatus.pending &&
task.status !== WorkflowTaskStatus.processing
) {
throw new HTTPException(400, {
message: `Only pending or processing tasks can be stopped. Current status: ${task.status}`,
})
}
// Notify workflow engine (DBOS) to cancel workflow execution
try {
await cancelWorkflowTask(task.id)
} catch (err) {
logger.error({ taskId: task.id, err }, 'Failed to cancel workflow task in workflow engine')
throw new HTTPException(502, {
message: `Failed to cancel workflow task in engine: ${err instanceof Error ? err.message : String(err)}`,
})
}
const errorMsg = 'Task was stopped by user'
const prevOutput =
typeof task.output === 'object' && task.output !== null
? (task.output as Record<string, unknown>)
: {}
const updated = await prisma.$transaction(async (tx) => {
const updateResult = await tx.workflowTask.updateMany({
where: {
id: task.id,
status: { in: [WorkflowTaskStatus.pending, WorkflowTaskStatus.processing] },
},
data: {
status: WorkflowTaskStatus.failed,
output: {
...prevOutput,
error: errorMsg,
},
},
})
if (updateResult.count === 0) {
const latestTask = await tx.workflowTask.findUniqueOrThrow({
where: { id: task.id },
})
throw new HTTPException(409, {
message: `Task is no longer active. Current status: ${latestTask.status}`,
})
}
if (task.assetId) {
await tx.$executeRaw`
UPDATE "assets"
SET
"status" = 'failed'::"AssetStatus",
"media" = COALESCE("media", '{}'::jsonb) || jsonb_build_object('error', ${errorMsg}::text)
WHERE "id" = ${task.assetId}
AND "is_deleted" = false
AND "status" != 'processed'::"AssetStatus"
`
}
return await tx.workflowTask.findUniqueOrThrow({
where: { id: task.id },
})
})
logger.info({ taskId: task.id, teamId: params.teamId }, 'Workflow task stopped by user')
const [enriched] = await this.enrichWorkflowTasks([updated])
return {
success: true,
task: enriched,
}
}
}
export const workflowTaskService = new WorkflowTaskService()
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/db",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -4,7 +4,7 @@ import { PrismaClient, WorkflowTask } from './generated/prisma/client'
import { PrismaTestingHelper } from './prisma-testing-helper'
import { generateNgrams } from './utils/ngram'
// Ensure BigInt serializes to JSON number gracefully for Temporal/HTTP payloads
// Ensure BigInt serializes to JSON number gracefully for DBOS/HTTP payloads
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(BigInt.prototype as any).toJSON = function () {
return Number(this)
+7
View File
@@ -360,6 +360,13 @@ declare global {
imageAnnotation?: ImageAnnotationSpec
pdfPages?: PdfPagesSpec
watermark?: WatermarkTaskPayload
recoveryAttempts?: number
}
export interface WorkflowTaskOutput {
error?: string
recoveryAttempts?: number
[key: string]: unknown
}
// ----------------------------------------------------------------------
+9
View File
@@ -23,6 +23,15 @@ export async function globalTestSetup() {
},
})
console.log('Prisma Migrations completed.')
console.log('Applying DBOS Migrations...')
await execAsync('bun packages/workflow-core/src/migrate.ts', {
env: {
...process.env,
DATABASE_URL: databaseUrl,
},
})
console.log('DBOS Migrations completed.')
}
export async function setup() {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/dtos",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1
View File
@@ -28,3 +28,4 @@ export * from './kanban'
export * from './recents'
export * from './context'
export * from './media-generation'
export * from './workflowTask'
+79
View File
@@ -0,0 +1,79 @@
import { z } from 'zod'
import { WorkflowTaskType, WorkflowTaskStatus } from '@shumai/db/enums'
export { WorkflowTaskType, WorkflowTaskStatus }
export const workflowTaskTypeSchema = z.nativeEnum(WorkflowTaskType)
export type WorkflowTaskTypeEnum = z.infer<typeof workflowTaskTypeSchema>
export const workflowTaskStatusSchema = z.nativeEnum(WorkflowTaskStatus)
export type WorkflowTaskStatusEnum = z.infer<typeof workflowTaskStatusSchema>
export const workflowTaskItemSchema = z.object({
id: z.string(),
uid: z.string(),
teamId: z.string().nullable().optional(),
projectId: z.string().nullable().optional(),
projectName: z.string().nullable().optional(),
assetId: z.string(),
fileName: z.string().nullable().optional(),
fileSize: z.number().nullable().optional(),
type: z.nativeEnum(WorkflowTaskType).nullable().optional(),
status: z.nativeEnum(WorkflowTaskStatus).nullable().optional(),
payload: z.unknown().optional(),
output: z.unknown().optional(),
heartbeat: z.string().nullable().optional(),
inputTokens: z.number(),
outputTokens: z.number(),
model: z.string().nullable().optional(),
sessionId: z.string().nullable().optional(),
createdAt: z.string(),
updatedAt: z.string(),
})
export type WorkflowTaskItem = z.infer<typeof workflowTaskItemSchema>
export const listWorkflowTasksQuerySchema = z.object({
first: z.coerce.number().optional(),
after: z.string().optional(),
status: z
.union([z.nativeEnum(WorkflowTaskStatus), z.array(z.nativeEnum(WorkflowTaskStatus))])
.transform((val) => (Array.isArray(val) ? val : [val]))
.optional(),
types: z
.union([z.nativeEnum(WorkflowTaskType), z.array(z.nativeEnum(WorkflowTaskType))])
.transform((val) => (Array.isArray(val) ? val : [val]))
.optional(),
})
export type ListWorkflowTasksQuery = z.infer<typeof listWorkflowTasksQuerySchema>
export const listWorkflowTasksResponseSchema = z.object({
nodes: z.array(workflowTaskItemSchema),
pageInfo: z.object({
hasNextPage: z.boolean(),
endCursor: z.string().nullable(),
}),
total: z.number(),
})
export type ListWorkflowTasksResponse = z.infer<typeof listWorkflowTasksResponseSchema>
export const batchWorkflowTaskStatusSchema = z.object({
taskIds: z.array(z.string()).max(100),
})
export type BatchWorkflowTaskStatus = z.infer<typeof batchWorkflowTaskStatusSchema>
export const batchWorkflowTaskStatusResponseSchema = z.object({
tasks: z.array(workflowTaskItemSchema),
})
export type BatchWorkflowTaskStatusResponse = z.infer<typeof batchWorkflowTaskStatusResponseSchema>
export const stopWorkflowTaskResponseSchema = z.object({
success: z.boolean(),
task: workflowTaskItemSchema,
})
export type StopWorkflowTaskResponse = z.infer<typeof stopWorkflowTaskResponseSchema>
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/e2e",
"version": "0.5.3",
"version": "0.6.1",
"private": true,
"type": "module",
"dependencies": {
+1 -1
View File
@@ -14,7 +14,7 @@ export default defineConfig({
include: ['packages/e2e/workflow/**/*.test.ts'],
server: {
deps: {
inline: ['zod', '@shumai/workflow-core', '@temporalio/activity'],
inline: ['zod', '@shumai/workflow-core', '@dbos-inc/dbos-sdk'],
},
},
},
+288 -322
View File
@@ -6,356 +6,322 @@ import { initAgentWorkflows } from '@shumai/agent'
import { initTranscodeWorkflows } from '@shumai/transcode'
import { s3Service } from '@shumai/core/src/s3/s3'
import { AgentHarness } from '@earendil-works/pi-agent-core'
import { fileURLToPath } from 'url'
import * as path from 'path'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const agentWorkflowsPath = path.resolve(currentDir, '../../../apps/agent/src/workflows.ts')
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
describe('Workflow E2E - agentAutofillMedia', () => {
setupTestDbHooks()
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - agentAutofillMedia (executor: %s)',
(mode) => {
setupTestDbHooks()
beforeAll(async () => {
// Set bucket environment
process.env.S3_BUCKET = 'shumai-e2e-test-bucket'
let agentWorkerPromise: Promise<void> | null = null
let transcodeWorkerPromise: Promise<void> | null = null
// Initialize registries
initAgentWorkflows()
initTranscodeWorkflows()
beforeAll(async () => {
// Set bucket environment
process.env.S3_BUCKET = 'shumai-e2e-test-bucket'
// Spy on the harness prompt method to intercept the LLM call while running real activities & tools
vi.spyOn(AgentHarness.prototype, 'prompt').mockImplementation(async function (
this: AgentHarness,
...args: unknown[]
) {
// Find the real tools configured on the harness
const tools = this.getTools()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Tool execution mock for E2E harness test
const readAssetTool = tools.find((t) => t.name === 'read_asset') as any
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Tool execution mock for E2E harness test
const autofillTool = tools.find((t) => t.name === 'autofill_metadata') as any
// Configure the workflow service executor dynamically
workflowService.setExecutorType(mode)
expect(readAssetTool).toBeDefined()
expect(autofillTool).toBeDefined()
// Initialize registries
initAgentWorkflows()
initTranscodeWorkflows()
const promptText = typeof args[0] === 'string' ? args[0] : ''
const assetIdMatch = promptText.match(/ID:\s*"([^"]+)"/)
// Spy on the harness prompt method to intercept the LLM call while running real activities & tools
vi.spyOn(AgentHarness.prototype, 'prompt').mockImplementation(async function (
this: AgentHarness,
...args: unknown[]
) {
// Find the real tools configured on the harness
const tools = this.getTools()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Tool execution mock for E2E harness test
const readAssetTool = tools.find((t) => t.name === 'read_asset') as any
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Tool execution mock for E2E harness test
const autofillTool = tools.find((t) => t.name === 'autofill_metadata') as any
expect(readAssetTool).toBeDefined()
expect(autofillTool).toBeDefined()
const promptText = typeof args[0] === 'string' ? args[0] : ''
const assetIdMatch = promptText.match(/ID:\s*"([^"]+)"/)
if (readAssetTool && assetIdMatch) {
const assetId = assetIdMatch[1]
const readResult = await readAssetTool.execute(
'call-read-1',
{ assetId },
undefined,
undefined,
undefined,
)
expect(readResult).toBeDefined()
expect(readResult.content).toBeDefined()
}
if (autofillTool) {
// Execute the real tool callback to perform the actual DB updates
await autofillTool.execute(
'call-123',
{
title: 'E2E Title Extracted',
confidence: 0.95,
completed: true,
},
undefined,
undefined,
undefined,
)
}
// Return a successful assistant response mock
/* eslint-disable @typescript-eslint/no-explicit-any -- Mock assistant response structure */
return {
content: [{ type: 'text', text: 'Success' }],
usage: { input: 10, output: 20 },
stopReason: 'stop',
} as any
/* eslint-enable @typescript-eslint/no-explicit-any */
})
if (mode === 'temporal') {
console.log('Starting background workers for Temporal E2E tests...')
agentWorkerPromise = workflowService.startWorkers(TaskQueueAgent, {
workflowsPath: agentWorkflowsPath,
})
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
// Wait a moment for workers to establish connections
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
if (readAssetTool && assetIdMatch) {
const assetId = assetIdMatch[1]
const readResult = await readAssetTool.execute(
'call-read-1',
{ assetId },
undefined,
undefined,
undefined,
)
expect(readResult).toBeDefined()
expect(readResult.content).toBeDefined()
}
if (autofillTool) {
// Execute the real tool callback to perform the actual DB updates
await autofillTool.execute(
'call-123',
{
title: 'E2E Title Extracted',
confidence: 0.95,
completed: true,
},
undefined,
undefined,
undefined,
)
}
// Return a successful assistant response mock
/* eslint-disable @typescript-eslint/no-explicit-any -- Mock assistant response structure */
return {
content: [{ type: 'text', text: 'Success' }],
usage: { input: 10, output: 20 },
stopReason: 'stop',
} as any
/* eslint-enable @typescript-eslint/no-explicit-any */
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([agentWorkerPromise, transcodeWorkerPromise].filter(Boolean))
}
await workflowService.startWorkers([TaskQueueAgent, TaskQueueTranscode])
})
workflowService.close()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
// Restore spied methods
vi.restoreAllMocks()
// Restore spied methods
vi.restoreAllMocks()
// Clean up E2E files in local filesystem storage
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
// Clean up E2E files in local filesystem storage
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
it('should run agentAutofillMedia workflow and update asset metadata successfully', async () => {
// 1. Seed Database Models
const team = await prisma.team.create({
data: { name: 'E2E Test Team' },
})
it('should run agentAutofillMedia workflow and update asset metadata successfully', async () => {
// 1. Seed Database Models
const team = await prisma.team.create({
data: { name: 'E2E Test Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Test Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Test Project', teamId: team.id },
})
// Create Metadata Field (autofillSource = CONTENT, text)
await prisma.metadataField.create({
data: {
key: 'title',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Title', type: 'text', autofillSource: 'CONTENT' },
description: 'Auto-extracted title',
},
})
// Create Metadata Field (autofillSource = CONTENT, text)
await prisma.metadataField.create({
data: {
key: 'title',
scope: 'PROJECT',
// Create Metadata Field (autofillSource = CONTENT, number)
await prisma.metadataField.create({
data: {
key: 'confidence',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Confidence', type: 'number', autofillSource: 'CONTENT' },
description: 'AI extraction confidence',
},
})
// Create Metadata Field (autofillSource = CONTENT, boolean)
await prisma.metadataField.create({
data: {
key: 'completed',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Completed', type: 'toggle', autofillSource: 'CONTENT' },
description: 'Completed flag',
},
})
// Create Metadata Field (autofillSource = NONE, text)
await prisma.metadataField.create({
data: {
key: 'manual_notes',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Manual Notes', type: 'text', autofillSource: 'NONE' },
description: 'Manual notes (should not be auto-filled)',
},
})
// Create Agent User
const agentUser = await prisma.user.create({
data: {
name: 'E2E Autofill Agent User',
email: 'e2e-autofill-agent@shumai.ai',
type: 'agent',
},
})
// Link Agent User to Team
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
},
})
// Setup Agent Provider & Model configuration
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
},
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'autofill',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
},
})
// Create StorageKey first
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-image.png',
},
})
// Create Asset
const asset = await prisma.asset.create({
data: {
name: 'test-image.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- partial MediaInfo mock for testing
media: { proxyType: 'image' } as any,
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage (LocalStorageService) with a valid 1x1 base64-encoded PNG image
const base64Png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII='
const pngBuffer = Buffer.from(base64Png, 'base64')
await s3Service.putObject(
'shumai-e2e-test-bucket',
'projects/e2e/test-image.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
// 3. Create Workflow Task (ai_metadata_autofill, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_metadata_autofill',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
teamId: team.id,
config: { name: 'Title', type: 'text', autofillSource: 'CONTENT' },
description: 'Auto-extracted title',
agent: { sessionId: 'mock-session-123', agentId: agent.id },
},
})
},
})
// Create Metadata Field (autofillSource = CONTENT, number)
await prisma.metadataField.create({
data: {
key: 'confidence',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Confidence', type: 'number', autofillSource: 'CONTENT' },
description: 'AI extraction confidence',
},
})
// 4. Wait for workflow to complete
console.log(`Submitted E2E Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// Create Metadata Field (autofillSource = CONTENT, boolean)
await prisma.metadataField.create({
data: {
key: 'completed',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Completed', type: 'toggle', autofillSource: 'CONTENT' },
description: 'Completed flag',
},
})
// 5. Verification
expect(completedTask.status).toBe('completed')
// Create Metadata Field (autofillSource = NONE, text)
await prisma.metadataField.create({
data: {
key: 'manual_notes',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Manual Notes', type: 'text', autofillSource: 'NONE' },
description: 'Manual notes (should not be auto-filled)',
},
})
// Verify placeholder comment is updated correctly
const comments = await prisma.assetComment.findMany({
where: { assetId: asset.id },
orderBy: { createdAt: 'asc' },
})
expect(comments.length).toBeGreaterThan(0)
// The last updated message of placeholder comment should indicate success
const finalComment = comments[comments.length - 1]
expect(finalComment.message).toBe('Autofill completed successfully.')
// Create Agent User
const agentUser = await prisma.user.create({
data: {
name: 'E2E Autofill Agent User',
email: 'e2e-autofill-agent@shumai.ai',
type: 'agent',
},
})
// Link Agent User to Team
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
},
})
// Setup Agent Provider & Model configuration
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
},
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'autofill',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
},
})
// Create StorageKey first
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-image.png',
},
})
// Create Asset
const asset = await prisma.asset.create({
data: {
name: 'test-image.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- partial MediaInfo mock for testing
media: { proxyType: 'image' } as any,
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage (LocalStorageService) with a valid 1x1 base64-encoded PNG image
const base64Png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII='
const pngBuffer = Buffer.from(base64Png, 'base64')
await s3Service.putObject(
'shumai-e2e-test-bucket',
'projects/e2e/test-image.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
// 3. Create Workflow Task (ai_metadata_autofill, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_metadata_autofill',
status: 'pending',
// Verify AssetMetadataValue contains the AI-extracted title
const titleVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
agent: { sessionId: 'mock-session-123', agentId: agent.id },
},
fieldKey: 'title',
},
})
},
})
expect(titleVal).toBeDefined()
expect(titleVal?.stringValue).toBe('E2E Title Extracted')
// 4. Wait for workflow to complete
console.log(`Submitted E2E Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify placeholder comment is updated correctly
const comments = await prisma.assetComment.findMany({
where: { assetId: asset.id },
orderBy: { createdAt: 'asc' },
})
expect(comments.length).toBeGreaterThan(0)
// The last updated message of placeholder comment should indicate success
const finalComment = comments[comments.length - 1]
expect(finalComment.message).toBe('Autofill completed successfully.')
// Verify AssetMetadataValue contains the AI-extracted title
const titleVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'title',
},
// Verify AssetMetadataValue contains the AI-extracted confidence
const confidenceVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'confidence',
},
})
expect(titleVal).toBeDefined()
expect(titleVal?.stringValue).toBe('E2E Title Extracted')
},
})
expect(confidenceVal).toBeDefined()
expect(confidenceVal?.numberValue).toBe(0.95)
// Verify AssetMetadataValue contains the AI-extracted confidence
const confidenceVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'confidence',
},
// Verify AssetMetadataValue contains the AI-extracted completed flag
const completedVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'completed',
},
})
expect(confidenceVal).toBeDefined()
expect(confidenceVal?.numberValue).toBe(0.95)
},
})
expect(completedVal).toBeDefined()
expect(completedVal?.booleanValue).toBe(true)
// Verify AssetMetadataValue contains the AI-extracted completed flag
const completedVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'completed',
},
// Verify manual_notes is NOT auto-filled (remains null/undefined in DB)
const manualNotesVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'manual_notes',
},
})
expect(completedVal).toBeDefined()
expect(completedVal?.booleanValue).toBe(true)
// Verify manual_notes is NOT auto-filled (remains null/undefined in DB)
const manualNotesVal = await prisma.assetMetadataValue.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_fieldKey: {
assetId: asset.id,
fieldKey: 'manual_notes',
},
},
})
expect(manualNotesVal).toBeNull()
}, 50000)
},
)
},
})
expect(manualNotesVal).toBeNull()
}, 50000)
})
+3 -24
View File
@@ -5,22 +5,14 @@ import { workflowService, TaskQueueAgent } from '@shumai/workflow-core'
import { initAgentWorkflows } from '@shumai/agent'
import { s3Service } from '@shumai/core/src/s3/s3'
import { AgentHarness } from '@earendil-works/pi-agent-core'
import { fileURLToPath } from 'url'
import * as path from 'path'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const agentWorkflowsPath = path.resolve(currentDir, '../../../apps/agent/src/workflows.ts')
describe.each(['local', 'temporal'] as const)('Workflow E2E - agentChat (executor: %s)', (mode) => {
describe('Workflow E2E - agentChat', () => {
setupTestDbHooks()
let agentWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-chat'
process.env.GEMINI_API_KEY = 'dummy-key'
workflowService.setExecutorType(mode)
initAgentWorkflows()
vi.spyOn(AgentHarness.prototype, 'prompt').mockResolvedValue({
@@ -29,24 +21,11 @@ describe.each(['local', 'temporal'] as const)('Workflow E2E - agentChat (executo
stopReason: 'stop',
} as unknown as Awaited<ReturnType<typeof AgentHarness.prototype.prompt>>)
if (mode === 'temporal') {
console.log('Starting background worker for chat Temporal E2E tests...')
agentWorkerPromise = workflowService.startWorkers(TaskQueueAgent, {
workflowsPath: agentWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
await workflowService.startWorkers(TaskQueueAgent)
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([agentWorkerPromise].filter(Boolean))
}
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
+353 -385
View File
@@ -29,453 +29,421 @@ vi.mock('@google/genai', () => {
})
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const agentWorkflowsPath = path.resolve(currentDir, '../../../apps/agent/src/workflows.ts')
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - agentEmbeddingMedia (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - agentEmbeddingMedia', () => {
setupTestDbHooks()
let agentWorkerPromise: Promise<void> | null = null
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
// Set bucket environment
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-embedding'
process.env.GEMINI_API_KEY = 'dummy-key'
beforeAll(async () => {
// Set bucket environment
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-embedding'
process.env.GEMINI_API_KEY = 'dummy-key'
// Initialize registries
initAgentWorkflows()
initTranscodeWorkflows()
// Configure the workflow service executor dynamically
workflowService.setExecutorType(mode)
await workflowService.startWorkers([TaskQueueAgent, TaskQueueTranscode])
})
// Initialize registries
initAgentWorkflows()
initTranscodeWorkflows()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
if (mode === 'temporal') {
console.log('Starting background workers for Temporal E2E tests...')
agentWorkerPromise = workflowService.startWorkers(TaskQueueAgent, {
workflowsPath: agentWorkflowsPath,
})
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
// Restore spied methods
vi.restoreAllMocks()
// Wait a moment for workers to establish connections
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
// Clean up E2E files in local filesystem storage
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-embedding', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
it('should run agentEmbeddingMedia workflow for an image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Embedding Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([agentWorkerPromise, transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
// Restore spied methods
vi.restoreAllMocks()
// Clean up E2E files in local filesystem storage
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-embedding', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E Image Embedding Project', teamId: team.id },
})
it('should run agentEmbeddingMedia workflow for an image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Embedding Team' },
})
const agentUser = await prisma.user.create({
data: {
name: 'E2E Image Embedding Agent User',
email: 'e2e-img-emb-agent@shumai.ai',
type: 'agent',
},
})
const project = await prisma.project.create({
data: { name: 'E2E Image Embedding Project', teamId: team.id },
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
},
})
const agentUser = await prisma.user.create({
data: {
name: 'E2E Image Embedding Agent User',
email: 'e2e-img-emb-agent@shumai.ai',
type: 'agent',
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
})
},
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
})
},
})
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-image.png',
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-image.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-image.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-image.png',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-image.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
agent: { agentId: agentUser.id },
},
})
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-image.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Embedding Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
agent: { agentId: agentUser.id },
},
},
})
// 5. Verification
expect(completedTask.status).toBe('completed')
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Embedding Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// Verify AssetComment indicates success
const comments = await prisma.assetComment.findMany({
where: { assetId: asset.id },
orderBy: { createdAt: 'asc' },
})
expect(comments.length).toBeGreaterThan(0)
const finalComment = comments[comments.length - 1]
expect(finalComment.message).toBe('Embedding completed successfully.')
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify AssetComment indicates success
const comments = await prisma.assetComment.findMany({
where: { assetId: asset.id },
orderBy: { createdAt: 'asc' },
})
expect(comments.length).toBeGreaterThan(0)
const finalComment = comments[comments.length - 1]
expect(finalComment.message).toBe('Embedding completed successfully.')
// Verify raw embedding is saved in DB
const dbEmbeddings = (await prisma.$queryRaw`
// Verify raw embedding is saved in DB
const dbEmbeddings = (await prisma.$queryRaw`
SELECT id, asset_id as "assetId", start_time as "startTime", end_time as "endTime"
FROM asset_embeddings
WHERE asset_id = ${asset.id}
`) as { id: string; startTime: number | null; endTime: number | null }[]
expect(dbEmbeddings.length).toBe(1)
expect(dbEmbeddings[0].startTime).toBeNull()
expect(dbEmbeddings[0].endTime).toBeNull()
}, 50000)
expect(dbEmbeddings.length).toBe(1)
expect(dbEmbeddings[0].startTime).toBeNull()
expect(dbEmbeddings[0].endTime).toBeNull()
}, 50000)
it('should run agentEmbeddingMedia workflow for a video asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Embedding Team' },
})
it('should run agentEmbeddingMedia workflow for a video asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Embedding Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Embedding Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Embedding Project', teamId: team.id },
})
const agentUser = await prisma.user.create({
data: {
name: 'E2E Video Embedding Agent User',
email: 'e2e-vid-emb-agent@shumai.ai',
type: 'agent',
const agentUser = await prisma.user.create({
data: {
name: 'E2E Video Embedding Agent User',
email: 'e2e-vid-emb-agent@shumai.ai',
type: 'agent',
},
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
},
})
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
})
},
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
})
},
})
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-video.mp4',
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text', 'image'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
media: { duration: 1.0 } as unknown as PrismaJson.MediaInfo, // 1-second video
projectId: project.id,
storageKeyId: storageKey.id,
},
})
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-video.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
media: { duration: 1.0 } as unknown as PrismaJson.MediaInfo, // 1-second video
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
agent: { agentId: agentUser.id },
},
})
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Video Embedding Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
agent: { agentId: agentUser.id },
},
},
})
// 5. Verification
expect(completedTask.status).toBe('completed')
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Video Embedding Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify raw embedding is saved in DB with start and end times
const dbEmbeddings = (await prisma.$queryRaw`
// Verify raw embedding is saved in DB with start and end times
const dbEmbeddings = (await prisma.$queryRaw`
SELECT id, asset_id as "assetId", start_time as "startTime", end_time as "endTime"
FROM asset_embeddings
WHERE asset_id = ${asset.id}
ORDER BY start_time ASC
`) as { id: string; startTime: number | null; endTime: number | null }[]
expect(dbEmbeddings.length).toBeGreaterThan(0)
expect(dbEmbeddings[0].startTime).toBe(0)
expect(dbEmbeddings[0].endTime).toBe(1)
}, 50000)
expect(dbEmbeddings.length).toBeGreaterThan(0)
expect(dbEmbeddings[0].startTime).toBe(0)
expect(dbEmbeddings[0].endTime).toBe(1)
}, 50000)
it('should fail agentEmbeddingMedia workflow for an unsupported audio asset', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Audio Embedding Team' },
})
it('should fail agentEmbeddingMedia workflow for an unsupported audio asset', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Audio Embedding Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Audio Embedding Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Audio Embedding Project', teamId: team.id },
})
const agentUser = await prisma.user.create({
data: {
name: 'E2E Audio Embedding Agent User',
email: 'e2e-aud-emb-agent@shumai.ai',
type: 'agent',
const agentUser = await prisma.user.create({
data: {
name: 'E2E Audio Embedding Agent User',
email: 'e2e-aud-emb-agent@shumai.ai',
type: 'agent',
},
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
},
})
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
})
},
})
await prisma.teamMember.create({
data: {
teamId: team.id,
userId: agentUser.id,
role: 'editor',
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
})
},
})
const provider = await prisma.provider.create({
data: {
name: 'google',
teamId: team.id,
config: { api: 'google-generative-ai', apiKey: 'dummy-google-api-key' },
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-audio.wav',
},
})
const model = await prisma.model.create({
data: {
modelId: 'gemini',
name: 'Gemini',
providerId: provider.id,
config: {
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 2048,
reasoning: false,
},
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-audio.wav',
type: 'file',
status: 'uploaded',
mediaType: 'audio/wav',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'embedding',
enabled: true,
providerId: provider.id,
modelId: model.id,
config: {
provider: 'google',
model: 'gemini',
},
},
})
// 2. Seed S3 Storage from Fixture
const wavPath = path.join(fixturesDir, 'small.wav')
const wavBuffer = fs.readFileSync(wavPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-audio.wav',
wavBuffer,
wavBuffer.length,
'audio/wav',
)
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-audio.wav',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test-audio.wav',
type: 'file',
status: 'uploaded',
mediaType: 'audio/wav',
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
agent: { agentId: agentUser.id },
},
})
},
})
// 2. Seed S3 Storage from Fixture
const wavPath = path.join(fixturesDir, 'small.wav')
const wavBuffer = fs.readFileSync(wavPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-embedding',
'projects/e2e/test-audio.wav',
wavBuffer,
wavBuffer.length,
'audio/wav',
)
// 4. Wait for workflow to fail and verify error message
console.log(`Submitted E2E Audio Embedding Workflow Task. ID: ${task.id}. Awaiting failure...`)
await expect(workflowService.executeWait(task, 45000)).rejects.toThrow(
'unsupported media type for embeddings: audio/wav',
)
// 3. Create Workflow Task (ai_embedding, pending)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_embedding',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
agent: { agentId: agentUser.id },
},
},
})
// 4. Wait for workflow to fail and verify error message
console.log(
`Submitted E2E Audio Embedding Workflow Task. ID: ${task.id}. Awaiting failure...`,
)
await expect(workflowService.executeWait(task, 45000)).rejects.toThrow(
'unsupported media type for embeddings: audio/wav',
)
// Verify task status in DB is failed
const dbTask = await prisma.workflowTask.findUnique({
where: { id: task.id },
})
expect(dbTask?.status).toBe(WorkflowTaskStatus.failed)
}, 50000)
},
)
// Verify task status in DB is failed
const dbTask = await prisma.workflowTask.findUnique({
where: { id: task.id },
})
expect(dbTask?.status).toBe(WorkflowTaskStatus.failed)
}, 50000)
})
+4 -23
View File
@@ -11,16 +11,11 @@ import {
type RunningTestMcpServer,
} from '@shumai/core/src/mcp/mcp-test-server'
import { AgentHarness } from '@earendil-works/pi-agent-core'
import { fileURLToPath } from 'url'
import * as path from 'path'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const agentWorkflowsPath = path.resolve(currentDir, '../../../apps/agent/src/workflows.ts')
describe.each(['local', 'temporal'] as const)('Workflow E2E - MCP tools (executor: %s)', (mode) => {
describe('Workflow E2E - MCP tools', () => {
setupTestDbHooks()
const mode = 'dbos'
let agentWorkerPromise: Promise<void> | null = null
let mcpTestServer: RunningTestMcpServer
beforeAll(async () => {
@@ -35,7 +30,6 @@ describe.each(['local', 'temporal'] as const)('Workflow E2E - MCP tools (executo
serverInfo: { name: 'e2e-mcp' },
})
workflowService.setExecutorType(mode)
initAgentWorkflows()
// Mock ONLY the AI model call. The mocked prompt locates the real `mcp`
@@ -74,24 +68,11 @@ describe.each(['local', 'temporal'] as const)('Workflow E2E - MCP tools (executo
} as unknown as Awaited<ReturnType<typeof AgentHarness.prototype.prompt>>
})
if (mode === 'temporal') {
console.log('Starting background worker for MCP Temporal E2E tests...')
agentWorkerPromise = workflowService.startWorkers(TaskQueueAgent, {
workflowsPath: agentWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
await workflowService.startWorkers(TaskQueueAgent)
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([agentWorkerPromise].filter(Boolean))
}
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
await mcpService.closeAllConnections()
@@ -9,127 +9,106 @@ import * as path from 'path'
import * as fs from 'fs'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - overlayImageAnnotationWorkflow (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - overlayImageAnnotationWorkflow', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
})
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
it('should run transcodeMedia image annotation overlay successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Annotation Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E Image Annotation Project', teamId: team.id },
})
it('should run transcodeMedia image annotation overlay successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Annotation Team' },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/image-ann.png',
},
})
const project = await prisma.project.create({
data: { name: 'E2E Image Annotation Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'image-ann.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/image-ann.png',
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/image-ann.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
const asset = await prisma.asset.create({
data: {
name: 'image-ann.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image_annotation',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/image-ann.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image_annotation',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
imageAnnotation: {
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0.1, 0.1],
[0.9, 0.9],
],
},
],
},
imageAnnotation: {
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0.1, 0.1],
[0.9, 0.9],
],
},
],
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Annotation Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Annotation Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify task output contains overlay image key
const output = completedTask.output as unknown as { key: string }
expect(output).toBeDefined()
expect(output.key).toContain('annotations/')
}, 50000)
},
)
// Verify task output contains overlay image key
const output = completedTask.output as unknown as { key: string }
expect(output).toBeDefined()
expect(output.key).toContain('annotations/')
}, 50000)
})
+75 -97
View File
@@ -9,118 +9,96 @@ import * as path from 'path'
import * as fs from 'fs'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - renderPdfPagesWorkflow (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - renderPdfPagesWorkflow', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
})
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
it('should run renderPdfPages workflow for a PDF asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PDF Pages Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E PDF Pages Project', teamId: team.id },
})
it('should run renderPdfPages workflow for a PDF asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PDF Pages Team' },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-pages.pdf',
},
})
const project = await prisma.project.create({
data: { name: 'E2E PDF Pages Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'test-pages.pdf',
type: 'file',
status: 'uploaded',
mediaType: 'application/pdf',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test-pages.pdf',
},
})
// 2. Seed S3 Storage from fixture test.pdf
const pdfPath = path.join(fixturesDir, 'test.pdf')
const pdfBuffer = fs.readFileSync(pdfPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test-pages.pdf',
pdfBuffer,
pdfBuffer.length,
'application/pdf',
)
const asset = await prisma.asset.create({
data: {
name: 'test-pages.pdf',
type: 'file',
status: 'uploaded',
mediaType: 'application/pdf',
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf_pages',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.pdf
const pdfPath = path.join(fixturesDir, 'test.pdf')
const pdfBuffer = fs.readFileSync(pdfPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test-pages.pdf',
pdfBuffer,
pdfBuffer.length,
'application/pdf',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf_pages',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
pdfPages: {
start: 1,
end: 2,
},
pdfPages: {
start: 1,
end: 2,
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(`Submitted E2E PDF Pages Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(`Submitted E2E PDF Pages Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify output contains rendered pages
const output = completedTask.output as unknown as { pages: { key: string; page: number }[] }
expect(output).toBeDefined()
expect(output.pages).toBeDefined()
expect(output.pages.length).toBeGreaterThan(0)
}, 50000)
},
)
// Verify output contains rendered pages
const output = completedTask.output as unknown as { pages: { key: string; page: number }[] }
expect(output).toBeDefined()
expect(output.pages).toBeDefined()
expect(output.pages.length).toBeGreaterThan(0)
}, 50000)
})
@@ -10,89 +10,164 @@ import * as fs from 'fs'
import sharp from 'sharp'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - takeVideoScreenshotsWorkflow (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - takeVideoScreenshotsWorkflow', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
})
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
it('should run transcodeMedia screenshot extraction successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Project', teamId: team.id },
})
it('should run transcodeMedia screenshot extraction successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Team' },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot.mp4',
},
})
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'video-shot.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot.mp4',
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
const asset = await prisma.asset.create({
data: {
name: 'video-shot.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_screenshot',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
screenshot: {
start: 0,
end: 1,
count: 1,
},
},
})
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Video Screenshot Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 3. Create Workflow Task
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify task output contains list of screenshots
const output = completedTask.output as unknown as { screenshots: { key: string }[] }
expect(output).toBeDefined()
expect(output.screenshots).toBeDefined()
expect(output.screenshots.length).toBe(1)
expect(output.screenshots[0].key).toMatch(/^projects\/e2e\/screenshots\/shot-.*\.webp$/)
}, 50000)
it('should draw annotation box correctly for 5 random comment timestamps', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Annotation Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Annotation Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot-ann.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-shot-ann.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot-ann.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 3. Generate 5 random comment timestamps between 0.05s and 0.95s
const randomTimestamps: number[] = []
for (let i = 0; i < 5; i++) {
const randomTs = 0.05 + Math.random() * 0.9
randomTimestamps.push(randomTs)
}
console.log('Testing random timestamps:', randomTimestamps)
const boxAnnotation = [
{
type: 'box' as const,
color: '#ff0000',
points: [
[0.2, 0.2],
[0.8, 0.8],
],
},
]
for (const commentTimestamp of randomTimestamps) {
// Create Workflow Task with commentTimestamp & annotations
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_screenshot',
@@ -105,217 +180,7 @@ describe.each(['local', 'temporal'] as const)(
screenshot: {
start: 0,
end: 1,
count: 1,
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Video Screenshot Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify task output contains list of screenshots
const output = completedTask.output as unknown as { screenshots: { key: string }[] }
expect(output).toBeDefined()
expect(output.screenshots).toBeDefined()
expect(output.screenshots.length).toBe(1)
expect(output.screenshots[0].key).toMatch(/^projects\/e2e\/screenshots\/shot-.*\.webp$/)
}, 50000)
it('should draw annotation box correctly for 5 random comment timestamps', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Annotation Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Annotation Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot-ann.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-shot-ann.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot-ann.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 3. Generate 5 random comment timestamps between 0.05s and 0.95s
const randomTimestamps: number[] = []
for (let i = 0; i < 5; i++) {
const randomTs = 0.05 + Math.random() * 0.9
randomTimestamps.push(randomTs)
}
console.log('Testing random timestamps:', randomTimestamps)
const boxAnnotation = [
{
type: 'box' as const,
color: '#ff0000',
points: [
[0.2, 0.2],
[0.8, 0.8],
],
},
]
for (const commentTimestamp of randomTimestamps) {
// Create Workflow Task with commentTimestamp & annotations
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_screenshot',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
screenshot: {
start: 0,
end: 1,
count: 5,
commentTimestamp,
annotations: boxAnnotation,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const output = completedTask.output as unknown as {
screenshots: { key: string; timestamp: number }[]
}
expect(output).toBeDefined()
expect(output.screenshots).toBeDefined()
// Find the screenshot matching commentTimestamp
const matchingShot = output.screenshots.find(
(s) => Math.abs(s.timestamp - commentTimestamp) < 1e-6,
)
expect(matchingShot).toBeDefined()
// Download screenshot buffer from S3
const s3Obj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
matchingShot!.key,
)
const shotBuffer = s3Obj.buffer
// Verify pixel data using sharp to check if red box annotation was overlaid
const { data, info } = await sharp(shotBuffer).raw().toBuffer({ resolveWithObject: true })
// Target pixel inside box (e.g. x = 50%, y = 20% near top edge of red box)
const pixelX = Math.floor(info.width * 0.5)
const pixelY = Math.floor(info.height * 0.2)
const offset = (pixelY * info.width + pixelX) * info.channels
const r = data[offset]
const g = data[offset + 1]
const b = data[offset + 2]
// Red box color #ff0000 should result in high red channel and low green/blue channels
expect(r).toBeGreaterThan(150)
expect(g).toBeLessThan(120)
expect(b).toBeLessThan(120)
}
}, 90000)
it('should overlay annotation when commentTimestamp has rounding mismatch with start/end', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Rounding Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Rounding Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot-rounding.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-shot-rounding.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot-rounding.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// Reproduce log scenario: exact timestamp is 0.566666666666667, Agent requested rounded 0.57
const commentTimestamp = 0.566666666666667
const start = 0.57
const end = 0.57
const boxAnnotation = [
{
type: 'box' as const,
color: '#ff0000',
points: [
[0.2, 0.2],
[0.8, 0.8],
],
},
]
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_screenshot',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
screenshot: {
start,
end,
count: 1,
count: 5,
commentTimestamp,
annotations: boxAnnotation,
},
@@ -331,16 +196,21 @@ describe.each(['local', 'temporal'] as const)(
}
expect(output).toBeDefined()
expect(output.screenshots).toBeDefined()
expect(output.screenshots.length).toBe(1)
const s3Obj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
output.screenshots[0].key,
// Find the screenshot matching commentTimestamp
const matchingShot = output.screenshots.find(
(s) => Math.abs(s.timestamp - commentTimestamp) < 1e-6,
)
expect(matchingShot).toBeDefined()
// Download screenshot buffer from S3
const s3Obj = await s3Service.getObject('shumai-e2e-test-bucket-transcode', matchingShot!.key)
const shotBuffer = s3Obj.buffer
// Verify pixel data using sharp to check if red box annotation was overlaid
const { data, info } = await sharp(shotBuffer).raw().toBuffer({ resolveWithObject: true })
// Target pixel inside box (e.g. x = 50%, y = 20% near top edge of red box)
const pixelX = Math.floor(info.width * 0.5)
const pixelY = Math.floor(info.height * 0.2)
const offset = (pixelY * info.width + pixelX) * info.channels
@@ -349,9 +219,115 @@ describe.each(['local', 'temporal'] as const)(
const g = data[offset + 1]
const b = data[offset + 2]
// Red box color #ff0000 should result in high red channel and low green/blue channels
expect(r).toBeGreaterThan(150)
expect(g).toBeLessThan(120)
expect(b).toBeLessThan(120)
}, 50000)
},
)
}
}, 90000)
it('should overlay annotation when commentTimestamp has rounding mismatch with start/end', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Video Screenshot Rounding Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Video Screenshot Rounding Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-shot-rounding.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-shot-rounding.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const mp4Path = path.join(fixturesDir, 'small.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-shot-rounding.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// Reproduce log scenario: exact timestamp is 0.566666666666667, Agent requested rounded 0.57
const commentTimestamp = 0.566666666666667
const start = 0.57
const end = 0.57
const boxAnnotation = [
{
type: 'box' as const,
color: '#ff0000',
points: [
[0.2, 0.2],
[0.8, 0.8],
],
},
]
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_screenshot',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
screenshot: {
start,
end,
count: 1,
commentTimestamp,
annotations: boxAnnotation,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const output = completedTask.output as unknown as {
screenshots: { key: string; timestamp: number }[]
}
expect(output).toBeDefined()
expect(output.screenshots).toBeDefined()
expect(output.screenshots.length).toBe(1)
const s3Obj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
output.screenshots[0].key,
)
const shotBuffer = s3Obj.buffer
const { data, info } = await sharp(shotBuffer).raw().toBuffer({ resolveWithObject: true })
const pixelX = Math.floor(info.width * 0.5)
const pixelY = Math.floor(info.height * 0.2)
const offset = (pixelY * info.width + pixelX) * info.channels
const r = data[offset]
const g = data[offset + 1]
const b = data[offset + 2]
expect(r).toBeGreaterThan(150)
expect(g).toBeLessThan(120)
expect(b).toBeLessThan(120)
}, 50000)
})
@@ -1,47 +1,9 @@
import { setup as dbSetup, teardown as dbTeardown } from '../../db/src/test-global-setup'
import { GenericContainer, StartedTestContainer } from 'testcontainers'
let temporalContainer: StartedTestContainer | null = null
export async function setup() {
// 1. Start Postgres DB container & run migrations
await dbSetup()
// 2. Start Temporal Dev Server container for the test suite
console.log('Starting Temporal Dev Server container using testcontainers...')
try {
temporalContainer = await new GenericContainer('temporalio/temporal:latest')
.withExposedPorts(7233, 8233)
.withCommand(['server', 'start-dev', '--ip', '0.0.0.0'])
.start()
const host = temporalContainer.getHost()
const mappedPort = temporalContainer.getMappedPort(7233)
const address = `${host}:${mappedPort}`
process.env.TEMPORAL_ADDRESS = address
console.log(`Temporal Dev Server container started. Address: ${address}`)
// Brief sleep to ensure temporal dev server is fully ready to accept gRPC clients
await new Promise((resolve) => setTimeout(resolve, 2000))
} catch (err) {
console.error('Failed to start Temporal Dev Server container:', err)
throw err
}
}
export async function teardown() {
// 1. Stop Temporal container if running
if (temporalContainer) {
console.log('Stopping Temporal Dev Server container...')
try {
await temporalContainer.stop()
} catch (err) {
console.error('Failed to stop Temporal Dev Server container:', err)
}
temporalContainer = null
}
// 2. Stop Postgres DB container
await dbTeardown()
}
+155 -177
View File
@@ -9,198 +9,176 @@ import * as path from 'path'
import * as fs from 'fs'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - 3D Asset Transcode (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - 3D Asset Transcode', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let mockTurntableServer: any = null
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let mockTurntableServer: any = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-3d'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-3d'
const smallPngBuffer = fs.readFileSync(path.join(fixturesDir, 'small.png'))
const smallMp4Buffer = fs.readFileSync(path.join(fixturesDir, 'small.mp4'))
const smallPngBuffer = fs.readFileSync(path.join(fixturesDir, 'small.png'))
const smallMp4Buffer = fs.readFileSync(path.join(fixturesDir, 'small.mp4'))
mockTurntableServer = Bun.serve({
port: 0,
fetch(req) {
const url = new URL(req.url)
if (url.pathname === '/version') {
return Response.json({ version: '1.0.0', runtime: 'bun', blender: '5.2.0' })
}
if (url.pathname === '/v1/files' && req.method === 'POST') {
return Response.json({ id: 'f_input3d', name: 'model.glb', size: 100 }, { status: 201 })
}
if (url.pathname === '/v1/render' && req.method === 'POST') {
return Response.json(
{
taskId: 't_render3d',
status: 'completed',
poster: {
fileId: 'f_poster3d',
size: smallPngBuffer.length,
contentType: 'image/png',
width: 1080,
height: 1080,
},
metadata: { format: 'glb' },
},
{ status: 201 },
)
}
if (url.pathname === '/v1/render/tasks/t_render3d') {
if (req.method === 'DELETE') {
return Response.json({ ok: true })
}
return Response.json({
mockTurntableServer = Bun.serve({
port: 0,
fetch(req) {
const url = new URL(req.url)
if (url.pathname === '/version') {
return Response.json({ version: '1.0.0', runtime: 'bun', blender: '5.2.0' })
}
if (url.pathname === '/v1/files' && req.method === 'POST') {
return Response.json({ id: 'f_input3d', name: 'model.glb', size: 100 }, { status: 201 })
}
if (url.pathname === '/v1/render' && req.method === 'POST') {
return Response.json(
{
taskId: 't_render3d',
status: 'completed',
posterFileId: 'f_poster3d',
video: {
fileId: 'f_video3d',
size: smallMp4Buffer.length,
contentType: 'video/mp4',
poster: {
fileId: 'f_poster3d',
size: smallPngBuffer.length,
contentType: 'image/png',
width: 1080,
height: 1080,
},
})
}
if (url.pathname === '/v1/files/f_poster3d') {
return new Response(smallPngBuffer, {
headers: { 'Content-Type': 'image/png' },
})
}
if (url.pathname === '/v1/files/f_video3d') {
return new Response(smallMp4Buffer, {
headers: { 'Content-Type': 'video/mp4' },
})
}
return new Response('Not Found', { status: 404 })
},
})
process.env.TURNTABLE_RENDERER_URL = `http://localhost:${mockTurntableServer.port}`
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
}, 120000)
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
if (mockTurntableServer) {
mockTurntableServer.stop()
}
delete process.env.TURNTABLE_RENDERER_URL
try {
console.log('Cleaning up E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-3d', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
}, 120000)
it('should process 3D model transcode workflow task', async () => {
const team = await prisma.team.create({
data: { name: 'E2E 3D Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E 3D Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/model.glb',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'model.glb',
type: 'file',
status: 'uploaded',
mediaType: 'model/gltf-binary',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const dummyGlbBuffer = Buffer.from('dummy glb content')
await s3Service.putObject(
'shumai-e2e-test-bucket-3d',
'projects/e2e/model.glb',
dummyGlbBuffer,
dummyGlbBuffer.length,
'model/gltf-binary',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_3d',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
metadata: { format: 'glb' },
},
{ status: 201 },
)
}
if (url.pathname === '/v1/render/tasks/t_render3d') {
if (req.method === 'DELETE') {
return Response.json({ ok: true })
}
return Response.json({
taskId: 't_render3d',
status: 'completed',
posterFileId: 'f_poster3d',
video: {
fileId: 'f_video3d',
size: smallMp4Buffer.length,
contentType: 'video/mp4',
width: 1080,
height: 1080,
},
})
}
if (url.pathname === '/v1/files/f_poster3d') {
return new Response(smallPngBuffer, {
headers: { 'Content-Type': 'image/png' },
})
}
if (url.pathname === '/v1/files/f_video3d') {
return new Response(smallMp4Buffer, {
headers: { 'Content-Type': 'video/mp4' },
})
}
return new Response('Not Found', { status: 404 })
},
})
process.env.TURNTABLE_RENDERER_URL = `http://localhost:${mockTurntableServer.port}`
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
}, 120000)
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
if (mockTurntableServer) {
mockTurntableServer.stop()
}
delete process.env.TURNTABLE_RENDERER_URL
try {
console.log('Cleaning up E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-3d', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
}, 120000)
it('should process 3D model transcode workflow task', async () => {
const team = await prisma.team.create({
data: { name: 'E2E 3D Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E 3D Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/model.glb',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'model.glb',
type: 'file',
status: 'uploaded',
mediaType: 'model/gltf-binary',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const dummyGlbBuffer = Buffer.from('dummy glb content')
await s3Service.putObject(
'shumai-e2e-test-bucket-3d',
'projects/e2e/model.glb',
dummyGlbBuffer,
dummyGlbBuffer.length,
'model/gltf-binary',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_3d',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
},
})
},
})
console.log(
`Submitted E2E 3D Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
console.log(`Submitted E2E 3D Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 60000)
expect(completedTask.status).toBe('completed')
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
videoPreview?: { key: string }
poster?: { key: string }
sprite?: { key: string }
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('3d')
expect(mediaInfo.videoPreview?.key).toBe('projects/e2e/turntable.mp4')
expect(mediaInfo.poster?.key).toBe('projects/e2e/poster.webp')
expect(mediaInfo.sprite?.key).toBe('projects/e2e/sprite.webp')
}, 90000)
},
)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
videoPreview?: { key: string }
poster?: { key: string }
sprite?: { key: string }
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('3d')
expect(mediaInfo.videoPreview?.key).toBe('projects/e2e/turntable.mp4')
expect(mediaInfo.poster?.key).toBe('projects/e2e/poster.webp')
expect(mediaInfo.sprite?.key).toBe('projects/e2e/sprite.webp')
}, 90000)
})
+246 -266
View File
@@ -10,312 +10,292 @@ import * as fs from 'fs'
import { GenericContainer, StartedTestContainer } from 'testcontainers'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - Real Gotenberg Transcode (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - Real Gotenberg Transcode', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
let gotenbergContainer: StartedTestContainer | null = null
let gotenbergContainer: StartedTestContainer | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-gotenberg'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-gotenberg'
console.log('Starting Gotenberg container via testcontainers...')
try {
gotenbergContainer = await new GenericContainer('gotenberg/gotenberg:8')
.withExposedPorts(3000)
.start()
const host = gotenbergContainer.getHost()
const port = gotenbergContainer.getMappedPort(3000)
process.env.GOTENBERG_URL = `http://${host}:${port}`
console.log(`Gotenberg container ready at ${process.env.GOTENBERG_URL}`)
} catch (err) {
console.error('Failed to start Gotenberg container:', err)
throw err
}
console.log('Starting Gotenberg container via testcontainers...')
try {
gotenbergContainer = await new GenericContainer('gotenberg/gotenberg:8')
.withExposedPorts(3000)
.start()
const host = gotenbergContainer.getHost()
const port = gotenbergContainer.getMappedPort(3000)
process.env.GOTENBERG_URL = `http://${host}:${port}`
console.log(`Gotenberg container ready at ${process.env.GOTENBERG_URL}`)
} catch (err) {
console.error('Failed to start Gotenberg container:', err)
throw err
}
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
}, 120000)
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
}, 120000)
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
if (gotenbergContainer) {
console.log('Stopping Gotenberg container...')
await gotenbergContainer.stop()
}
if (gotenbergContainer) {
console.log('Stopping Gotenberg container...')
await gotenbergContainer.stop()
}
try {
console.log('Cleaning up E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-gotenberg', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
}, 120000)
try {
console.log('Cleaning up E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-gotenberg', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
}, 120000)
it('should process real office Word document (.docx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg DOCX Team' },
})
it('should process real office Word document (.docx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg DOCX Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg DOCX Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg DOCX Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.docx',
status: 'active',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.docx',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.docx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.docx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
const docxPath = path.join(fixturesDir, 'test.docx')
const docxBuffer = fs.readFileSync(docxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.docx',
docxBuffer,
docxBuffer.length,
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const docxPath = path.join(fixturesDir, 'test.docx')
const docxBuffer = fs.readFileSync(docxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.docx',
docxBuffer,
docxBuffer.length,
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
transcode: {
poster: true,
sprite: true,
},
},
})
},
})
console.log(
`Submitted E2E Gotenberg DOCX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
console.log(
`Submitted E2E Gotenberg DOCX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
expect(completedTask.status).toBe('completed')
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
it('should process real office PowerPoint presentation (.pptx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg PPTX Team' },
})
it('should process real office PowerPoint presentation (.pptx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg PPTX Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg PPTX Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg PPTX Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.pptx',
status: 'active',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.pptx',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.pptx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
const asset = await prisma.asset.create({
data: {
name: 'test.pptx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const pptxPath = path.join(fixturesDir, 'test.pptx')
const pptxBuffer = fs.readFileSync(pptxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.pptx',
pptxBuffer,
pptxBuffer.length,
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const pptxPath = path.join(fixturesDir, 'test.pptx')
const pptxBuffer = fs.readFileSync(pptxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.pptx',
pptxBuffer,
pptxBuffer.length,
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
transcode: {
poster: true,
sprite: true,
},
},
})
},
})
console.log(
`Submitted E2E Gotenberg PPTX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
console.log(
`Submitted E2E Gotenberg PPTX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
expect(completedTask.status).toBe('completed')
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
it('should process real office Excel spreadsheet (.xlsx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg XLSX Team' },
})
it('should process real office Excel spreadsheet (.xlsx) transcode PDF task using Gotenberg', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Gotenberg XLSX Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg XLSX Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Gotenberg XLSX Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.xlsx',
status: 'active',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.xlsx',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.xlsx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
const asset = await prisma.asset.create({
data: {
name: 'test.xlsx',
type: 'file',
status: 'uploaded',
mediaType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const xlsxPath = path.join(fixturesDir, 'test.xlsx')
const xlsxBuffer = fs.readFileSync(xlsxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.xlsx',
xlsxBuffer,
xlsxBuffer.length,
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const xlsxPath = path.join(fixturesDir, 'test.xlsx')
const xlsxBuffer = fs.readFileSync(xlsxPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-gotenberg',
'projects/e2e/test.xlsx',
xlsxBuffer,
xlsxBuffer.length,
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
transcode: {
poster: true,
sprite: true,
},
},
})
},
})
console.log(
`Submitted E2E Gotenberg XLSX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
console.log(
`Submitted E2E Gotenberg XLSX Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 60000)
expect(completedTask.status).toBe('completed')
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
},
)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 90000)
})
+294 -315
View File
@@ -9,371 +9,350 @@ import * as path from 'path'
import * as fs from 'fs'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - transcodeImageWorkflow (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - transcodeImageWorkflow', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
})
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
it('should run transcodeMedia workflow for an image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Transcode Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E Image Transcode Project', teamId: team.id },
})
it('should run transcodeMedia workflow for an image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Image Transcode Team' },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/image-trans.png',
},
})
const project = await prisma.project.create({
data: { name: 'E2E Image Transcode Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'image-trans.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/image-trans.png',
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/image-trans.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
const asset = await prisma.asset.create({
data: {
name: 'image-trans.png',
type: 'file',
status: 'uploaded',
mediaType: 'image/png',
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture
const pngPath = path.join(fixturesDir, 'small.png')
const pngBuffer = fs.readFileSync(pngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/image-trans.png',
pngBuffer,
pngBuffer.length,
'image/png',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
transcode: {
thumbnail: true,
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
it('should run transcodeMedia workflow for a PSD image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PSD Transcode Team' },
})
it('should run transcodeMedia workflow for a PSD image asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PSD Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E PSD Transcode Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E PSD Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.psd',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.psd',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.psd',
type: 'file',
status: 'uploaded',
mediaType: 'image/vnd.adobe.photoshop',
const asset = await prisma.asset.create({
data: {
name: 'test.psd',
type: 'file',
status: 'uploaded',
mediaType: 'image/vnd.adobe.photoshop',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.psd
const psdPath = path.join(fixturesDir, 'test.psd')
const psdBuffer = fs.readFileSync(psdPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.psd',
psdBuffer,
psdBuffer.length,
'image/vnd.adobe.photoshop',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.psd
const psdPath = path.join(fixturesDir, 'test.psd')
const psdBuffer = fs.readFileSync(psdPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.psd',
psdBuffer,
psdBuffer.length,
'image/vnd.adobe.photoshop',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
transcode: {
thumbnail: true,
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E PSD Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E PSD Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
it('should run transcodeMedia workflow for a RAW image asset (.dng) successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E RAW Transcode Team' },
})
it('should run transcodeMedia workflow for a RAW image asset (.dng) successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E RAW Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E RAW Transcode Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E RAW Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.dng',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.dng',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.dng',
type: 'file',
status: 'uploaded',
mediaType: 'image/x-adobe-dng',
const asset = await prisma.asset.create({
data: {
name: 'test.dng',
type: 'file',
status: 'uploaded',
mediaType: 'image/x-adobe-dng',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.dng
const dngPath = path.join(fixturesDir, 'test.dng')
const dngBuffer = fs.readFileSync(dngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.dng',
dngBuffer,
dngBuffer.length,
'image/x-adobe-dng',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.dng
const dngPath = path.join(fixturesDir, 'test.dng')
const dngBuffer = fs.readFileSync(dngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.dng',
dngBuffer,
dngBuffer.length,
'image/x-adobe-dng',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
transcode: {
thumbnail: true,
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E RAW Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E RAW Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
it('should transcode a rotated camera photo upright', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Rotated Transcode Team' },
})
it('should transcode a rotated camera photo upright', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E Rotated Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Rotated Transcode Project', teamId: team.id },
})
const project = await prisma.project.create({
data: { name: 'E2E Rotated Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/portrait-rotated.jpg',
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/portrait-rotated.jpg',
},
})
const asset = await prisma.asset.create({
data: {
name: 'portrait-rotated.jpg',
type: 'file',
status: 'uploaded',
mediaType: 'image/jpeg',
const asset = await prisma.asset.create({
data: {
name: 'portrait-rotated.jpg',
type: 'file',
status: 'uploaded',
mediaType: 'image/jpeg',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture portrait-rotated.jpg: stored 600x400 with EXIF
// orientation 6, the way cameras save portrait shots, so it is displayed 400x600.
const jpgPath = path.join(fixturesDir, 'portrait-rotated.jpg')
const jpgBuffer = fs.readFileSync(jpgPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/portrait-rotated.jpg',
jpgBuffer,
jpgBuffer.length,
'image/jpeg',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture portrait-rotated.jpg: stored 600x400 with EXIF
// orientation 6, the way cameras save portrait shots, so it is displayed 400x600.
const jpgPath = path.join(fixturesDir, 'portrait-rotated.jpg')
const jpgBuffer = fs.readFileSync(jpgPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/portrait-rotated.jpg',
jpgBuffer,
jpgBuffer.length,
'image/jpeg',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
transcode: {
thumbnail: true,
},
},
})
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Rotated Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 4. Wait for workflow to complete
console.log(
`Submitted E2E Rotated Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification: the asset and its transcode are portrait, not the stored landscape.
expect(completedTask.status).toBe('completed')
// 5. Verification: the asset and its transcode are portrait, not the stored landscape.
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
imageTranscodes: Array<{ width: number; height: number }>
metadata: { originalWidth: number; originalHeight: number }
}
expect(mediaInfo.metadata.originalWidth).toBe(400)
expect(mediaInfo.metadata.originalHeight).toBe(600)
expect(mediaInfo.imageTranscodes[0].width).toBe(400)
expect(mediaInfo.imageTranscodes[0].height).toBe(600)
}, 50000)
},
)
const mediaInfo = updatedAsset?.media as unknown as {
imageTranscodes: Array<{ width: number; height: number }>
metadata: { originalWidth: number; originalHeight: number }
}
expect(mediaInfo.metadata.originalWidth).toBe(400)
expect(mediaInfo.metadata.originalHeight).toBe(600)
expect(mediaInfo.imageTranscodes[0].width).toBe(400)
expect(mediaInfo.imageTranscodes[0].height).toBe(600)
}, 50000)
})
+362 -253
View File
@@ -4,204 +4,391 @@ import { setupTestDbHooks } from '@shumai/db/test'
import { workflowService, TaskQueueTranscode } from '@shumai/workflow-core'
import { initTranscodeWorkflows } from '@shumai/transcode'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService } from '@shumai/core/src/transcode/transcode'
import { fileURLToPath } from 'url'
import * as path from 'path'
import * as fs from 'fs'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - transcodePdfWorkflow (executor: %s)',
(mode) => {
setupTestDbHooks()
describe('Workflow E2E - transcodePdfWorkflow', () => {
setupTestDbHooks()
let transcodeWorkerPromise: Promise<void> | null = null
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
beforeAll(async () => {
process.env.S3_BUCKET = 'shumai-e2e-test-bucket-transcode'
initTranscodeWorkflows()
await workflowService.startWorkers(TaskQueueTranscode)
})
workflowService.setExecutorType(mode)
initTranscodeWorkflows()
afterAll(async () => {
await workflowService.shutdownWorkers()
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
})
if (mode === 'temporal') {
console.log('Starting background worker for transcode Temporal E2E tests...')
transcodeWorkerPromise = workflowService.startWorkers(TaskQueueTranscode, {
workflowsPath: transcodeWorkflowsPath,
})
await new Promise((resolve) => setTimeout(resolve, 2000))
} else {
console.log('Starting local workflow service polling...')
workflowService.start()
}
it('should run transcodeMedia workflow for a PDF asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PDF Transcode Team' },
})
afterAll(async () => {
if (mode === 'temporal') {
console.log('Shutting down Temporal workers...')
await workflowService.shutdownWorkers()
await Promise.all([transcodeWorkerPromise].filter(Boolean))
}
workflowService.close()
vi.restoreAllMocks()
try {
console.log('Cleaning up local E2E storage files...')
await s3Service.deletePrefix('shumai-e2e-test-bucket-transcode', '')
} catch (err) {
console.error('Failed to clean up E2E storage folder:', err)
}
const project = await prisma.project.create({
data: { name: 'E2E PDF Transcode Project', teamId: team.id },
})
it('should run transcodeMedia workflow for a PDF asset successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E PDF Transcode Team' },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.pdf',
},
})
const project = await prisma.project.create({
data: { name: 'E2E PDF Transcode Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'test.pdf',
type: 'file',
status: 'uploaded',
mediaType: 'application/pdf',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.pdf',
},
})
// 2. Seed S3 Storage from fixture test.pdf
const pdfPath = path.join(fixturesDir, 'test.pdf')
const pdfBuffer = fs.readFileSync(pdfPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.pdf',
pdfBuffer,
pdfBuffer.length,
'application/pdf',
)
const asset = await prisma.asset.create({
data: {
name: 'test.pdf',
type: 'file',
status: 'uploaded',
mediaType: 'application/pdf',
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.pdf
const pdfPath = path.join(fixturesDir, 'test.pdf')
const pdfBuffer = fs.readFileSync(pdfPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.pdf',
pdfBuffer,
pdfBuffer.length,
'application/pdf',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
transcode: {
poster: true,
sprite: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(`Submitted E2E PDF Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify Asset status is updated to processed
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
// Verify media info contains pdf details, poster, sprite, and pageCount
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
poster: { key: string }
sprite: { key: string }
frames?: number
pageCount?: number
metadata?: { totalFrames?: number; pageCount?: number }
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.poster).toBeDefined()
expect(mediaInfo.poster.key).toContain('poster.webp')
expect(mediaInfo.sprite).toBeDefined()
expect(mediaInfo.sprite.key).toContain('sprite.webp')
expect(mediaInfo.frames ?? mediaInfo.metadata?.totalFrames).toBeGreaterThan(0)
}, 50000)
it('should run transcodeMedia workflow for a TXT asset with CJK text to generate PDF proxy successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E CJK TXT Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E CJK TXT Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/cjk-test.txt',
},
})
const asset = await prisma.asset.create({
data: {
name: 'cjk-test.txt',
type: 'file',
status: 'uploaded',
mediaType: 'text/plain',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage with CJK text
const cjkText =
'Hello World\n你好世界 (Chinese)\nこんにちは世界 (Japanese)\n안녕하세요世界 (Korean)\nLine 5 of random text.'
const txtBuffer = Buffer.from(cjkText, 'utf-8')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/cjk-test.txt',
txtBuffer,
txtBuffer.length,
'text/plain',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E CJK TXT Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 50000)
it('should process markdown (.md) transcode PDF task correctly', async () => {
// 1. Create Team, Project, Asset & StorageKey
const team = await prisma.team.create({
data: { name: 'E2E MD Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E MD Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/doc.md',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'doc.md',
type: 'file',
status: 'uploaded',
mediaType: 'text/markdown',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage with Markdown text
const mdText = '# Heading\n\nThis is a **markdown** document to be rendered to PDF proxy.'
const mdBuffer = Buffer.from(mdText, 'utf-8')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/doc.md',
mdBuffer,
mdBuffer.length,
'text/markdown',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(`Submitted E2E MD Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 50000)
it('should recover and complete workflow when intermediate local proxy is deleted before downstream steps', async () => {
// 1. Create Team, Project, Asset & StorageKey
const team = await prisma.team.create({
data: { name: 'E2E Recovery Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Recovery Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/recovery-doc.md',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'recovery-doc.md',
type: 'file',
status: 'uploaded',
mediaType: 'text/markdown',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage with Markdown text
const mdText = '# Recovery Test\n\nVerifying that intermediate proxy.pdf recovers from S3.'
const mdBuffer = Buffer.from(mdText, 'utf-8')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/recovery-doc.md',
mdBuffer,
mdBuffer.length,
'text/markdown',
)
// 3. Spy on createTempDir and s3Service to intercept proxy.pdf upload and delete it locally
let capturedTmpDir: string | undefined
const origCreateTempDir = transcodeService.createTempDir.bind(transcodeService)
const createTempDirSpy = vi
.spyOn(transcodeService, 'createTempDir')
.mockImplementation((prefix: string) => {
const dir = origCreateTempDir(prefix)
if (prefix.startsWith('transcode-')) {
capturedTmpDir = dir
}
return dir
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E PDF Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
const origPutObject = s3Service.putObject.bind(s3Service)
const origDownloadToFile = s3Service.downloadToFile.bind(s3Service)
const downloadedKeys: string[] = []
// 5. Verification
expect(completedTask.status).toBe('completed')
// Verify Asset status is updated to processed
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
const downloadSpy = vi
.spyOn(s3Service, 'downloadToFile')
.mockImplementation(async (bucket, key, localPath) => {
downloadedKeys.push(key)
return origDownloadToFile(bucket, key, localPath)
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
// Verify media info contains pdf details, poster, sprite, and pageCount
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
poster: { key: string }
sprite: { key: string }
frames?: number
pageCount?: number
metadata?: { totalFrames?: number; pageCount?: number }
const putSpy = vi.spyOn(s3Service, 'putObject').mockImplementation(async (...args) => {
const result = await origPutObject(...args)
const key = String(args[1] || '')
if (key.endsWith('proxy.pdf') && capturedTmpDir) {
// Once proxy.pdf has been uploaded to S3 by generatePdfProxyActivity,
// delete the local file to simulate empty filesystem on a recovery/worker node.
const localProxyPath = path.join(capturedTmpDir, 'proxy.pdf')
if (fs.existsSync(localProxyPath)) {
fs.unlinkSync(localProxyPath)
}
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.poster).toBeDefined()
expect(mediaInfo.poster.key).toContain('poster.webp')
expect(mediaInfo.sprite).toBeDefined()
expect(mediaInfo.sprite.key).toContain('sprite.webp')
expect(mediaInfo.frames ?? mediaInfo.metadata?.totalFrames).toBeGreaterThan(0)
}, 50000)
return result
})
it('should run transcodeMedia workflow for a TXT asset with CJK text to generate PDF proxy successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E CJK TXT Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E CJK TXT Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/cjk-test.txt',
},
})
const asset = await prisma.asset.create({
data: {
name: 'cjk-test.txt',
type: 'file',
status: 'uploaded',
mediaType: 'text/plain',
// 4. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage with CJK text
const cjkText =
'Hello World\n你好世界 (Chinese)\nこんにちは世界 (Japanese)\n안녕하세요世界 (Korean)\nLine 5 of random text.'
const txtBuffer = Buffer.from(cjkText, 'utf-8')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/cjk-test.txt',
txtBuffer,
txtBuffer.length,
'text/plain',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
transcode: {
poster: true,
sprite: true,
},
},
})
},
})
// 4. Wait for workflow to complete
try {
console.log(
`Submitted E2E CJK TXT Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
`Submitted E2E Recovery Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
@@ -226,92 +413,14 @@ describe.each(['local', 'temporal'] as const)(
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 50000)
it('should process markdown (.md) transcode PDF task correctly', async () => {
// 1. Create Team, Project, Asset & StorageKey
const team = await prisma.team.create({
data: { name: 'E2E MD Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E MD Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/doc.md',
status: 'active',
},
})
const asset = await prisma.asset.create({
data: {
name: 'doc.md',
type: 'file',
status: 'uploaded',
mediaType: 'text/markdown',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage with Markdown text
const mdText = '# Heading\n\nThis is a **markdown** document to be rendered to PDF proxy.'
const mdBuffer = Buffer.from(mdText, 'utf-8')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/doc.md',
mdBuffer,
mdBuffer.length,
'text/markdown',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_pdf',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
poster: true,
sprite: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E MD Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType?: string
pdfTranscode?: { key: string }
poster?: { key: string }
sprite?: { key: string }
frames?: number
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('pdf')
expect(mediaInfo.pdfTranscode?.key).toBe('projects/e2e/proxy.pdf')
expect(mediaInfo.poster?.key).toContain('poster.webp')
expect(mediaInfo.sprite?.key).toContain('sprite.webp')
expect(mediaInfo.frames).toBeGreaterThan(0)
}, 50000)
},
)
// Verify that proxy.pdf was re-downloaded from S3 via ensureLocalMediaFile
const proxyDownloadCall = downloadedKeys.find((k) => k.endsWith('proxy.pdf'))
expect(proxyDownloadCall).toBeDefined()
} finally {
createTempDirSpy.mockRestore()
downloadSpy.mockRestore()
putSpy.mockRestore()
}
}, 50000)
})
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More