* feat(storage): add archive storage support for assets
* fix(webui): show archive icon in front of file name in breadcrumb bar
* fix(webui): display archive icon in breadcrumbs when restoring
* refactor(transcode): simplify restore asset workflow to single while loop
* docs(configuration): add archive storage documentation and readme entries
* fix(webui): show archive icon in front of version names and prevent version card overflow
* fix(webui): fix overflow in manage versions dialog scroll area
* fix(webui): prevent manage versions dialog overflow on long file names
* docs(configuration): fix archive storage environment variables and defaults
* fix(storage): address code review findings for archive storage
Replace the monolithic agent read_asset tool with a reader registry in
@shumai/core/src/asset-read (model3d, image, video, document) behind a
shared pipeline that handles access checks, version stacks, annotations,
render tasks, and waiting.
- Add read_asset and get_asset_render Operations (agent surface only).
Renders that outlast the caller's wait budget return "pending" with the
workflow task ID; the read request is stored on the task payload so
get_asset_render works across restarts and instances (requester only).
- Add workflowService.waitFor and presign expiry support.
- Keep agent output compatible via an agent tool extension (s3KeyOnly,
image blocks, session image details); autofill keeps its team-scoped tool.
- Cap video frames at 20 per call; add the get_asset_render agent toggle.
* feat(dashboard): add workflow tasks monitoring and stopping to team dashboard
* fix(asset): retain workflow tasks on asset purge and mark active tasks as failed
* style(webui): add edge padding to audit logs and workflow tasks tables
* fix(workflow): atomically update asset status to failed when task is stopped
* fix(workflow): propagate cancellation errors, guard concurrent task stop, and localize UI
* docs(changelog): keep only added log for workflow tasks dashboard
* test(provider): mock upstream catalog api calls using saved fixtures
* feat(dashboard): show file name, size, and project name in workflow tasks list
* feat(workflow): migrate workflow engine from temporal to dbos
Replace Temporal and LocalExecutor with DBOS Transact (@dbos-inc/dbos-sdk) as the unified workflow engine. Background tasks, media transcoding, and AI agent workflows now execute with durable steps and task queues directly against PostgreSQL, eliminating the need for an external Temporal cluster and separate polling executors.
* fix(workflow): submit tasks via DBOSClient to allow enqueueing from steps
Enqueuing workflows using DBOS.enqueueWorkflowWithOptions threw a
DBOSInvalidWorkflowTransitionError when triggered from within a running
step or activity (e.g. creating embedding or autofill tasks during
video transcoding).
Use DBOSClient to submit and cancel workflow tasks directly against the
system database, detaching execution from the ambient workflow/step
context.
* feat(web): support SHUMAI_WORKERS environment variable
* fix(workflow): ensure CLI tasks are enqueued to DBOS and run DBOS migrations in global setup
- Initialize workflows and wait for pending DBOS submissions in shumai regenerate-embeddings CLI
- Remove no-op submit bypass and enqueue tasks via DBOSClient in producer and CLI contexts
- Add DBOS migration runner in test global setup to avoid concurrent migration deadlocks across test forks
- Update changelog and add regression tests
* fix(workflow): ensure unique per-instance executor identity for worker replicas
- Derive unique per-instance DBOS executor IDs across worker replicas
- Support SHUMAI_WORKER_ID, DBOS__VMID, and POD_NAME (Kubernetes Downward API)
- Disambiguate multi-replica bare metal/container deployments with host, PID, and instance suffix
- Prevent startup recovery from re-enqueueing workflows actively executed by other live workers
- Add multi-instance startup recovery tests and update deployment documentation
* fix(workflow): only register listened queues and update concurrency on change
- Only register queues that the executor instance actively listens to
- Use always_update conflict resolution so environment variable changes take effect on restart
- Prevent dedicated workers from overwriting unmanaged queue concurrency configurations
- Add tests verifying concurrency updates and queue isolation across worker roles
* refactor(workflow): simplify activities to use defineStep and direct calls (#482)
* refactor(workflow): simplify activities to use defineStep and direct calls
- Wrap activities using defineStep from @shumai/workflow-core
- Remove obsolete getActivities, registerActivities, dynamic activity proxies, and workerQueue arguments
- Remove executor.ts and registry.ts, consolidating workflow registration into workflow.ts
- Call activities directly across transcode and agent workflows
- Update all corresponding workflow tests and developer guide documentation
* fix(workflow): properly detect DBOSWorkflowCancelledError in runOnCancel fallback
- DBOSWorkflowCancelledError in DBOS SDK does not set name, inheriting 'Error'
- Implement isDbosWorkflowCancelledError checking instanceof, constructor.name, and dbosErrorCode
- Update unit tests to verify real DBOS SDK cancellation error instances
* docs(workflow): update documentation and readme for dbos migration
- Rename docs/configuration/temporal.mdx to docs/configuration/distributed.mdx
- Document DBOS-based distributed worker architecture and task queues
- Remove obsolete docker-compose/temporal/
- Update environment variables documentation in docs/configuration/env-variables.mdx
- Update references in README.md, README_CN.md, docs/docs.json, and related guides
* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding (#483)
* feat(deploy): add shumai-transcode docker image and k8s autoscaling example with garage s3
* docs(k8s): document port-forwarding and in-cluster s3 domain mapping
* feat(storage): support public s3 endpoint for browser uploads and update k8s example
* fix(k8s): configure s3 bucket cors rules in garage initialization
* fix(k8s): increase transcode worker memory limit to 4Gi for 4K video encoding
* feat(deploy): add AWS EKS deployment with Graviton3 Spot nodegroups and autoscaling
* docs(deploy): redesign EKS documentation, rename k8s to eks, and add scalable deployment feature
* docs: rephrase DBOS feature bullet to Durable Workflows via DBOS
* docs(eks): upgrade default instances to Graviton5 c9g.large and c9g.xlarge
* feat(eks): use On-Demand for general nodegroup and add spot capacity diversification
* docs(eks): fix Architecture diagram to locate all transcode worker pods in Transcode Compute Pool
* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding
* feat(transcode): support idle-timeout auto-drain for KEDA scaled jobs
* fix(deploy): fix EKS deployment manifests and documentation based on cluster verification
* feat(search): generate query embeddings directly for semantic search
* fix(search): resolve embedding provider from env, redact query logs, and preserve case in cache
* feat(transcode): add 3D asset support and turntable viewer
* perf(turntable): stream file via Bun.file and increase upload timeout
* feat(turntable): increase upload timeout to 15m to support 5GB uploads
* fix(webui): align 3D model viewer layout and full-width control bar with video viewer
* feat(webui): unify 3D viewer bottom bar and redesign cyclic ruler slider
- Place RulerSlider in single row with playback and zoom controls, centered with max 320px width
- Flank RulerSlider with minus and plus buttons to step angle by 15°
- Redesign RulerSlider with fixed center indicator and moving cyclic track with continuous 360° wrapping
- Render major ticks longer at 60° intervals without numeric text labels on track
- Display floating degree tooltip above center indicator only during dragging
* fix(webui): prevent RulerSlider from collapsing to 0px width in control bar
- Use flex layout with flex-1 on left and right clusters to preserve center cluster width
- Enforce min-w-[140px] and w-full on RulerSlider
- Fix tooltip positioning and remove outer overflow-hidden
* feat(webui): snap RulerSlider track directly to discrete 15° ticks during drag
* fix(webui): prevent ruler snap-back caused by floating point second truncation
- Use Math.round instead of Math.floor in secondToDegree to avoid 0.999996 dropping to previous frame
- Ensure final dragDegree is committed on pointer up
* fix(webui): fix arrow key frame rounding and duplicate event bubbling
- Use Math.round instead of Math.floor for keyboard arrow frame navigation in ModelViewer
- Add e.stopPropagation() to RulerSlider keyboard events to prevent duplicate bubbling to window listener
* feat(webui): redesign RulerSlider with encapsulated step buttons and capsule pill track
* feat(webui): make RulerSlider responsive on mobile and remove bottom bar reset button
* fix(webui): load 3d video directly without poster and invert drag rotation to counterclockwise
* fix(webui): fix 3d asset viewer frame seeking and drag jitter
* feat(transcode): show 3D asset poster preview immediately after render start
* fix(viewer): synchronize 3D turntable playback via frame callback to prevent skipped degrees
* fix(viewer): handle touchpad zoom gestures and prevent page zoom in 3D viewer
* fix(viewer): eliminate 3D model rotation jitter at 345 degrees
* fix(share): watermark 3D asset turntable video on share links
* fix(share): preserve video transcode resolution on watermarked share links
* fix(viewer): download original 3D model file instead of rendered turntable MP4
* fix(asset): mark asset as failed and display failure state in webui when transcoding fails
* fix(transcode): do not mark asset as failed if media processing already succeeded
* test(workflow): add e2e test for video transcode failure status transition
* fix(webui): update failed file card badge to top-right Transcoding Failed label with error tooltip
* fix(transcode): preserve trashed status during background jobs and handle permanent purge
* fix(workflow): clear heartbeat and add info logs on task cancellation
* fix(transcode): terminate media processes and guard task activities on cancellation
- Pass AbortSignal to ffmpeg and pdftoppm child processes in transcodeService to terminate processes on task cancellation
- Add local task AbortController context and cancellation guards in executeActivity to prevent subsequent workflow activities
- Forward taskId and cancellation signals in transcode, screenshot, and PDF activities
- Use updateMany instead of update in task activities to eliminate Prisma P2025 errors on purged task records
* fix(workflow): allow cleanup activities in finally blocks during task cancellation
Remove blanket cancellation check in executeActivity so that workflows can execute cleanup and failure handling activities in catch and finally blocks upon cancellation, while relying on AbortSignal in active activities to abort execution.
* docs(changelog): improve user friendliness of changelog entries and formatting example
* docs(agents): update changelog guidelines to emphasize end-user friendly descriptions
* fix(transcode): handle queued cancellation and delete races in media workflow
* chore(deps): upgrade bun to 1.4.2 and update packages
- Upgrade Bun to 1.4.2 in Dockerfile, CI workflows, and release workflows.
- Update dependencies across workspace with Bun 1.4.2.
- Preserve @earendil-works/pi-agent-core at 0.82.1.
- Retain stable compatible versions for database adapters and pdfjs-dist.
- Add explicit Buffer type annotation in transcode service.
* chore(deps): upgrade prisma to 7.10.0, pg to 8.23.0, and pdfjs-dist to 6.3.289
* chore(deps): revert dotenv-expand to 13.0.0
* fix(webui): update sidebar selection styling, kanban active state, and pointer cursors
* feat(webui): move dashboard to left sidebar and prevent mount animation
* fix(e2e): update create-project test to click Home button in sidebar
* feat(sharelink): add sharelink watermark support
* fix(sharelink): harden watermark feature against races, crashes, and tenant leaks
Addresses review findings on the sharelink watermark feature:
- fix(watermark): read back sharelink via transaction client so the
updateShareLinkWatermark response reflects the just-committed state
(global-client reads on a pooled connection miss uncommitted changes)
- fix(watermark): use prisma upsert on the config hash instead of
find-then-create to avoid P2002 races on concurrent enables
- fix(watermark): resolve nested symlinks when walking the share tree and
batch the walk level-by-level instead of one query per node
- fix(watermark): keep watermark tasks in flight from being purged by
orphan-config GC (avoids FK violations in completing activities)
- fix(transcode): re-transcode stale 'processing' watermark rows instead
of silently skipping (crash recovery); downscale embedded block images;
use pino structured logging; drop unused any casts
- fix(api): scope watermark-template list/create to the caller's teams,
reject teamless template creation, and make teamless templates
read-only; validate watermark config required when enabling (400);
map sharelink-not-found to 404
- fix(api): resolve symlink target ids once in public share children
endpoint instead of twice per item
- refactor(db): dedicated WatermarkFileStatus enum (migration included)
- test: activity unit tests (init crash recovery, image + video transcode
paths), concurrent upsert, nested-symlink resolution, API scoping and
validation, e2e video watermark path
* refactor(transcode): move sharp usage out of watermark activity into transcodeService
The transcode package does not declare sharp as a dependency (only
@shumai/core does), so importing it directly in the watermark activity
broke CI typecheck for the transcode package.
- add transcodeService.renderSvgToPng, downscaleImageToPng and
compositeOverlayToWebpFile encapsulating the sharp logic
- watermark activity now calls these methods instead of importing sharp
- activity tests mock the new methods (no sharp/real fixture needed)
- add unit tests for the new transcodeService helpers
* feat(watermark): restrict watermark transcode to existing image and video transcodes
* refactor(transcode): reuse transcodeVideo in watermark activity
* refactor(watermark): use Bun.hash instead of node crypto for watermark config hash
* refactor(watermark): move computeWatermarkConfigHash from dtos to core watermark-hash module
* refactor(api): delegate watermark DB queries from public-share route to watermarkService
* refactor(share): replace dynamic watermarkService import with static import
* feat(watermark): add waitForWatermarkFileActivity for in-flight watermark transcoding tasks
* refactor(asset): replace dynamic watermarkService import with static import
* test(watermark): harden purgeOrphanWatermarkConfigs coverage
Replace the single happy-path GC test with a dedicated suite covering:
- orphan config with proxies: purged, watermarkFiles removed, all four
proxy keys deleted from S3 while the original key is preserved
- config still referenced by a share link is not purged
- config with pending/processing transcode_watermark tasks is not purged
(in-flight guard), but completed tasks do not block purging
- pending tasks of other types (transcode_video) do not protect a config
- an S3 delete failure is non-fatal: remaining keys still deleted and the
config is still purged
- mixed run: only the orphan config is purged while a referenced one
survives (exact count assertion)
* feat(webui): add sharelink watermark editor and sidebar controls
* style(webui): enlarge watermark editor dialog and allocate 2/3 width to preview column
* fix(webui): override sm:max-w-lg in DialogContent to allow full 1400px width
* fix(webui): stop block click propagation and remove preview mode toggles
* fix(webui): store watermark block size as fraction of canvas width
The editor stored block.size in pixels (slider 10-120px, preview
fontSize: `${block.size}px`) while the SVG generator
(generateWatermarkSvg) interprets size as a fraction (0..1) of canvas
width. A 36px block therefore rendered as a ~36x-canvas glyph covering
the entire image.
- Keep backend fraction semantics (canvasWidth * size) so watermarks
scale proportionally with output resolution across proxies
- Editor: defaults (text 5%, image 15%), slider ranges (1-12% / 2-50%)
and number input now work in percent, storing a fraction
- Preview: scales blocks by the measured canvas width so WYSIWYG
- DTO schema: size constrained to (0..1) to enforce the contract
- Update DTO/editor tests to fraction sizes
* feat(webui): add single-row aspect ratio selector and 5 predefined background colors to watermark editor preview
* feat(webui): update aspect ratio to multi-toggle group and enable dot-grid pattern for all background colors
* fix(webui): update background swatches to distinct colors and share PRESET_BACKGROUND_OPTIONS array
* fix(webui): hide redundant absolute close button on watermark editor dialog header
* fix(transcode): skip raw video transcode when generating watermark proxies
The raw marker entry (isRaw: true) in videoTranscodes points at the
original source file and is only used to offer the 'Original' playback
option. It is not a transcoded proxy, so watermarking it produced a
redundant full-resolution transcode (the most expensive pass) plus an
unneeded S3 upload.
Skip entries with isRaw: true in transcodeWatermarkMediaActivity, and
cover the behavior with a unit regression test plus an e2e assertion
that exactly one watermark proxy is produced for a video asset.
* feat(webui): replace watermark image asset ID with 1MB file uploader
* fix(webui): correct watermark editor preview block sizing
The preview canvas width was measured in a useEffect([open]) that ran
before the Radix Dialog had mounted its content, so canvasRef.current was
still null, canvasWidthPx stayed 0, and every block silently fell back to
the hardcoded 640px width (block.size * (canvasWidthPx || 640)). A 100%
size therefore rendered at ~640px on an ~876px canvas (~72%), making the
preview smaller than the real watermarked output for both image and text
blocks.
- Measure the canvas via a ref callback (fires exactly when the node
mounts, regardless of dialog mount timing) and keep a ResizeObserver
attached for aspect-ratio/window changes.
- Use clientWidth (padding-box width) instead of getBoundingClientRect(),
which includes the dialog's zoom-in-95 animation transform.
- Drop max-w-full/max-h-full on the preview <img> so logos scale up to
the block size like the backend SVG rendering does, instead of being
capped at their natural pixel size.
* fix(webui): keep watermark editor preview aspect ratio and restore config scroll
- Compute the preview canvas size with a JS contain-fit (ResizeObserver on the
frame) instead of relying on CSS aspect-ratio + w-full/max-h-full, which
broke the selected ratio on wide-and-short (or tall-and-narrow) windows.
- Add min-h-0 to the config ScrollArea so it shrinks to the column's remaining
height instead of growing to its content height, restoring the scroll range.
* fix(webui): remove toast notification on template load
* fix(share): serve per-file watermark proxies in public share APIs
Stop gating watermark replacement on the share-link-wide watermarkStatus.
Public share endpoints now resolve each asset's own completed watermark
proxy and:
- replace videoTranscodes/imageTranscodes with the watermarked entries
when the watermark file is completed;
- empty those arrays when no completed proxy exists yet (still
transcoding or failed) so original, unwatermarked media is never
exposed while a newly added file is being processed;
- leave non-video/image media (pdf, audio, ...) untouched, since they
never receive watermark transcodes.
This removes the previous behavior where adding a file to a ready
watermark share flipped the whole share link to processing and served
original media for every file until the new transcode finished.
Adds API tests for detail/children endpoints (ready, in-flight, pdf, and
watermark-disabled cases) and a workflow E2E test for adding an asset to
an already-ready watermark sharelink.