Commit Graph
20 Commits
Author SHA1 Message Date
yiling a71723d455 docs: add BETTER_AUTH_SECRET to docker-compose environment variables 2026-11-06 10:50:18 +08:00
yiling ab250a9f0a docs: clarify BETTER_AUTH_URL usage and scope 2026-11-06 10:36:22 +08:00
Yiling-J f304caea00 fix(auth,web): gracefully degrade MCP on remote HTTP and cache production static assets (#519)
* fix: gracefully handle remote HTTP URLs for MCP and cache production static assets

* test(mcp): add remote http mcp integration test and fix static etag normalization and fallback error
2026-11-05 22:21:11 +08:00
Yiling-J 9529c63c0a feat(upload): add 12-hour upload expiration and continuous backoff retry (#508)
* feat(upload): add 12-hour upload expiration and continuous backoff retry

* fix(upload): clamp jittered backoff delay to maxDelayMs and update presign JSDoc
2026-10-30 16:06:34 +08:00
Yiling-J 687b5a79d8 fix(core): strip trailing slashes when normalizing auth and MCP resource URLs (#506)
* fix(core): strip trailing slashes when normalizing auth and MCP resource URLs

* docs: document BETTER_AUTH_URL in README and quickstart guides

* docs: prioritize AWS_ENDPOINT_URL_S3 and clarify applicability to local storage

* docs: refine AWS_ENDPOINT_URL_S3 guidance for local storage deployments
2026-10-29 17:24:31 +08:00
Yiling-J 176b7f0eab feat(workflow): migrate workflow engine from temporal to dbos (#481)
* feat(workflow): migrate workflow engine from temporal to dbos

Replace Temporal and LocalExecutor with DBOS Transact (@dbos-inc/dbos-sdk) as the unified workflow engine. Background tasks, media transcoding, and AI agent workflows now execute with durable steps and task queues directly against PostgreSQL, eliminating the need for an external Temporal cluster and separate polling executors.

* fix(workflow): submit tasks via DBOSClient to allow enqueueing from steps

Enqueuing workflows using DBOS.enqueueWorkflowWithOptions threw a
DBOSInvalidWorkflowTransitionError when triggered from within a running
step or activity (e.g. creating embedding or autofill tasks during
video transcoding).

Use DBOSClient to submit and cancel workflow tasks directly against the
system database, detaching execution from the ambient workflow/step
context.

* feat(web): support SHUMAI_WORKERS environment variable

* fix(workflow): ensure CLI tasks are enqueued to DBOS and run DBOS migrations in global setup

- Initialize workflows and wait for pending DBOS submissions in shumai regenerate-embeddings CLI
- Remove no-op submit bypass and enqueue tasks via DBOSClient in producer and CLI contexts
- Add DBOS migration runner in test global setup to avoid concurrent migration deadlocks across test forks
- Update changelog and add regression tests

* fix(workflow): ensure unique per-instance executor identity for worker replicas

- Derive unique per-instance DBOS executor IDs across worker replicas

- Support SHUMAI_WORKER_ID, DBOS__VMID, and POD_NAME (Kubernetes Downward API)

- Disambiguate multi-replica bare metal/container deployments with host, PID, and instance suffix

- Prevent startup recovery from re-enqueueing workflows actively executed by other live workers

- Add multi-instance startup recovery tests and update deployment documentation

* fix(workflow): only register listened queues and update concurrency on change

- Only register queues that the executor instance actively listens to

- Use always_update conflict resolution so environment variable changes take effect on restart

- Prevent dedicated workers from overwriting unmanaged queue concurrency configurations

- Add tests verifying concurrency updates and queue isolation across worker roles

* refactor(workflow): simplify activities to use defineStep and direct calls (#482)

* refactor(workflow): simplify activities to use defineStep and direct calls

- Wrap activities using defineStep from @shumai/workflow-core
- Remove obsolete getActivities, registerActivities, dynamic activity proxies, and workerQueue arguments
- Remove executor.ts and registry.ts, consolidating workflow registration into workflow.ts
- Call activities directly across transcode and agent workflows
- Update all corresponding workflow tests and developer guide documentation

* fix(workflow): properly detect DBOSWorkflowCancelledError in runOnCancel fallback

- DBOSWorkflowCancelledError in DBOS SDK does not set name, inheriting 'Error'
- Implement isDbosWorkflowCancelledError checking instanceof, constructor.name, and dbosErrorCode
- Update unit tests to verify real DBOS SDK cancellation error instances

* docs(workflow): update documentation and readme for dbos migration

- Rename docs/configuration/temporal.mdx to docs/configuration/distributed.mdx
- Document DBOS-based distributed worker architecture and task queues
- Remove obsolete docker-compose/temporal/
- Update environment variables documentation in docs/configuration/env-variables.mdx
- Update references in README.md, README_CN.md, docs/docs.json, and related guides

* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding (#483)

* feat(deploy): add shumai-transcode docker image and k8s autoscaling example with garage s3

* docs(k8s): document port-forwarding and in-cluster s3 domain mapping

* feat(storage): support public s3 endpoint for browser uploads and update k8s example

* fix(k8s): configure s3 bucket cors rules in garage initialization

* fix(k8s): increase transcode worker memory limit to 4Gi for 4K video encoding

* feat(deploy): add AWS EKS deployment with Graviton3 Spot nodegroups and autoscaling

* docs(deploy): redesign EKS documentation, rename k8s to eks, and add scalable deployment feature

* docs: rephrase DBOS feature bullet to Durable Workflows via DBOS

* docs(eks): upgrade default instances to Graviton5 c9g.large and c9g.xlarge

* feat(eks): use On-Demand for general nodegroup and add spot capacity diversification

* docs(eks): fix Architecture diagram to locate all transcode worker pods in Transcode Compute Pool

* feat(deploy): implement KEDA ScaledJob and scale-to-zero On-Demand transcoding

* feat(transcode): support idle-timeout auto-drain for KEDA scaled jobs

* fix(deploy): fix EKS deployment manifests and documentation based on cluster verification
2026-10-19 15:35:08 +08:00
Yiling-J 8805d5799a feat(transcode): add dcraw_emu fallback for RAW image preview generation (#446) 2026-10-03 11:03:31 +08:00
Yiling-J 94ea4c917a feat(transcode): add VA-API hardware acceleration support (#430)
* feat(transcode): add VA-API hardware acceleration support

* fix(docker): switch to jellyfin-ffmpeg 8.1.2-5 for VA-API support

* fix(transcode): probe hardware encoder usability to skip uninitialized drivers

* ci(release): add workflow_dispatch trigger with custom tag input

* fix(transcode): use 256x256 frame dimension for encoder usability probe

* feat(docker): add hwaccel transcoding compose presets matching immich

* docs: add hardware transcoding documentation and quickstart references

* feat(transcode): prioritize h264_vaapi before h264_qsv on Linux

* fix(docker): extend device permission loop to rkmpp and wsl accelerator nodes
2026-09-26 20:55:57 +08:00
yiling e1d3dd46db feat(comments): allow marking comments as complete or incomplete 2026-07-06 12:24:40 +08:00
yiling b98b710222 update doc 2026-07-06 11:22:32 +08:00
yiling af8bc30409 docs: improve documentation and examples for S3 compatible storage 2026-07-06 10:13:12 +08:00
yiling ab9b33320d docs(s3): document SHUMAI_SERVER_PORT and S3 endpoint port configurations 2026-06-25 18:48:23 +08:00
yiling efd6163238 refactor(s3): remove automatic port appending to AWS_ENDPOINT_URL_S3 2026-06-25 18:46:14 +08:00
yiling 5e70b6f3ee docs: add note blocks for running binaries under Bun runtime 2026-06-22 15:39:47 +08:00
yiling ce457e1161 fix(s3): remove port from AWS_ENDPOINT_URL_S3 and sync with server port in local mode 2026-06-18 10:22:16 +08:00
yiling 718f50ef10 docs: add temporal deploy guide and link from quickstart 2026-06-16 14:53:03 +08:00
yiling 006774042a docs: refine documentation to be user-oriented and remove database jargon 2026-06-16 14:44:42 +08:00
yiling 2b058bde40 docs: remove legacy introduction and docker docs 2026-06-16 14:39:04 +08:00
yiling fc6ad20329 docs: improve and restructure shumai documentation 2026-06-16 12:15:53 +08:00
yiling 11b34736d2 feat(docs): add mintlify documentation with docker compose guide 2026-06-04 11:19:52 +08:00