Compare commits

..
13 Commits
Author SHA1 Message Date
yiling 013c4a8156 Release v0.5.0 2026-10-02 20:28:37 +08:00
Yiling-J 20b5ebeefd fix(asset): mark asset as failed and display failure state in webui when transcoding fails (#445)
* fix(asset): mark asset as failed and display failure state in webui when transcoding fails

* fix(transcode): do not mark asset as failed if media processing already succeeded

* test(workflow): add e2e test for video transcode failure status transition

* fix(webui): update failed file card badge to top-right Transcoding Failed label with error tooltip
2026-10-02 20:09:32 +08:00
Yiling-J daa20aed20 fix(webui): add video loading state and poster placeholder (#444)
* fix(webui): add video loading state and poster placeholder

* fix(webui): prevent buffer progress events from canceling stall indicator
2026-10-02 16:19:49 +08:00
Yiling-J 3564d4d164 fix(transcode): avoid selecting attached picture streams for video transcoding (#443)
* fix(transcode): avoid selecting attached picture streams for video transcoding

Filter out attached picture streams (disposition.attached_pic) during video probing and stream selection, matching Immich's default and bitrate selection strategy. Forward explicit primary video and audio stream indices to FFmpeg pipelines to prevent picking embedded cover art or thumbnail streams.

* fix(transcode): skip poster and sprite generation for audio-only assets and add multi-stream e2e tests

* test(workflow): scope pixel format to h264 stream and use yuvj420p for mjpeg cover art
2026-10-02 13:51:17 +08:00
Yiling-J 206763e966 feat(auth): add demo user login button and update demo description (#442)
* feat(auth): add demo user login button and update description

* feat(auth): move demo hint below button and add discord link
2026-10-01 22:04:15 +08:00
Yiling-J 4e4da595a6 feat(settings): redesign video transcoding settings into dedicated cards and rename team AI settings (#441)
* feat(settings): redesign video transcoding settings into dedicated cards and rename team AI settings

* feat(settings): rename Video Strategy to MP4 Transcoding with sub-description
2026-10-01 21:19:25 +08:00
Yiling-J a21a0c9655 fix(settings): prevent deselecting last ladder and coalesce rapid toggles (#440) 2026-10-01 16:35:47 +08:00
Yiling-J fc7282581f fix(share): prevent unwatermarked HLS playback on public shares before transcode completion (#439) 2026-10-01 15:41:45 +08:00
Yiling-J e007054f47 feat(transcode): add multi-resolution strategy with configurable ladders and fallback (#438)
* feat(transcode): add multi-resolution strategy with configurable ladders and fallback

* feat(settings): show toast on settings update and add manual verification plan to PR template

* fix(webui): preserve video resolution and fix 480p label in download dropdowns

* fix(webui): display standard short-side resolution for vertical videos in seekbar

* fix(webui): fix portrait transcode resolution labeling and serialize settings toggles
2026-10-01 13:27:50 +08:00
Yiling-J a84da8678e feat(webui): switch from video.js to hls.js and native video player (#437)
* feat(webui): switch from video.js to hls.js and native video player

* fix(webui): prevent stale seek hijacking and guard native HLS compare mode
2026-09-30 20:23:44 +08:00
Yiling-J 123a8cb5f4 feat(transcode): add HLS live streaming support (#436)
* feat(transcode): add HLS live streaming support

* fix(hls): resolve version stacks for public HLS and add workflow E2E test
2026-09-30 16:31:40 +08:00
Yiling-J 5fe22ac52c feat(transcode): add initial camera RAW image support (#435)
- Add centralized RAW file extension set and detection helper for 23 common camera RAW formats
- Update getProxyType and EXTENSION_MIME_TYPES to route RAW formats into the image pipeline
- Add exiftool-vendored runtime dependency and mark as external in build-npm script
- Implement embedded JPEG extraction to temporary file with priority JpgFromRaw2 -> JpgFromRaw -> PreviewImage
- Read and apply EXIF orientation from RAW container to prevent rotation/mirroring issues
- Integrate into TranscodeService.getImageInfo and transcodeImage with proper temporary file cleanup
- Add hasUsablePreview guard in transcodeImageWorkflow to gracefully handle RAW files without embedded previews
- Add unit, service, workflow, and E2E tests for RAW processing
2026-09-29 21:09:27 +08:00
Yiling-J 11d21444cf fix(trash): prevent empty trash dialog from closing before API call completes (#434)
AlertDialogAction from Radix UI auto-closes the dialog on click.
Added e.preventDefault() so the dialog stays open and shows the
'Deleting...' loading state until the operation finishes.

Fixes both desktop and mobile file browsers.
2026-09-29 10:40:13 +08:00
106 changed files with 9070 additions and 1326 deletions
+7
View File
@@ -92,6 +92,13 @@ Briefly describe what this PR changes and why.
- Describe the checks or tests you ran.
- Include relevant outputs if applicable.
## Manual Verification Plan
- Provide concise instructions for manually verifying this change.
- Assume familiarity with the project and existing development workflow.
- Skip boilerplate steps (e.g. starting the app, logging in, opening common pages) unless they are part of the change.
- Focus on what needs to be verified, including scenarios, edge cases, and expected results.
## Notes
Add any additional context, caveats, or follow-up work.
+23
View File
@@ -12,6 +12,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Changed
## [0.5.0] - 2026-10-02
### Added
* **transcode**: Add configurable target resolution ladders (480p, 720p, 1080p, 1440p, 2160p) for MP4 Multi Resolutions transcoding in team settings, with unified resolution matching and lowest-resolution fallback for both MP4 and HLS
* **transcode**: Add HLS live streaming with fMP4/CMAF adaptive bitrate ladders (480p, 720p, 1080p, 1440p, 2160p), configurable in Transcode Settings, with automatic quality switching, manual resolution selection in the video viewer, seamless rendition switching, and full watermarking support for shared links
* **transcode**: Add basic camera RAW image support for popular camera formats (including Sony ARW, Canon CR2/CR3, Nikon NEF, Adobe DNG, Fujifilm RAF, and Panasonic RW2), extracting high-quality embedded JPEG previews and automatically applying camera orientation for fast thumbnail and proxy generation
### Fixed
- **asset**: Fix an issue where media files remained stuck in a perpetual "Preparing..." state when transcoding failed, by displaying a clear failed status and error details in the file list and file viewer
* **transcode**: Fix an issue where videos or audio-only files containing attached cover art or picture streams could cause transcoding to fail during poster generation or select a single static image instead of the video
* **webui**: Fix inconsistent video resolution labels across download menus, mobile file headers, and the video player by correctly deriving resolutions from the video's short side, including portrait videos and 480p videos
* **trash**: Fix an issue where the "Empty Trash" confirmation dialog would close immediately instead of showing a loading state while files were being permanently deleted
### Changed
* **settings**: Redesign team Video Transcoding settings with dedicated cards for Transcode Policy, Hardware Acceleration, and FFmpeg Threads, clearly explaining MP4 and HLS playback vs download behavior and storage implications
* **settings**: Rename AI Settings group to Team AI Settings in settings navigation
## [0.4.9] - 2026-09-28
### Added
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/premiere-uxp",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "commonjs",
"scripts": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"dependencies": {
+35 -50
View File
@@ -33,7 +33,7 @@
},
"apps/agent": {
"name": "@shumai/agent-app",
"version": "0.4.9",
"version": "0.5.0",
"bin": {
"shumai-agent": "./src/index.ts",
},
@@ -45,7 +45,7 @@
},
"apps/cli": {
"name": "@shumai/cli",
"version": "0.4.9",
"version": "0.5.0",
"bin": {
"shumai-cli": "./src/index.ts",
},
@@ -60,7 +60,7 @@
},
"apps/premiere-uxp": {
"name": "@shumai/premiere-uxp",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@shumai/api": "workspace:*",
"@shumai/dtos": "workspace:*",
@@ -114,7 +114,7 @@
},
"apps/transcode": {
"name": "@shumai/transcode-app",
"version": "0.4.9",
"version": "0.5.0",
"bin": {
"shumai-transcode": "./src/index.ts",
},
@@ -126,7 +126,7 @@
},
"apps/web": {
"name": "@shumai/web-app",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@shumai/agent": "workspace:*",
"@shumai/api": "workspace:*",
@@ -146,7 +146,7 @@
},
"packages/agent": {
"name": "@shumai/agent",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.74",
"@earendil-works/pi-agent-core": "0.82.1",
@@ -171,7 +171,7 @@
},
"packages/api": {
"name": "@shumai/api",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@hono/zod-validator": "0.7.6",
"@shumai/core": "workspace:*",
@@ -185,7 +185,7 @@
},
"packages/core": {
"name": "@shumai/core",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@ai-sdk/alibaba": "2.0.38",
"@ai-sdk/amazon-bedrock": "5.0.67",
@@ -223,6 +223,7 @@
"date-fns": "4.4.0",
"dotenv": "17.4.2",
"dotenv-expand": "13.0.0",
"exiftool-vendored": "38.1.0",
"extract-zip": "2.0.1",
"hono": "4.13.5",
"jittered-fractional-indexing": "1.0.1",
@@ -246,7 +247,7 @@
},
"packages/db": {
"name": "@shumai/db",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@prisma/adapter-pg": "7.10.0",
@@ -262,7 +263,7 @@
},
"packages/dtos": {
"name": "@shumai/dtos",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"zod": "4.5.4",
@@ -270,7 +271,7 @@
},
"packages/e2e": {
"name": "@shumai/e2e",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@google/genai": "2.19.0",
@@ -285,11 +286,11 @@
},
"packages/tableprinter": {
"name": "@shumai/tableprinter",
"version": "0.4.9",
"version": "0.5.0",
},
"packages/transcode": {
"name": "@shumai/transcode",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -303,7 +304,7 @@
},
"packages/webui": {
"name": "@shumai/webui",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@base-ui/react": "1.7.0",
"@dnd-kit/abstract": "0.5.0",
@@ -363,6 +364,7 @@
"cmdk": "1.1.1",
"date-fns": "4.4.0",
"embla-carousel-react": "8.6.0",
"hls.js": "1.7.3",
"hono": "4.13.5",
"input-otp": "1.5.0",
"konva": "10.3.2",
@@ -386,7 +388,6 @@
"ulid": "3.0.2",
"use-image": "1.1.4",
"vaul": "1.1.2",
"video.js": "8.24.0",
"zod": "4.5.4",
"zustand": "5.0.15",
},
@@ -405,7 +406,7 @@
},
"packages/workflow-core": {
"name": "@shumai/workflow-core",
"version": "0.4.9",
"version": "0.5.0",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -911,6 +912,8 @@
"@oxc-project/types": ["@oxc-project/types@0.147.0", "", {}, "sha512-IJ3s6ltHLp45S0bh7phkX+gJO7A1Wuz2EaqpAhb8WjqDwbzMiWKHhyyT42tskaWjEYXtHtVCPpnBJVT9+dcRLg=="],
"@photostructure/tz-lookup": ["@photostructure/tz-lookup@11.7.0", "", {}, "sha512-QbKHXvleluMIlAHf2JiQTJNU5rLQBeQhLobyRqgXZ6NNGKqNikaRkjYviJ4nMU7Y75QE7bBYM8xjiFGjAukyhw=="],
"@pinojs/redact": ["@pinojs/redact@0.4.0", "", {}, "sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg=="],
"@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="],
@@ -1561,6 +1564,8 @@
"@types/lodash": ["@types/lodash@4.17.25", "", {}, "sha512-+K1NIO8I+F9/wNulfVvu23QYd0Pe9/OCqRrim4NoYIf1VoEDL90Ve4ClzpyqBLc7NpGGWRvYNCKZ1BE/Jpf8dQ=="],
"@types/luxon": ["@types/luxon@3.7.5", "", {}, "sha512-jJ41Q4z6ZVO260MNDdHfW7+7a5iMiX8Mr6ZJHcmgrvhZha6dz5704o/lF2kKl6URjH6ivEL97w9xS/MgpJEphg=="],
"@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="],
"@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="],
@@ -1627,12 +1632,6 @@
"@vercel/oidc": ["@vercel/oidc@3.2.0", "", {}, "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug=="],
"@videojs/http-streaming": ["@videojs/http-streaming@3.17.5", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.2", "aes-decrypter": "^4.0.2", "global": "^4.4.0", "m3u8-parser": "^7.2.0", "mpd-parser": "^1.4.0", "mux.js": "7.1.0", "video.js": "^7 || ^8" } }, "sha512-PRRQVu0hYW5h0XvRWpIyIpEatpMZzZtf7L1avV89Hnu+69f+0c7oLsSL9jwpg2nbvzGkdX5rQxQkz5Vc7tFR9w=="],
"@videojs/vhs-utils": ["@videojs/vhs-utils@4.1.2", "", { "dependencies": { "@babel/runtime": "^7.12.5", "global": "^4.4.0" } }, "sha512-1Dsv5dtWOq0LP1G37O46FQFA2tKAZON5UMe6dFPxT7JvPhEx8QIXcFQxan8+hHI3fX6aomFRFrUBJNgRYQDSXA=="],
"@videojs/xhr": ["@videojs/xhr@2.7.0", "", { "dependencies": { "@babel/runtime": "^7.5.5", "global": "~4.4.0", "is-function": "^1.0.1" } }, "sha512-giab+EVRanChIupZK7gXjHy90y3nncA2phIOyG3Ne5fvpiMJzvqYwiTOnEVW2S4CoYcuKJkomat7bMXA/UoUZQ=="],
"@visx/curve": ["@visx/curve@4.0.1-alpha.0", "", { "dependencies": { "@visx/vendor": "4.0.0-alpha.0" } }, "sha512-jRu61Uz274pV1zyioXmboyrLutYbnKsgjj4njSGCnhdXj5GkZvZbg+ThDb6oOzoAnJOBRLz4rzPlWvNJOzuVMg=="],
"@visx/event": ["@visx/event@4.0.1-alpha.0", "", { "dependencies": { "@types/react": "*", "@visx/point": "4.0.1-alpha.0" } }, "sha512-EQqCMSv/s8NbFjo+hz3FKsvvYfP+2QslsFJ/24/O5l/W+7UC6J6aAvO0ujVwrTwdYbuQ+vhxKi1xdPdKR/qj1g=="],
@@ -1697,8 +1696,6 @@
"@workflow/serde": ["@workflow/serde@4.1.0", "", {}, "sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ=="],
"@xmldom/xmldom": ["@xmldom/xmldom@0.8.15", "", {}, "sha512-/5NV/vDALVFDXgLmfsy9TRCBlKwO2LNBFzpzvb9iIj+jR+eSc6DLYYvVOdivT/jm7MtU6TebYuRmzEOI7w40UA=="],
"@xtuc/ieee754": ["@xtuc/ieee754@1.2.0", "", {}, "sha512-DX8nKgqcGwsc0eJSqYt5lwP4DH5FlHnmuWWBRy7X0NcaGR0ZtuyeESgMwTYVEtxmsNGY+qit4QYT/MIYTOTPeA=="],
"@xtuc/long": ["@xtuc/long@4.2.2", "", {}, "sha512-NuHqBY1PB/D8xU6s/thBgOAiAP7HOYDQ32+BFZILJ8ivkUkAHQnWfn6WhL79Owj1qmUnoN/YPhktdIoucipkAQ=="],
@@ -1711,8 +1708,6 @@
"adm-zip": ["adm-zip@0.6.0", "", {}, "sha512-XleryMhbuksdKtofnWZ9Sk+4CUTbms4Mb/EU32SZwToAyZ5RgVos/ki8n+yr0LWHOGKuakbXTuuYNHLQjhddgg=="],
"aes-decrypter": ["aes-decrypter@4.0.2", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.1", "global": "^4.4.0", "pkcs7": "^1.0.4" } }, "sha512-lc+/9s6iJvuaRe5qDlMTpCFjnwpkeOXp8qP3oiZ5jsj1MRg+SBVUmmICrhxHvc8OELSmc+fEyyxAuppY6hrWzw=="],
"agent-base": ["agent-base@7.1.4", "", {}, "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ=="],
"ai": ["ai@7.0.84", "", { "dependencies": { "@ai-sdk/gateway": "4.0.68", "@ai-sdk/provider": "4.0.8", "@ai-sdk/provider-utils": "5.0.33" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-znHNBWADOy79k8EK7636kDm5LmnjF4tMzegC2TfmlLDobDZbs0z0OXD2sxtH3eDTp5Re+pJQESvjinWJ6+kadQ=="],
@@ -1795,6 +1790,8 @@
"baseline-browser-mapping": ["baseline-browser-mapping@2.11.20", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-H0ulySigv6icDJ1F7SjtdCD6PrhTpdYCmP0CactWy1+ekh0AFd0o1Wn5T8b+hnTmdBx19u9yhL6wvCylXMY7zw=="],
"batch-cluster": ["batch-cluster@19.2.0", "", {}, "sha512-mvozaN/7TYxl+lmXDYH+HocCcRMvW25QmghrJux2y93o+NhHPQ4wir0nBS1p8oTlinjc/l5LaNX+VFxNISrqgQ=="],
"bcrypt-pbkdf": ["bcrypt-pbkdf@1.0.2", "", { "dependencies": { "tweetnacl": "^0.14.3" } }, "sha512-qeFIXtP4MSoi6NLqO12WfqARWWuCKi2Rn/9hJLEmtB5yTNr9DqFWkJRCf2qShWzPeAMRnOgCrq0sg/KLv5ES9w=="],
"better-auth": ["better-auth@1.6.11", "", { "dependencies": { "@better-auth/core": "1.6.11", "@better-auth/drizzle-adapter": "1.6.11", "@better-auth/kysely-adapter": "1.6.11", "@better-auth/memory-adapter": "1.6.11", "@better-auth/mongo-adapter": "1.6.11", "@better-auth/prisma-adapter": "1.6.11", "@better-auth/telemetry": "1.6.11", "@better-auth/utils": "0.4.0", "@better-fetch/fetch": "1.1.21", "@noble/ciphers": "^2.1.1", "@noble/hashes": "^2.0.1", "better-call": "1.3.5", "defu": "^6.1.4", "jose": "^6.1.3", "kysely": "^0.28.17", "nanostores": "^1.1.1", "zod": "^4.3.6" }, "peerDependencies": { "@lynx-js/react": "*", "@prisma/client": "^5.0.0 || ^6.0.0 || ^7.0.0", "@sveltejs/kit": "^2.0.0", "@tanstack/react-start": "^1.0.0", "@tanstack/solid-start": "^1.0.0", "better-sqlite3": "^12.0.0", "drizzle-kit": ">=0.31.4", "drizzle-orm": "^0.45.2", "mongodb": "^6.0.0 || ^7.0.0", "mysql2": "^3.0.0", "next": "^14.0.0 || ^15.0.0 || ^16.0.0", "pg": "^8.0.0", "prisma": "^5.0.0 || ^6.0.0 || ^7.0.0", "react": "^18.0.0 || ^19.0.0", "react-dom": "^18.0.0 || ^19.0.0", "solid-js": "^1.0.0", "svelte": "^4.0.0 || ^5.0.0", "vitest": "^2.0.0 || ^3.0.0 || ^4.0.0", "vue": "^3.0.0" }, "optionalPeers": ["@lynx-js/react", "@prisma/client", "@sveltejs/kit", "@tanstack/react-start", "@tanstack/solid-start", "better-sqlite3", "drizzle-kit", "drizzle-orm", "mongodb", "mysql2", "next", "pg", "prisma", "react", "react-dom", "solid-js", "svelte", "vitest", "vue"] }, "sha512-Wwt6+q07dwIhsp6XiM7L1qSXVUWBEtNl+eZvwM778CguFqDZFBN9Pt6LtFaHl55t8Z+Zc//5kxcbgDY8/79vFQ=="],
@@ -2001,8 +1998,6 @@
"dom-accessibility-api": ["dom-accessibility-api@0.5.16", "", {}, "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg=="],
"dom-walk": ["dom-walk@0.1.2", "", {}, "sha512-6QvTW9mrGeIegrFXdtQi9pk7O/nSK6lSdXW2eqUspN5LWD7UTji2Fqw5V2YLjBpHEoU9Xl/eUWNpDeZvoyOv2w=="],
"dotenv": ["dotenv@17.4.2", "", {}, "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw=="],
"dotenv-expand": ["dotenv-expand@13.0.0", "", { "dependencies": { "dotenv": "^17.4.2" } }, "sha512-aBfBS8eYIeXmpHI9ThIlA7/WLq+SLt18iXUZhb52rW89QLKQFoIpPG1bPeewoPZsTyjSSO3T7234FBVUM1V2rA=="],
@@ -2105,6 +2100,12 @@
"eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="],
"exiftool-vendored": ["exiftool-vendored@38.1.0", "", { "dependencies": { "@photostructure/tz-lookup": "^11.7.0", "@types/luxon": "^3.7.5", "batch-cluster": "^19.2.0", "he": "^1.2.0", "luxon": "^3.7.2" }, "optionalDependencies": { "exiftool-vendored.exe": "13.59.2", "exiftool-vendored.pl": "13.59.2" } }, "sha512-LIERqM/Qb0oqJlSswLFKWsxGVH81skp7omsX2BPKA/dEzfLP5syNO5KL7k5UlMCf+m0jqEXrKTQr/xV5vScPow=="],
"exiftool-vendored.exe": ["exiftool-vendored.exe@13.59.2", "", { "os": "win32" }, "sha512-U0l0CApIGO+AjpFYCiNCApQsMdlf6sqCyEl4wlKKqvwVAEVcvMRw91k7zj32OPFZE+/+XlLFsc6876MpOqBLiw=="],
"exiftool-vendored.pl": ["exiftool-vendored.pl@13.59.2", "", { "os": "!win32", "bin": { "exiftool": "bin/exiftool" } }, "sha512-5B80JT9phZSUsc7zz77rQ6DfW0YssTpJ5qW/1bOH1N8XQq/jcn+9eRW9HFwhVT9uoIliw9Fg007ST3pUzQIzyw=="],
"expect-type": ["expect-type@1.4.0", "", {}, "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA=="],
"exsolve": ["exsolve@1.1.1", "", {}, "sha512-9U/jZUgjnSGyntRr6y5Muu1MJcwFl6kPu7k8qLF0IMNfLqvw0NZ4nnVDq0RVoZ0RvCyumib4Ez3KYrVfilrw+g=="],
@@ -2213,8 +2214,6 @@
"glob-to-regex.js": ["glob-to-regex.js@1.2.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-QMwlOQKU/IzqMUOAZWubUOT8Qft+Y0KQWnX9nK3ch0CJg0tTp4TvGZsTfudYKv2NzoQSyPcnA6TYeIQ3jGichQ=="],
"global": ["global@4.4.0", "", { "dependencies": { "min-document": "^2.19.0", "process": "^0.11.10" } }, "sha512-wv/LAoHdRE3BeTGz53FAamhGlPLhlssK45usmGFThIi4XqnBmjKQ16u+RNbP7WvigRZDxUsM0J3gcQ5yicaL0w=="],
"globals": ["globals@17.11.0", "", {}, "sha512-Z2I8hM+PbJDXQDq3Icgpzv+mPdwr68iZUU9d5WW4FuXfDUQfkZaZuvjMv42/5crNyw154+9+VWXbYrUgDXbxNw=="],
"globalthis": ["globalthis@1.0.4", "", { "dependencies": { "define-properties": "^1.2.1", "gopd": "^1.0.1" } }, "sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ=="],
@@ -2253,10 +2252,14 @@
"hast-util-whitespace": ["hast-util-whitespace@3.0.0", "", { "dependencies": { "@types/hast": "^3.0.0" } }, "sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw=="],
"he": ["he@1.2.0", "", { "bin": { "he": "bin/he" } }, "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw=="],
"heap-js": ["heap-js@2.7.1", "", {}, "sha512-EQfezRg0NCZGNlhlDR3Evrw1FVL2G3LhU7EgPoxufQKruNBSYA8MiRPHeWbU+36o+Fhel0wMwM+sLEiBAlNLJA=="],
"help-me": ["help-me@5.0.0", "", {}, "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg=="],
"hls.js": ["hls.js@1.7.3", "", {}, "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A=="],
"hono": ["hono@4.13.5", "", {}, "sha512-O6+/eCYRkzzzy0rPWwKLiGBR1nFuUPZynnwjxN1MBA62NNqbT0wQEzQyK2gSO5yDIDB336sXQleAhOHrzlYyKw=="],
"html-url-attributes": ["html-url-attributes@3.0.1", "", {}, "sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ=="],
@@ -2317,8 +2320,6 @@
"is-fullwidth-code-point": ["is-fullwidth-code-point@3.0.0", "", {}, "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg=="],
"is-function": ["is-function@1.0.2", "", {}, "sha512-lw7DUp0aWXYg+CBCN+JKkcE0Q2RayZnSvnZBlwgxHBQhqt5pZNVy4Ri7H9GmmXkdu7LUthszM+Tor1u/2iBcpQ=="],
"is-generator-function": ["is-generator-function@1.1.2", "", { "dependencies": { "call-bound": "^1.0.4", "generator-function": "^2.0.0", "get-proto": "^1.0.1", "has-tostringtag": "^1.0.2", "safe-regex-test": "^1.1.0" } }, "sha512-upqt1SkGkODW9tsGNG5mtXTXtECizwtS2kA161M+gJPc1xdb/Ax629af6YrTwcOeQHbewrPNlE5Dx7kzvXTizA=="],
"is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="],
@@ -2471,9 +2472,9 @@
"lucide-react": ["lucide-react@1.37.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-LPsB4rD1TD6wZu1djKOf9vUnS1jTNaHbolXebXDgiTdb6jeA1agIJhJsIybCmjKmQClcOaal1o1OaiYahEftyQ=="],
"lz-string": ["lz-string@1.5.0", "", { "bin": { "lz-string": "bin/bin.js" } }, "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ=="],
"luxon": ["luxon@3.7.2", "", {}, "sha512-vtEhXh/gNjI9Yg1u4jX/0YVPMvxzHuGgCm6tC5kZyb08yjGWGnqAjGJvcXbqQR2P3MyMEFnRbpcdFS6PBcLqew=="],
"m3u8-parser": ["m3u8-parser@7.2.0", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.1", "global": "^4.4.0" } }, "sha512-CRatFqpjVtMiMaKXxNvuI3I++vUumIXVVT/JpCpdU/FynV/ceVw1qpPyyBNindL+JlPMSesx+WX1QJaZEJSaMQ=="],
"lz-string": ["lz-string@1.5.0", "", { "bin": { "lz-string": "bin/bin.js" } }, "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ=="],
"magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="],
@@ -2545,8 +2546,6 @@
"mime-match": ["mime-match@1.0.2", "", { "dependencies": { "wildcard": "^1.1.0" } }, "sha512-VXp/ugGDVh3eCLOBCiHZMYWQaTNUHv2IJrut+yXA6+JbLPXHglHwfS/5A5L0ll+jkCY7fIzRJcH6OIunF+c6Cg=="],
"min-document": ["min-document@2.19.2", "", { "dependencies": { "dom-walk": "^0.1.0" } }, "sha512-8S5I8db/uZN8r9HSLFVWPdJCvYOejMcEC82VIzNUc6Zkklf/d1gg2psfE79/vyhWOj4+J8MtwmoOz3TmvaGu5A=="],
"minimatch": ["minimatch@10.2.6", "", { "dependencies": { "brace-expansion": "^5.0.8" } }, "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A=="],
"minimist": ["minimist@1.2.8", "", {}, "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA=="],
@@ -2559,12 +2558,8 @@
"mkdirp-classic": ["mkdirp-classic@0.5.3", "", {}, "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A=="],
"mpd-parser": ["mpd-parser@1.4.0", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.2", "@xmldom/xmldom": "^0.8.3", "global": "^4.4.0" }, "bin": { "mpd-to-m3u8-json": "bin/parse.js" } }, "sha512-blbA7XpAaOdC/PR0Tu8GiUxlx6CpBuRowQPYV7lhi9Yr9G9+dkPXIUjqjzc/NCn/uZZPopluaJlIYntIEI/VLw=="],
"ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
"mux.js": ["mux.js@7.1.0", "", { "dependencies": { "@babel/runtime": "^7.11.2", "global": "^4.4.0" }, "bin": { "muxjs-transmux": "bin/transmux.js" } }, "sha512-NTxawK/BBELJrYsZThEulyUMDVlLizKdxyAsMuzoCD1eFj97BVaA8D/CvKsKu6FOLYkFojN5CbM9h++ZTZtknA=="],
"mysql2": ["mysql2@3.15.3", "", { "dependencies": { "aws-ssl-profiles": "^1.1.1", "denque": "^2.1.0", "generate-function": "^2.3.1", "iconv-lite": "^0.7.0", "long": "^5.2.1", "lru.min": "^1.0.0", "named-placeholders": "^1.1.3", "seq-queue": "^0.0.5", "sqlstring": "^2.3.2" } }, "sha512-FBrGau0IXmuqg4haEZRBfHNWB5mUARw6hNwPDXXGg0XzVJ50mr/9hb267lvpVMnhZ1FON3qNd4Xfcez1rbFwSg=="],
"named-placeholders": ["named-placeholders@1.1.6", "", { "dependencies": { "lru.min": "^1.1.0" } }, "sha512-Tz09sEL2EEuv5fFowm419c1+a/jSMiBjI9gHxVLrVdbUkkNUUfjsVYs9pVZu5oCon/kmRh9TfLEObFtkVxmY0w=="],
@@ -2695,8 +2690,6 @@
"pkce-challenge": ["pkce-challenge@5.0.1", "", {}, "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ=="],
"pkcs7": ["pkcs7@1.0.4", "", { "dependencies": { "@babel/runtime": "^7.5.5" }, "bin": { "pkcs7": "bin/cli.js" } }, "sha512-afRERtHn54AlwaF2/+LFszyAANTCggGilmcmILUzEjvs3XgFZT+xE6+QWQcAGmu4xajy+Xtj7acLOPdx5/eXWQ=="],
"pkg-types": ["pkg-types@2.3.1", "", { "dependencies": { "confbox": "^0.2.4", "exsolve": "^1.0.8", "pathe": "^2.0.3" } }, "sha512-y+ichcgc2LrADuhLNAx8DFjVfgz91pRxfZdI3UDhxHvcVEZsenLO+7XaU5vOp0u/7V/wZ+plyuQxtrDlZJ+yeg=="],
"playwright": ["playwright@1.61.1", "", { "dependencies": { "playwright-core": "1.61.1" }, "optionalDependencies": { "fsevents": "2.3.2" }, "bin": { "playwright": "cli.js" } }, "sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ=="],
@@ -3101,14 +3094,6 @@
"victory-vendor": ["victory-vendor@37.3.6", "", { "dependencies": { "@types/d3-array": "^3.0.3", "@types/d3-ease": "^3.0.0", "@types/d3-interpolate": "^3.0.1", "@types/d3-scale": "^4.0.2", "@types/d3-shape": "^3.1.0", "@types/d3-time": "^3.0.0", "@types/d3-timer": "^3.0.0", "d3-array": "^3.1.6", "d3-ease": "^3.0.1", "d3-interpolate": "^3.0.1", "d3-scale": "^4.0.2", "d3-shape": "^3.1.0", "d3-time": "^3.0.0", "d3-timer": "^3.0.1" } }, "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ=="],
"video.js": ["video.js@8.24.0", "", { "dependencies": { "@babel/runtime": "^7.28.4", "@videojs/http-streaming": "^3.17.5", "@videojs/vhs-utils": "^4.1.2", "@videojs/xhr": "2.7.0", "aes-decrypter": "^4.0.2", "global": "4.4.0", "m3u8-parser": "^7.2.0", "mpd-parser": "^1.4.0", "mux.js": "^7.0.1", "videojs-contrib-quality-levels": "4.1.0", "videojs-font": "4.2.0", "videojs-vtt.js": "0.15.5" } }, "sha512-ZUuxTlDRkEWhpp2mvGCSFzI0Tc3zcC6IUZbIhBcZ3QW0Ms+IRp/tzYkhqexcMYwrcukRFiWWJmRANsLKvv8Vsg=="],
"videojs-contrib-quality-levels": ["videojs-contrib-quality-levels@4.1.0", "", { "dependencies": { "global": "^4.4.0" }, "peerDependencies": { "video.js": "^8" } }, "sha512-TfrXJJg1Bv4t6TOCMEVMwF/CoS8iENYsWNKip8zfhB5kTcegiFYezEA0eHAJPU64ZC8NQbxQgOwAsYU8VXbOWA=="],
"videojs-font": ["videojs-font@4.2.0", "", {}, "sha512-YPq+wiKoGy2/M7ccjmlvwi58z2xsykkkfNMyIg4xb7EZQQNwB71hcSsB3o75CqQV7/y5lXkXhI/rsGAS7jfEmQ=="],
"videojs-vtt.js": ["videojs-vtt.js@0.15.5", "", { "dependencies": { "global": "^4.3.1" } }, "sha512-yZbBxvA7QMYn15Lr/ZfhhLPrNpI/RmCSCqgIff57GC2gIrV5YfyzLfLyZMj0NnZSAz8syB4N0nHXpZg9MyrMOQ=="],
"vite": ["vite@8.2.2", "", { "dependencies": { "lightningcss": "^1.33.0", "picomatch": "^4.0.5", "postcss": "^8.5.26", "rolldown": "~1.2.4", "tinyglobby": "^0.2.17" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.4.0 || ^0.5.0", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q=="],
"vite-tsconfig-paths": ["vite-tsconfig-paths@6.1.1", "", { "dependencies": { "debug": "^4.1.1", "globrex": "^0.1.2", "tsconfck": "^3.0.3" }, "peerDependencies": { "vite": "*" } }, "sha512-2cihq7zliibCCZ8P9cKJrQBkfgdvcFkOOc3Y02o3GWUDLgqjWsZudaoiuOwO/gzTzy17cS5F7ZPo4bsnS4DGkg=="],
+25
View File
@@ -3,6 +3,31 @@ title: "Changelog"
description: "Recent updates, new features, and improvements in Shumai"
---
<Update label="v0.5.0" description="October 2, 2026">
### Added
* **transcode**: Add configurable target resolution ladders (480p, 720p, 1080p, 1440p, 2160p) for MP4 Multi Resolutions transcoding in team settings, with unified resolution matching and lowest-resolution fallback for both MP4 and HLS
* **transcode**: Add HLS live streaming with fMP4/CMAF adaptive bitrate ladders (480p, 720p, 1080p, 1440p, 2160p), configurable in Transcode Settings, with automatic quality switching, manual resolution selection in the video viewer, seamless rendition switching, and full watermarking support for shared links
* **transcode**: Add basic camera RAW image support for popular camera formats (including Sony ARW, Canon CR2/CR3, Nikon NEF, Adobe DNG, Fujifilm RAF, and Panasonic RW2), extracting high-quality embedded JPEG previews and automatically applying camera orientation for fast thumbnail and proxy generation
### Fixed
- **asset**: Fix an issue where media files remained stuck in a perpetual "Preparing..." state when transcoding failed, by displaying a clear failed status and error details in the file list and file viewer
* **transcode**: Fix an issue where videos or audio-only files containing attached cover art or picture streams could cause transcoding to fail during poster generation or select a single static image instead of the video
* **webui**: Fix inconsistent video resolution labels across download menus, mobile file headers, and the video player by correctly deriving resolutions from the video's short side, including portrait videos and 480p videos
* **trash**: Fix an issue where the "Empty Trash" confirmation dialog would close immediately instead of showing a loading state while files were being permanently deleted
### Changed
* **settings**: Redesign team Video Transcoding settings with dedicated cards for Transcode Policy, Hardware Acceleration, and FFmpeg Threads, clearly explaining MP4 and HLS playback vs download behavior and storage implications
* **settings**: Rename AI Settings group to Team AI Settings in settings navigation
</Update>
<Update label="v0.4.9" description="September 28, 2026">
### Added
+61 -21
View File
@@ -1,31 +1,77 @@
---
title: Media Transcoding Settings
description: Configure video transcode strategies and preview proxy generation.
title: Video Transcoding
description: Configure video transcode policies, MP4 and HLS streaming resolutions, hardware acceleration, and FFmpeg threads.
---
High-resolution camera formats and raw media files cannot be rendered natively in web browsers. Shumai transcodes these assets into web-friendly streaming formats in the background.
High-resolution camera formats and raw media files cannot be rendered natively in web browsers. Shumai transcodes video assets into web-friendly streaming and preview formats in the background.
Team Owners can customize video transcoding behavior from the **Team Settings > Video Transcoding** page.
Regardless of the selected transcoding strategy, Shumai's pipeline will **always** generate at least one lightweight `180p` proxy version of the raw file to ensure fast scrubbing, thumbnail generation, and instant previews.
---
## Video Transcode Strategies
## Transcode Policy
Team Owners can select how the transcoding engine handles uploaded files. Shumai evaluates the original file's dimensions against its standard target resolutions (2160p, 1080p, 720p, 540p, and 360p) to determine the output.
The **Transcode Policy** card configures both MP4 video transcoding and HLS adaptive streaming.
### 1. Best Match
### Playback and Download Behavior
- **Behavior**: Shumai checks the longest dimension (width or height) of the uploaded file. It then finds the highest standard resolution that is equal to or smaller than the original file, preventing unnecessary upscaling. If the original video is exceptionally small, it defaults to a minimum target of `360p`.
- **Output**: Generates exactly two web-optimized files: the universal `180p` proxy and the single calculated "best match" resolution.
- **Use Case**: This is the recommended option. It ensures web compatibility without wasting processing time and storage space on redundant lower-resolution versions.
Shumai uses a dual-format strategy for video delivery:
### 2. Transcode All
- **When HLS is disabled**: Both in-browser video playback and file downloads will use the generated MP4 file(s).
- **When HLS is enabled**: In-browser video playback uses HLS (HTTP Live Streaming) for smooth, adaptive bitrate playback, while file downloads will use the generated MP4 file(s).
- **Behavior**: Shumai calculates the "best match" resolution exactly as it does in the previous strategy. However, instead of stopping at one version, it processes the video into the best match _and_ every standard resolution smaller than it.
- **Output**: Generates a full cascade of files. For example, if the best match for an upload is `1080p`, Shumai will generate the `180p` proxy, `1080p`, `720p`, `540p`, and `360p` versions.
- **Use Case**: Ideal for environments requiring adaptive bitrate streaming, ensuring consistent playback performance across all client devices regardless of varying network conditions.
<Warning>
**Storage Usage Notice**: Enabling HLS generates multi-bitrate segment files (`.ts` / `.m4s`) and
playlist manifests in addition to standard MP4 proxies. This can significantly increase storage
consumption on your storage bucket.
</Warning>
![Transcode Settings](../transcode.webp)
### MP4 Transcoding
Generate web-compatible MP4 files for file downloads and standard video playback. Team Owners can select how MP4 transcodes are generated for uploaded files:
#### 1. Best Match
- **Behavior**: Shumai checks the longest dimension of the uploaded video and generates a single optimal resolution matching or smaller than the source quality, preventing unnecessary upscaling.
- **Output**: Generates the lightweight `180p` proxy and the calculated best-match MP4 file.
- **Use Case**: Recommended default. Minimizes processing time and storage space while ensuring universal playback and download compatibility.
#### 2. Multi Resolutions
- **Behavior**: Select specific target resolution ladders to generate: `480p`, `720p`, `1080p`, `1440p`, and `2160p`. Resolutions are only generated up to the source video resolution (content is never upscaled).
- **Output**: Generates the `180p` proxy plus all selected resolution ladders that do not exceed the source resolution.
- **Use Case**: Ideal when users need to download multiple specific resolutions (e.g. 720p for fast sharing and 1080p for review).
### HLS Streaming
When **HLS Streaming** is toggled on, Shumai produces an adaptive bitrate HLS package for uploaded videos:
- **Resolution Ladders**: Configure which resolutions to generate for HLS streaming (`480p`, `720p`, `1080p`, `1440p`, `2160p`). Ladders are only generated up to the original video resolution.
- **Player Integration**: The web player will automatically stream via HLS when available, dynamically switching bitrates based on client network bandwidth.
![Video Transcoding](../transcode.webp)
---
## Hardware Acceleration
To offload encoding workloads from CPUs to GPUs, Shumai supports hardware-accelerated video encoding:
- **Disabled (Software)**: Uses the standard CPU `libx264` software encoder with Constant Rate Factor (CRF).
- **Auto Detect**: Automatically probes and leverages available GPU/hardware acceleration (NVIDIA NVENC, Intel QuickSync, AMD/Intel VA-API, Apple VideoToolbox, Rockchip RKMPP).
See the [Hardware Transcoding Guide](/configuration/hardware-transcoding) for Docker Compose configuration, GPU passthrough, and driver prerequisites.
---
## FFmpeg Threads
The **FFmpeg Threads** slider controls the maximum number of CPU threads used by FFmpeg during transcoding tasks:
- **Auto (0)**: FFmpeg automatically selects the optimal thread count based on host CPU core availability.
- **Manual (1 - 32)**: Restricts FFmpeg to a fixed number of threads, useful for throttling CPU consumption on shared worker nodes.
---
@@ -33,10 +79,4 @@ Team Owners can select how the transcoding engine handles uploaded files. Shumai
Transcoding is resource-intensive. Depending on your scaling needs, your administrator can run the transcode engine on the main server (Single-Server mode) or delegate tasks to specialized transcoding worker machines (Temporal Worker mode).
Note: In all modes, the processing machine must have the standard system media utilities (`ffmpeg` and `ffprobe`) installed.
---
## Hardware Acceleration
To offload encoding workloads to GPUs (NVIDIA NVENC, Intel QuickSync, AMD/Intel VA-API, or Rockchip RKMPP), Shumai supports hardware-accelerated transcoding. See the [Hardware Transcoding Guide](/configuration/hardware-transcoding) for Docker Compose configuration, GPU passthrough, and driver prerequisites.
In all modes, the processing machine must have system media utilities (`ffmpeg` and `ffprobe`) installed. See the [Temporal Deployment Guide](/configuration/temporal) for setting up distributed transcoding workers.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"repository": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
+4
View File
@@ -154,6 +154,10 @@ export async function generateEmbeddingActivity(params: GenerateEmbeddingParams)
start.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-map',
'0:a:0?',
'-t',
(end - start).toString(),
'-c',
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/api",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
+43
View File
@@ -774,4 +774,47 @@ describe('file api', () => {
mockGetProjectIds.mockRestore()
})
describe('GET /files/:fileId/m3u8 routes', () => {
it('returns master playlist with correct content-type', async () => {
const mockMaster = vi
.spyOn(assetService, 'getHlsMasterPlaylist')
.mockResolvedValue('#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n')
const app = new Hono().use('*', authMiddleware).route('/', fileRoute)
const res = await app.request('/files/asset-1/m3u8/master.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('#EXTM3U')
expect(text).toContain('1080p/index.m3u8')
expect(mockMaster).toHaveBeenCalledWith({ assetId: 'asset-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user1', name: 'Test User' },
permission: Permission.Read,
type: 'asset',
id: 'asset-1',
})
mockMaster.mockRestore()
})
it('returns variant playlist with rewritten URLs', async () => {
const mockVariant = vi
.spyOn(assetService, 'getHlsVariantPlaylist')
.mockResolvedValue(
'#EXTM3U\n#EXT-X-MAP:URI="http://presigned/init.mp4"\nhttp://presigned/seg0.m4s\n',
)
const app = new Hono().use('*', authMiddleware).route('/', fileRoute)
const res = await app.request('/files/asset-1/m3u8/1080p/index.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('http://presigned/init.mp4')
expect(mockVariant).toHaveBeenCalledWith({ assetId: 'asset-1', resolution: '1080p' })
mockVariant.mockRestore()
})
})
})
+65
View File
@@ -480,5 +480,70 @@ const route = new Hono<{ Variables: { user: User } }>()
return c.json({ url })
},
)
.get('/files/:fileId/m3u8', async (c) => {
const fileId = c.req.param('fileId')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsMasterPlaylist({ assetId: fileId })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/master.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsMasterPlaylist({ assetId: fileId })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/:resolution', async (c) => {
const fileId = c.req.param('fileId')
let resolution = c.req.param('resolution')
if (resolution.endsWith('.m3u8')) {
resolution = resolution.replace(/\.m3u8$/, '')
}
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsVariantPlaylist({ assetId: fileId, resolution })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/:resolution/index.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const resolution = c.req.param('resolution')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsVariantPlaylist({ assetId: fileId, resolution })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
export default route
+129 -6
View File
@@ -38,6 +38,8 @@ import type { AssetInfo } from '@shumai/dtos'
async function applyWatermarkToAssetMedia(
asset: AssetInfo,
watermarkMedia: PrismaJson.MediaInfo | null,
shareId?: string,
password?: string,
): Promise<AssetInfo> {
if (!asset.media) return asset
if (asset.proxyType !== 'video' && asset.proxyType !== 'image') return asset
@@ -74,6 +76,19 @@ async function applyWatermarkToAssetMedia(
)
: []
if (watermarkMedia?.isHls && watermarkMedia.hls?.key && shareId) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
updatedMedia.isHls = true
updatedMedia.hls = {
key: watermarkMedia.hls.key,
url: `/api/shares/${shareId}/files/${asset.id}/m3u8/master.m3u8${pwQuery}`,
resolutions: watermarkMedia.hls.resolutions,
}
} else {
updatedMedia.isHls = false
delete updatedMedia.hls
}
return {
...asset,
media: updatedMedia,
@@ -170,7 +185,8 @@ const route = app
zValidator('query', sharedChildrenRequestSchema),
async (c) => {
const folderId = c.req.param('folderId')
const password = c.req.header('x-share-password')
const password =
c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
const req = c.req.valid('query')
try {
@@ -197,7 +213,7 @@ const route = app
// Resolve symlink targets once and reuse for the media lookup, so each
// item costs a single query instead of two.
const targetIds = await Promise.all(
res.data.map((item) => assetService.resolveTargetAssetId(item.id)),
res.data.map((item) => assetService.resolveLatestVersionId(item.id)),
)
const wfMap = await watermarkService.getCompletedWatermarkMediaMap(
targetIds,
@@ -206,9 +222,21 @@ const route = app
res.data = await Promise.all(
res.data.map(async (item, index) =>
applyWatermarkToAssetMedia(item, wfMap.get(targetIds[index]) ?? null),
applyWatermarkToAssetMedia(
item,
wfMap.get(targetIds[index]) ?? null,
shareLink.id,
password,
),
),
)
} else if (res.data.length > 0) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
for (const item of res.data) {
if (item.media?.isHls && item.media.hls?.key) {
item.media.hls.url = `/api/shares/${shareLink.id}/files/${item.id}/m3u8/master.m3u8${pwQuery}`
}
}
}
return c.json(res)
@@ -219,7 +247,7 @@ const route = app
)
.get('/shares/:shareId/files/:fileId', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
@@ -232,12 +260,20 @@ const route = app
shareLink.watermarkConfigId &&
(asset.proxyType === 'video' || asset.proxyType === 'image')
) {
const targetAssetId = await assetService.resolveTargetAssetId(fileId)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const wfMap = await watermarkService.getCompletedWatermarkMediaMap(
[targetAssetId],
shareLink.watermarkConfigId,
)
asset = await applyWatermarkToAssetMedia(asset, wfMap.get(targetAssetId) ?? null)
asset = await applyWatermarkToAssetMedia(
asset,
wfMap.get(targetAssetId) ?? null,
shareLink.id,
password,
)
} else if (asset.media?.isHls && asset.media.hls?.key) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
asset.media.hls.url = `/api/shares/${shareLink.id}/files/${asset.id}/m3u8/master.m3u8${pwQuery}`
}
return c.json(asset)
@@ -360,5 +396,92 @@ const route = app
}
},
)
.get('/shares/:shareId/files/:fileId/m3u8', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
let playlist = await assetService.getHlsMasterPlaylist({
assetId: targetAssetId,
watermarkConfigId: shareLink.watermarkConfigId,
})
if (password) {
playlist = playlist.replace(
/([0-9]+p\/index\.m3u8)/g,
`$1?p=${encodeURIComponent(password)}`,
)
}
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/master.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
let playlist = await assetService.getHlsMasterPlaylist({
assetId: targetAssetId,
watermarkConfigId: shareLink.watermarkConfigId,
})
if (password) {
playlist = playlist.replace(
/([0-9]+p\/index\.m3u8)/g,
`$1?p=${encodeURIComponent(password)}`,
)
}
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/:resolution', async (c) => {
const fileId = c.req.param('fileId')
let resolution = c.req.param('resolution')
if (resolution.endsWith('.m3u8')) {
resolution = resolution.replace(/\.m3u8$/, '')
}
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const playlist = await assetService.getHlsVariantPlaylist({
assetId: targetAssetId,
resolution,
watermarkConfigId: shareLink.watermarkConfigId,
})
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/:resolution/index.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const resolution = c.req.param('resolution')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const playlist = await assetService.getHlsVariantPlaylist({
assetId: targetAssetId,
resolution,
watermarkConfigId: shareLink.watermarkConfigId,
})
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
export default route
+113 -1
View File
@@ -277,6 +277,12 @@ describe('Share API', () => {
},
],
imageTranscodes: [],
isHls: true,
hls: {
key: 'files/orig-hls/master.m3u8',
url: '/api/files/video-pending/m3u8/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
original: { key: 'files/orig.mp4' },
},
},
@@ -331,9 +337,11 @@ describe('Share API', () => {
size: 0,
},
])
// Pending item → original transcodes emptied
// Pending item → original transcodes emptied and HLS cleared
expect(body.data[1].media.videoTranscodes).toEqual([])
expect(body.data[1].media.imageTranscodes).toEqual([])
expect(body.data[1].media.isHls).toBe(false)
expect(body.data[1].media.hls).toBeUndefined()
// Non-media item → untouched
expect(body.data[2].media.pdfTranscode).toEqual({ url: 'u', key: 'files/doc.pdf' })
})
@@ -478,6 +486,33 @@ describe('Share API', () => {
expect(body.media.imageTranscodes).toEqual([])
})
test('clears HLS fields while watermark transcoding is in flight', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue(shareLinkWithWatermark)
vi.spyOn(watermarkService, 'getCompletedWatermarkMediaMap').mockResolvedValue(new Map())
vi.spyOn(assetService, 'getAsset').mockResolvedValue({
...videoAsset,
media: {
...videoAsset.media,
isHls: true,
hls: {
key: 'files/hls/master.m3u8',
url: '/api/files/file1/m3u8/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
},
})
const res = await app.request('/shares/share1/files/file1', {
method: 'GET',
headers: { 'x-share-password': 'pass' },
})
expect(res.status).toBe(200)
const body = await res.json()
expect(body.media.isHls).toBe(false)
expect(body.media.hls).toBeUndefined()
})
test('leaves non-video/image media untouched when watermark enabled', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue(shareLinkWithWatermark)
vi.spyOn(assetService, 'getAsset').mockResolvedValue({
@@ -866,4 +901,81 @@ describe('Share API', () => {
expect(getDownloadUrlSpy).not.toHaveBeenCalled()
})
})
describe('GET /shares/:shareId/files/:fileId/m3u8 routes', () => {
test('returns master playlist with query password preserved in variant URLs', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: null,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'getHlsMasterPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n',
)
const res = await app.request('/shares/share1/files/file1/m3u8/master.m3u8?p=secret123')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('1080p/index.m3u8?p=secret123')
expect(shareService.verifyPublicAccess).toHaveBeenCalledWith('file1', 'secret123')
expect(assetService.getHlsMasterPlaylist).toHaveBeenCalledWith({
assetId: 'file1',
watermarkConfigId: null,
})
})
test('returns variant playlist with rewritten segment URLs', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: 'wm1',
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'getHlsVariantPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-MAP:URI="http://presigned/init.mp4"\nhttp://presigned/seg0.m4s\n',
)
const res = await app.request('/shares/share1/files/file1/m3u8/1080p/index.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('http://presigned/init.mp4')
expect(assetService.getHlsVariantPlaylist).toHaveBeenCalledWith({
assetId: 'file1',
resolution: '1080p',
watermarkConfigId: 'wm1',
})
})
test('returns 401 when password is invalid', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockRejectedValue(
new ShareLinkPasswordInvalidError('Invalid password'),
)
const res = await app.request('/shares/share1/files/file1/m3u8/master.m3u8?p=wrong')
expect(res.status).toBe(401)
const body = await res.json()
expect(body.error).toBe('Unauthorized')
})
test('resolves latest version id when fileId is a symlink to a version stack', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: null,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'resolveLatestVersionId').mockResolvedValue('child-version-file-id')
vi.spyOn(assetService, 'getHlsMasterPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n',
)
const res = await app.request('/shares/share1/files/symlink1/m3u8/master.m3u8')
expect(res.status).toBe(200)
expect(assetService.resolveLatestVersionId).toHaveBeenCalledWith('symlink1')
expect(assetService.getHlsMasterPlaylist).toHaveBeenCalledWith({
assetId: 'child-version-file-id',
watermarkConfigId: null,
})
})
})
})
+27 -4
View File
@@ -245,17 +245,17 @@ describe('team api', () => {
})
it('PATCH /teams/:teamId/settings updates settings', async () => {
mockUpdateSettings.mockResolvedValue({ transcode: { videoStrategy: 'all' } })
mockUpdateSettings.mockResolvedValue({ transcode: { videoStrategy: 'multi' } })
const res = await app.request('/teams/t1/settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ key: 'transcode.videoStrategy', value: 'all' }),
body: JSON.stringify({ key: 'transcode.videoStrategy', value: 'multi' }),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.transcode.videoStrategy).toBe('all')
expect(data.transcode.videoStrategy).toBe('multi')
expect(authzService.hasPermission).toHaveBeenCalledWith(
expect.objectContaining({
type: ResourceType.Team,
@@ -263,7 +263,30 @@ describe('team api', () => {
permission: Permission.Admin,
}),
)
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoStrategy', 'all')
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoStrategy', 'multi')
})
it('PATCH /teams/:teamId/settings updates transcode.videoResolutions', async () => {
mockUpdateSettings.mockResolvedValue({
transcode: { videoResolutions: ['720p', '1080p'] },
})
const res = await app.request('/teams/t1/settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
key: 'transcode.videoResolutions',
value: ['720p', '1080p'],
}),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.transcode.videoResolutions).toEqual(['720p', '1080p'])
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoResolutions', [
'720p',
'1080p',
])
})
it('PATCH /teams/:teamId/settings updates transcode.hardwareAcceleration', async () => {
+2 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/core",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -42,6 +42,7 @@
"date-fns": "4.4.0",
"dotenv": "17.4.2",
"dotenv-expand": "13.0.0",
"exiftool-vendored": "38.1.0",
"extract-zip": "2.0.1",
"hono": "4.13.5",
"jittered-fractional-indexing": "1.0.1",
+108
View File
@@ -16,6 +16,7 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
presign: vi.fn().mockResolvedValue('http://mock-s3-url'),
putObject: vi.fn().mockResolvedValue(undefined),
getObject: vi.fn().mockResolvedValue({ buffer: Buffer.from('') }),
deleteObject: vi.fn().mockResolvedValue(1),
deletePrefix: vi.fn().mockResolvedValue(1),
copyObject: vi.fn().mockResolvedValue(undefined),
@@ -5251,4 +5252,111 @@ describe('AssetService — natural sort by name', () => {
expect(restored?.status).toBe(AssetStatus.processed)
})
})
describe('HLS playlists', () => {
const setupHlsAssets = async () => {
const team = await prisma.team.create({ data: { name: 'hls-team-' + Date.now() } })
const project = await prisma.project.create({
data: { name: 'hls-proj-' + Date.now(), teamId: team.id },
})
const rootFolder = await prisma.asset.create({
data: {
name: 'root',
type: AssetType.folder,
projectId: project.id,
status: AssetStatus.processed,
},
})
return { team, project, rootFolder }
}
it('getHlsMasterPlaylist throws 404 if asset is not HLS', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'regular.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: false,
} as PrismaJson.MediaInfo,
},
})
await expect(assetService.getHlsMasterPlaylist({ assetId: file.id })).rejects.toThrow(
'HLS playlist not found',
)
})
it('getHlsMasterPlaylist returns playlist string when asset is HLS', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'hls-video.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [
{ resolution: '1080p', width: 1920, height: 1080 },
{ resolution: '720p', width: 1280, height: 720 },
],
},
} as PrismaJson.MediaInfo,
},
})
const masterContent = '#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n'
vi.mocked(s3Service.getObject).mockResolvedValueOnce({
buffer: Buffer.from(masterContent),
} as unknown as Awaited<ReturnType<typeof s3Service.getObject>>)
const res = await assetService.getHlsMasterPlaylist({ assetId: file.id })
expect(res).toBe(masterContent)
expect(s3Service.getObject).toHaveBeenCalledWith(
expect.any(String),
'files/asset/hls/master.m3u8',
)
})
it('getHlsVariantPlaylist fetches and rewrites segment URLs', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'hls-video.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
} as PrismaJson.MediaInfo,
},
})
const sampleVariant = '#EXTM3U\n#EXT-X-MAP:URI="init.mp4"\nsegment_000.m4s\n'
vi.mocked(s3Service.getObject).mockResolvedValueOnce({
buffer: Buffer.from(sampleVariant),
} as unknown as Awaited<ReturnType<typeof s3Service.getObject>>)
vi.mocked(s3Service.presign).mockResolvedValue('http://mock-s3-url')
const variant = await assetService.getHlsVariantPlaylist({
assetId: file.id,
resolution: '1080p',
})
expect(variant).toBeDefined()
expect(variant).toContain('#EXT-X-MAP:URI="http://mock-s3-url"')
expect(variant).toContain('http://mock-s3-url')
})
})
})
+93
View File
@@ -57,6 +57,7 @@ import { metadataService } from '@shumai/core/src/metadata/metadata'
import { uploadService } from '@shumai/core/src/upload/upload'
import path from 'path'
import { stemFromKey } from '@shumai/core/src/utils/filename'
import { rewriteM3u8WithPresignedUrls } from '@shumai/core/src/transcode/transcode'
export const assetInclude = {
creator: true,
@@ -2386,6 +2387,9 @@ export class AssetService {
'GET',
)
}
if (media && media.isHls && media.hls?.key) {
media.hls.url = `/api/files/${latestVersion.id}/m3u8/master.m3u8`
}
const key = latestVersion.storageKey?.key
if (key) {
@@ -3196,6 +3200,95 @@ export class AssetService {
req,
)
}
async getHlsMasterPlaylist(params: {
assetId: string
watermarkConfigId?: string | null
}): Promise<string> {
const bucket = process.env.S3_BUCKET || 'shumai'
const targetAssetId = await this.resolveLatestVersionId(params.assetId)
if (params.watermarkConfigId) {
const wf = await this.prismaClient.watermarkFile.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_watermarkConfigId: {
assetId: targetAssetId,
watermarkConfigId: params.watermarkConfigId,
},
},
})
const media = wf?.media as PrismaJson.MediaInfo | null
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const obj = await s3Service.getObject(bucket, media.hls.key)
return obj.buffer.toString('utf-8')
}
const asset = await this.prismaClient.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new HTTPException(404, { message: 'Asset not found' })
}
const media = asset.media as PrismaJson.MediaInfo | null
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const obj = await s3Service.getObject(bucket, media.hls.key)
return obj.buffer.toString('utf-8')
}
async getHlsVariantPlaylist(params: {
assetId: string
resolution: string
watermarkConfigId?: string | null
}): Promise<string> {
const bucket = process.env.S3_BUCKET || 'shumai'
if (!/^\d+p$/.test(params.resolution)) {
throw new HTTPException(400, { message: 'Invalid resolution format' })
}
const targetAssetId = await this.resolveLatestVersionId(params.assetId)
let media: PrismaJson.MediaInfo | null
if (params.watermarkConfigId) {
const wf = await this.prismaClient.watermarkFile.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_watermarkConfigId: {
assetId: targetAssetId,
watermarkConfigId: params.watermarkConfigId,
},
},
})
media = (wf?.media as PrismaJson.MediaInfo | null) ?? null
} else {
const asset = await this.prismaClient.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new HTTPException(404, { message: 'Asset not found' })
}
media = (asset.media as PrismaJson.MediaInfo | null) ?? null
}
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const matchedRes = media.hls.resolutions?.find((r) => r.resolution === params.resolution)
const variantKey =
matchedRes?.key ||
path.posix.join(path.posix.dirname(media.hls.key), params.resolution, 'index.m3u8')
const obj = await s3Service.getObject(bucket, variantKey)
const rawM3u8 = obj.buffer.toString('utf-8')
const s3Prefix = path.posix.dirname(variantKey)
return await rewriteM3u8WithPresignedUrls(rawM3u8, s3Prefix, bucket)
}
}
export const assetService = new AssetService()
+18 -2
View File
@@ -369,8 +369,11 @@ describe('TeamService', () => {
semanticSearchEnabled: false,
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
},
appearance: { hideAgent: false },
})
@@ -383,24 +386,37 @@ describe('TeamService', () => {
semanticSearchEnabled: false,
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
},
appearance: { hideAgent: false },
})
await teamService.updateSettings(team.id, 'transcode.videoStrategy', 'all')
await teamService.updateSettings(team.id, 'transcode.videoStrategy', 'multi')
await teamService.updateSettings(team.id, 'transcode.videoResolutions', ['720p', '1080p'])
await teamService.updateSettings(team.id, 'transcode.hardwareAcceleration', 'auto')
await teamService.updateSettings(team.id, 'transcode.threads', 4)
await teamService.updateSettings(team.id, 'transcode.hlsEnabled', true)
await teamService.updateSettings(team.id, 'transcode.hlsResolutions', [
'720p',
'1080p',
'2160p',
])
await teamService.updateSettings(team.id, 'appearance.hideAgent', true)
const finalSettings = await teamService.getSettings(team.id)
expect(finalSettings).toEqual({
theme: 'dark',
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 4,
hlsEnabled: true,
hlsResolutions: ['720p', '1080p', '2160p'],
},
appearance: { hideAgent: true },
semanticSearchEnabled: false,
+50 -1
View File
@@ -35,6 +35,7 @@ export class TeamService {
settings: {
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
},
@@ -312,8 +313,27 @@ export class TeamService {
if (!settings.transcode) {
settings.transcode = {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
}
} else {
const strategyStr = settings.transcode.videoStrategy as string | undefined
if (strategyStr === 'single' || strategyStr === 'disable') {
settings.transcode.videoStrategy = 'best_match'
} else if (strategyStr === 'full' || strategyStr === 'all') {
settings.transcode.videoStrategy = 'multi'
}
if (!settings.transcode.videoResolutions) {
settings.transcode.videoResolutions = ['480p', '720p', '1080p', '1440p', '2160p']
}
if (settings.transcode.hlsEnabled === undefined) {
settings.transcode.hlsEnabled = false
}
if (!settings.transcode.hlsResolutions) {
settings.transcode.hlsResolutions = ['480p', '720p', '1080p']
}
}
@@ -336,10 +356,25 @@ export class TeamService {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.videoStrategy = value as NonNullable<
const rawStrategy = value as string
let strategy = rawStrategy
if (rawStrategy === 'single' || rawStrategy === 'disable') {
strategy = 'best_match'
} else if (rawStrategy === 'full' || rawStrategy === 'all') {
strategy = 'multi'
}
settings.transcode.videoStrategy = strategy as NonNullable<
TeamSettingsResponse['transcode']
>['videoStrategy']
delete settings['transcode.videoStrategy']
} else if (key === 'transcode.videoResolutions') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.videoResolutions = value as NonNullable<
TeamSettingsResponse['transcode']
>['videoResolutions']
delete settings['transcode.videoResolutions']
} else if (key === 'transcode.hardwareAcceleration') {
if (!settings.transcode) {
settings.transcode = {}
@@ -354,6 +389,20 @@ export class TeamService {
}
settings.transcode.threads = Number(value)
delete settings['transcode.threads']
} else if (key === 'transcode.hlsEnabled') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.hlsEnabled = Boolean(value)
delete settings['transcode.hlsEnabled']
} else if (key === 'transcode.hlsResolutions') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.hlsResolutions = value as NonNullable<
TeamSettingsResponse['transcode']
>['hlsResolutions']
delete settings['transcode.hlsResolutions']
} else if (key === 'appearance.hideAgent') {
if (!settings.appearance) {
settings.appearance = {}
@@ -0,0 +1,283 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import * as fs from 'fs'
import {
extractEmbeddedJpeg,
validateExtractedJpeg,
extractAndValidateRawPreview,
EXIF_ORIENTATION_TO_ROTATION,
} from './raw-extract'
// Mock exiftool-vendored
vi.mock('exiftool-vendored', () => ({
exiftool: {
extractBinaryTag: vi.fn(),
read: vi.fn(),
},
}))
// Mock sharp
vi.mock('sharp', () => {
const mockMetadata = vi.fn()
const mockSharp = vi.fn(() => ({
metadata: mockMetadata,
}))
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(mockSharp as any)._mockMetadata = mockMetadata
return { default: mockSharp }
})
import { exiftool } from 'exiftool-vendored'
import sharp from 'sharp'
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mockMetadata = (sharp as any)._mockMetadata as ReturnType<typeof vi.fn>
const mockExtractBinaryTag = vi.mocked(exiftool.extractBinaryTag)
const mockRead = vi.mocked(exiftool.read)
describe('extractEmbeddedJpeg', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('extracts from JpgFromRaw2 first to a temporary file', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 1 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(1)
expect(mockExtractBinaryTag).toHaveBeenCalledWith(
'JpgFromRaw2',
'/path/to/photo.cr2',
expect.stringContaining('preview-JpgFromRaw2'),
)
// Test cleanup
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to JpgFromRaw when JpgFromRaw2 fails', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 6 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.nef')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(6)
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(2)
expect(mockExtractBinaryTag).toHaveBeenNthCalledWith(
2,
'JpgFromRaw',
'/path/to/photo.nef',
expect.stringContaining('preview-JpgFromRaw'),
)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to PreviewImage when JpgFromRaw2 and JpgFromRaw fail', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'preview-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 3 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.arw')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(3)
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(3)
expect(mockExtractBinaryTag).toHaveBeenNthCalledWith(
3,
'PreviewImage',
'/path/to/photo.arw',
expect.stringContaining('preview-PreviewImage'),
)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('returns null and cleans up when all tags fail', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).toBeNull()
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(3)
})
it('returns undefined orientation when EXIF read fails', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockRejectedValueOnce(new Error('EXIF read failed'))
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
it('returns undefined orientation when orientation value is out of range', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 9 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
it('returns undefined orientation when orientation is not a number', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 'Horizontal' } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
})
describe('validateExtractedJpeg', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('returns dimensions for a valid JPEG file', async () => {
mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 })
const result = await validateExtractedJpeg('/tmp/valid-jpeg.jpg')
expect(result).toEqual({ width: 6000, height: 4000 })
})
it('returns null when metadata has zero width', async () => {
mockMetadata.mockResolvedValueOnce({ width: 0, height: 4000 })
const result = await validateExtractedJpeg('/tmp/bad-jpeg.jpg')
expect(result).toBeNull()
})
it('returns null when metadata has no dimensions', async () => {
mockMetadata.mockResolvedValueOnce({})
const result = await validateExtractedJpeg('/tmp/bad-jpeg.jpg')
expect(result).toBeNull()
})
it('returns null when sharp throws', async () => {
mockMetadata.mockRejectedValueOnce(new Error('Invalid image'))
const result = await validateExtractedJpeg('/tmp/corrupt-jpeg.jpg')
expect(result).toBeNull()
})
})
describe('extractAndValidateRawPreview', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('returns previewPath, dimensions, orientation, and cleanup for valid RAW', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 6 } as never)
mockMetadata.mockResolvedValueOnce({ width: 4000, height: 3000 })
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.width).toBe(4000)
expect(result!.height).toBe(3000)
expect(result!.orientation).toBe(6)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('returns null when extraction fails', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).toBeNull()
})
it('returns null and cleans up when validation fails', async () => {
let createdPath = ''
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
createdPath = dest
fs.writeFileSync(dest, 'bad-jpeg')
})
mockRead.mockResolvedValueOnce({ Orientation: 1 } as never)
mockMetadata.mockRejectedValueOnce(new Error('Not a valid image'))
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).toBeNull()
expect(fs.existsSync(createdPath)).toBe(false)
})
})
describe('EXIF_ORIENTATION_TO_ROTATION', () => {
it('maps all 8 EXIF orientations', () => {
expect(Object.keys(EXIF_ORIENTATION_TO_ROTATION)).toHaveLength(8)
for (let i = 1; i <= 8; i++) {
expect(EXIF_ORIENTATION_TO_ROTATION[i]).toBeDefined()
}
})
it('has correct rotation for landscape (1)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[1]).toEqual({})
})
it('has correct rotation for 90° CW (6)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[6]).toEqual({ angle: 90 })
})
it('has correct rotation for 180° (3)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[3]).toEqual({ angle: 180 })
})
it('has correct rotation for 270° CW (8)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[8]).toEqual({ angle: 270 })
})
it('has correct rotation for mirror horizontal (2)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[2]).toEqual({ flop: true })
})
})
+153
View File
@@ -0,0 +1,153 @@
import { exiftool } from 'exiftool-vendored'
import sharp from 'sharp'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import { logger } from '@shumai/core/src/logger'
export interface RawExtractResult {
previewPath: string
cleanup: () => void
/** EXIF orientation from the RAW container (1–8), or undefined if absent. */
orientation?: number
}
/**
* Maps EXIF orientation values (1–8) to Sharp rotation operations.
* Follows the same mapping as Immich's ORIENTATION_TO_SHARP_ROTATION.
*/
export const EXIF_ORIENTATION_TO_ROTATION: Record<
number,
{ angle?: number; flip?: boolean; flop?: boolean }
> = {
1: {}, // Horizontal (normal)
2: { flop: true }, // Mirror horizontal
3: { angle: 180 }, // Rotate 180°
4: { angle: 180, flop: true }, // Mirror vertical
5: { angle: 270, flip: true }, // Mirror horizontal + rotate 270° CW
6: { angle: 90 }, // Rotate 90° CW
7: { angle: 90, flip: true }, // Mirror horizontal + rotate 90° CW
8: { angle: 270 }, // Rotate 270° CW
}
/**
* Attempt to extract an embedded JPEG preview from a camera RAW file to a temporary file.
*
* Tries tags in priority order (matching Immich, skipping PreviewJXL):
* 1. JpgFromRaw2 — secondary full-resolution JPEG (newer Sony/Canon/Nikon)
* 2. JpgFromRaw — primary full-size embedded JPEG
* 3. PreviewImage — smaller embedded preview fallback
*
* Also reads the EXIF orientation from the RAW container so callers can
* apply it to the extracted buffer/file (which often lacks its own orientation tag).
*
* Returns null if no usable embedded JPEG exists.
*/
export async function extractEmbeddedJpeg(
rawFilePath: string,
targetDir?: string,
): Promise<RawExtractResult | null> {
const tags = ['JpgFromRaw2', 'JpgFromRaw', 'PreviewImage'] as const
const ownTempDir = !targetDir
const workDir = targetDir ?? fs.mkdtempSync(path.join(os.tmpdir(), 'raw-extract-'))
const cleanup = () => {
try {
if (ownTempDir && fs.existsSync(workDir)) {
fs.rmSync(workDir, { recursive: true, force: true })
}
} catch (err) {
logger.warn({ workDir, err }, 'Failed to clean up raw extraction temporary directory')
}
}
for (const tag of tags) {
const candidatePath = path.join(workDir, `preview-${tag}-${Date.now()}.jpg`)
try {
await exiftool.extractBinaryTag(tag, rawFilePath, candidatePath)
if (fs.existsSync(candidatePath) && fs.statSync(candidatePath).size > 0) {
logger.debug({ tag, rawFilePath, candidatePath }, 'Extracted embedded JPEG from RAW')
// Read orientation from the RAW container's EXIF metadata.
// The extracted JPEG binary stream often does NOT carry its own
// orientation tag, so we must read it from the parent RAW file
// to avoid incorrectly-rotated previews.
let orientation: number | undefined
try {
const exif = await exiftool.read(rawFilePath)
const rawOrientation = exif.Orientation
if (typeof rawOrientation === 'number' && rawOrientation >= 1 && rawOrientation <= 8) {
orientation = rawOrientation
}
} catch {
logger.debug({ rawFilePath }, 'Could not read EXIF orientation from RAW container')
}
return { previewPath: candidatePath, cleanup, orientation }
}
} catch {
logger.debug({ tag, rawFilePath }, `Could not extract ${tag} from RAW file`)
try {
if (fs.existsSync(candidatePath)) {
fs.rmSync(candidatePath, { force: true })
}
} catch {
// ignore
}
}
}
cleanup()
logger.info({ rawFilePath }, 'No usable embedded JPEG found in RAW file')
return null
}
/**
* Validate that an extracted JPEG file is decodable.
* Returns dimensions if valid, null if corrupt/unusable.
*/
export async function validateExtractedJpeg(
filePath: string,
): Promise<{ width: number; height: number } | null> {
try {
const metadata = await sharp(filePath).metadata()
if (metadata.width && metadata.height && metadata.width > 0 && metadata.height > 0) {
return { width: metadata.width, height: metadata.height }
}
return null
} catch {
return null
}
}
/**
* Extract and validate an embedded JPEG from a RAW file to a temporary file.
* Returns a validated file path + dimensions + orientation + cleanup function, or null.
*/
export async function extractAndValidateRawPreview(
rawFilePath: string,
targetDir?: string,
): Promise<{
previewPath: string
cleanup: () => void
width: number
height: number
orientation?: number
} | null> {
const extracted = await extractEmbeddedJpeg(rawFilePath, targetDir)
if (!extracted) return null
const dims = await validateExtractedJpeg(extracted.previewPath)
if (!dims) {
logger.warn({ rawFilePath }, 'Extracted JPEG from RAW is not decodable, skipping preview')
extracted.cleanup()
return null
}
return {
previewPath: extracted.previewPath,
cleanup: extracted.cleanup,
orientation: extracted.orientation,
...dims,
}
}
+399 -1
View File
@@ -10,6 +10,11 @@ import {
buildSdrToneMapFilterChain,
getVaapiDevice,
parseBitrateKbps,
buildHlsMasterPlaylist,
rewriteM3u8WithPresignedUrls,
compareStreams,
selectPrimaryVideoStream,
selectPrimaryAudioStream,
} from './transcode'
import { logger } from '@shumai/core/src/logger'
import { s3Service } from '@shumai/core/src/s3/s3'
@@ -20,12 +25,16 @@ import * as fs from 'fs'
import { WorkflowTask } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import sharp from 'sharp'
import * as rawExtract from './raw-extract'
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
downloadToFile: vi.fn(),
putObject: vi.fn(),
resolveInput: vi.fn().mockImplementation(async (_bucket, key) => `http://mock-storage/${key}`),
presign: vi
.fn()
.mockImplementation(async (_bucket, key) => `https://presigned.example.com/${key}`),
},
}))
@@ -40,6 +49,9 @@ vi.mock('sharp', () => {
resize: vi.fn().mockReturnThis(),
toColorspace: vi.fn().mockReturnThis(),
webp: vi.fn().mockReturnThis(),
rotate: vi.fn().mockReturnThis(),
flip: vi.fn().mockReturnThis(),
flop: vi.fn().mockReturnThis(),
composite: vi.fn().mockReturnThis(),
png: vi.fn().mockReturnThis(),
toBuffer: vi.fn().mockResolvedValue(Buffer.from('fake-webp-buffer')),
@@ -448,6 +460,103 @@ describe('TranscodeService', () => {
expect(info.mimeType).toBe('psd')
})
it('should extract embedded preview for RAW image in getImageInfo', async () => {
const rawPath = path.join(tempDir, 'photo.cr2')
const cleanupSpy = vi.fn()
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({
previewPath: '/tmp/preview.jpg',
cleanup: cleanupSpy,
width: 6000,
height: 4000,
orientation: 1,
})
const info = await transcodeService.getImageInfo(rawPath)
expect(info.originalWidth).toBe(6000)
expect(info.originalHeight).toBe(4000)
expect(info.mimeType).toBe('jpeg')
expect(cleanupSpy).toHaveBeenCalled()
})
it('should swap dimensions in getImageInfo when RAW orientation is portrait (orientation 6)', async () => {
const rawPath = path.join(tempDir, 'portrait.dng')
const cleanupSpy = vi.fn()
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({
previewPath: '/tmp/preview.jpg',
cleanup: cleanupSpy,
width: 6000,
height: 4000,
orientation: 6,
})
const info = await transcodeService.getImageInfo(rawPath)
expect(info.originalWidth).toBe(4000)
expect(info.originalHeight).toBe(6000)
expect(info.mimeType).toBe('jpeg')
expect(cleanupSpy).toHaveBeenCalled()
})
it('should return zero dimensions in getImageInfo when RAW has no usable preview', async () => {
const rawPath = path.join(tempDir, 'nopreview.arw')
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce(null)
const info = await transcodeService.getImageInfo(rawPath)
expect(info.originalWidth).toBe(0)
expect(info.originalHeight).toBe(0)
expect(info.mimeType).toBe('')
})
it('should transcode RAW image by extracting preview to temp file and feeding to sharp', async () => {
const rawPath = path.join(tempDir, 'sample.nef')
const outputFile = path.join(tempDir, 'output-raw.webp')
const cleanupSpy = vi.fn()
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({
previewPath: '/tmp/extracted-raw.jpg',
cleanup: cleanupSpy,
width: 3000,
height: 2000,
orientation: 1,
})
await transcodeService.transcodeImage(rawPath, outputFile, 1920, 85)
expect(sharp).toHaveBeenCalledWith('/tmp/extracted-raw.jpg', { limitInputPixels: false })
const mockSharp = vi.mocked(sharp).mock.results[vi.mocked(sharp).mock.results.length - 1].value
expect(mockSharp.toColorspace).toHaveBeenCalledWith('srgb')
expect(mockSharp.webp).toHaveBeenCalledWith({ quality: 85 })
expect(mockSharp.toFile).toHaveBeenCalledWith(outputFile)
expect(cleanupSpy).toHaveBeenCalled()
})
it('should apply orientation rotation when transcoding RAW image', async () => {
const rawPath = path.join(tempDir, 'rotated.cr3')
const outputFile = path.join(tempDir, 'output-rotated.webp')
const cleanupSpy = vi.fn()
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({
previewPath: '/tmp/extracted-rotated.jpg',
cleanup: cleanupSpy,
width: 4000,
height: 3000,
orientation: 6, // 90° CW
})
await transcodeService.transcodeImage(rawPath, outputFile, 1920, 85)
const mockSharp = vi.mocked(sharp).mock.results[vi.mocked(sharp).mock.results.length - 1].value
expect(mockSharp.rotate).toHaveBeenCalledWith(90)
expect(cleanupSpy).toHaveBeenCalled()
})
it('should throw error when transcoding RAW without usable preview', async () => {
const rawPath = path.join(tempDir, 'corrupt.raf')
const outputFile = path.join(tempDir, 'output-fail.webp')
vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce(null)
await expect(transcodeService.transcodeImage(rawPath, outputFile, 1920, 85)).rejects.toThrow(
'Cannot generate preview for RAW file',
)
})
it('should fetch image if input is a URL', async () => {
const mockBuffer = Buffer.from('fake-image-data')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -606,6 +715,8 @@ describe('TranscodeService', () => {
'-i',
'input.wav',
'-vn',
'-map',
'0:a:0?',
'-c:a',
'aac',
'-b:a',
@@ -640,6 +751,8 @@ describe('TranscodeService', () => {
'-i',
'input.wav',
'-vn',
'-map',
'0:a:0?',
'-c:a',
'aac',
'-b:a',
@@ -676,6 +789,8 @@ describe('TranscodeService', () => {
'-i',
'input.wav',
'-vn',
'-map',
'0:a:0?',
'-c:a',
'aac',
'-b:a',
@@ -2725,6 +2840,187 @@ describe('TranscodeService', () => {
expect(info.rotation).toBe(180)
})
it('getVideoInfo ignores attached picture streams and selects the main video stream', async () => {
/* eslint-disable @typescript-eslint/naming-convention */
const mockOutput = JSON.stringify({
format: { duration: '10.0', bit_rate: '20000000' },
streams: [
{
index: 0,
codec_type: 'video',
codec_name: 'mjpeg',
width: 320,
height: 240,
avg_frame_rate: '0/0',
r_frame_rate: '90000/1',
disposition: {
attached_pic: 1,
},
},
{
index: 1,
codec_type: 'video',
codec_name: 'h264',
width: 1920,
height: 1080,
avg_frame_rate: '24/1',
r_frame_rate: '24/1',
nb_frames: '240',
disposition: {
default: 1,
attached_pic: 0,
},
},
],
})
/* eslint-enable @typescript-eslint/naming-convention */
mockExecFileStdout(mockOutput)
const info = await transcodeService.getVideoInfo('cover-art-first.mp4')
expect(info.originalWidth).toBe(1920)
expect(info.originalHeight).toBe(1080)
expect(info.videoStreamIndex).toBe(1)
})
it('getVideoInfo throws error when file only contains audio and attached picture', async () => {
/* eslint-disable @typescript-eslint/naming-convention */
const mockOutput = JSON.stringify({
format: { duration: '10.0', bit_rate: '320000' },
streams: [
{
index: 0,
codec_type: 'audio',
codec_name: 'aac',
channels: 2,
sample_rate: '44100',
},
{
index: 1,
codec_type: 'video',
codec_name: 'mjpeg',
width: 320,
height: 240,
disposition: {
attached_pic: 1,
},
},
],
})
/* eslint-enable @typescript-eslint/naming-convention */
mockExecFileStdout(mockOutput)
await expect(transcodeService.getVideoInfo('audio-with-cover.mp4')).rejects.toThrow(
'No video stream found',
)
})
it('compareStreams prioritizes default stream then highest bitrate', () => {
/* eslint-disable @typescript-eslint/naming-convention */
const s1 = { index: 0, bit_rate: '5000000', disposition: { default: 0 } }
const s2 = { index: 1, bit_rate: '2000000', disposition: { default: 1 } }
const s3 = { index: 2, bit_rate: '8000000', disposition: { default: 0 } }
const s4 = { index: 3, disposition: {} }
/* eslint-enable @typescript-eslint/naming-convention */
const sorted = [s1, s2, s3, s4].sort(compareStreams)
expect(sorted[0].index).toBe(1) // default: 1
expect(sorted[1].index).toBe(2) // 8000000 bps
expect(sorted[2].index).toBe(0) // 5000000 bps
expect(sorted[3].index).toBe(3) // 0 bps
})
it('selectPrimaryVideoStream filters out attached picture streams', () => {
/* eslint-disable @typescript-eslint/naming-convention */
const streams = [
{ index: 0, codec_type: 'video', disposition: { attached_pic: 1 }, bit_rate: '9999999' },
{ index: 1, codec_type: 'audio', disposition: { default: 1 } },
{ index: 2, codec_type: 'video', disposition: { default: 0 }, bit_rate: '1500000' },
{ index: 3, codec_type: 'video', disposition: { default: 1 }, bit_rate: '1000000' },
]
/* eslint-enable @typescript-eslint/naming-convention */
const selected = selectPrimaryVideoStream(streams)
expect(selected).toBeDefined()
expect(selected?.index).toBe(3) // default: 1 non-attached
})
it('selectPrimaryAudioStream picks default then highest bitrate audio stream', () => {
/* eslint-disable @typescript-eslint/naming-convention */
const streams = [
{ index: 0, codec_type: 'video', bit_rate: '2000000' },
{ index: 1, codec_type: 'audio', disposition: { default: 0 }, bit_rate: '320000' },
{ index: 2, codec_type: 'audio', disposition: { default: 1 }, bit_rate: '128000' },
]
/* eslint-enable @typescript-eslint/naming-convention */
const selected = selectPrimaryAudioStream(streams)
expect(selected).toBeDefined()
expect(selected?.index).toBe(2) // default: 1 audio
})
it('transcodeVideo uses explicit streamIndex and audioStreamIndex in mapping', async () => {
let executedArgs: string[] = []
;(
child_process.execFile as unknown as {
mockImplementation: (
fn: (
file: string,
args: string[],
callback: (error: Error | null, result: { stdout: string; stderr: string }) => void,
) => void,
) => void
}
).mockImplementation((_file, args, callback) => {
executedArgs = args
callback(null, { stdout: '', stderr: '' })
})
await transcodeService.transcodeVideo({
inputFile: 'input.mp4',
outputFile: 'output.mp4',
width: 1280,
height: 720,
hardwareAcceleration: 'off',
streamIndex: 2,
audioStreamIndex: 1,
})
const filterIdx = executedArgs.indexOf('-filter_complex')
expect(filterIdx).toBeGreaterThan(-1)
expect(executedArgs[filterIdx + 1]).toContain('[0:2]scale=')
const mapIndices = executedArgs
.map((arg, idx) => (arg === '-map' ? executedArgs[idx + 1] : null))
.filter(Boolean)
expect(mapIndices).toContain('0:1')
})
it('generatePoster uses explicit streamIndex in mapping', async () => {
let executedArgs: string[] = []
;(
child_process.execFile as unknown as {
mockImplementation: (
fn: (
file: string,
args: string[],
callback: (error: Error | null, result: { stdout: string; stderr: string }) => void,
) => void,
) => void
}
).mockImplementation((_file, args, callback) => {
executedArgs = args
callback(null, { stdout: '', stderr: '' })
})
await transcodeService.generatePoster('input.mp4', 'poster.webp', { streamIndex: 3 })
const mapIdx = executedArgs.indexOf('-map')
expect(mapIdx).toBeGreaterThan(-1)
expect(executedArgs[mapIdx + 1]).toBe('0:3')
})
it('transcodeVideo preserves HDR parameters and tags when hdr is true', async () => {
let executedArgs: string[] = []
;(
@@ -2912,7 +3208,7 @@ describe('TranscodeService', () => {
const filterIdx = spriteCmd!.indexOf('-filter_complex')
expect(filterIdx).toBeGreaterThan(-1)
const filterStr = spriteCmd![filterIdx + 1]
expect(filterStr).toContain('[0:v]fps=')
expect(filterStr).toContain('[0:V]fps=')
expect(filterStr).toContain('scale=w=300:h=-2,tile=10x10[sprite_out]')
})
@@ -3275,4 +3571,106 @@ describe('TranscodeService', () => {
).rejects.toThrow('Poster generation cancelled')
})
})
describe('HLS utilities', () => {
it('builds master playlist correctly with bandwidth and codecs', () => {
const playlist = buildHlsMasterPlaylist([
{ resolution: '1080p', width: 1920, height: 1080, bitrateBps: 4_500_000, isHdr: false },
{ resolution: '720p', width: 1280, height: 720, bitrateBps: 2_500_000, isHdr: false },
{ resolution: '480p', width: 854, height: 480, bitrateBps: 1_200_000, isHdr: false },
])
expect(playlist).toContain('#EXTM3U')
expect(playlist).toContain('#EXT-X-VERSION:7')
expect(playlist).toContain('#EXT-X-INDEPENDENT-SEGMENTS')
expect(playlist).toContain('RESOLUTION=1920x1080')
expect(playlist).toContain('BANDWIDTH=4628000')
expect(playlist).toContain('1080p/index.m3u8')
expect(playlist).toContain('RESOLUTION=1280x720')
expect(playlist).toContain('720p/index.m3u8')
expect(playlist).toContain('RESOLUTION=854x480')
expect(playlist).toContain('480p/index.m3u8')
})
it('builds master playlist with HDR codec when isHdr is true', () => {
const playlist = buildHlsMasterPlaylist([
{ resolution: '1080p', width: 1920, height: 1080, bitrateBps: 4_500_000, isHdr: true },
])
expect(playlist).toContain('CODECS="avc1.640028,mp4a.40.2"')
})
it('rewrites variant m3u8 playlist with presigned S3 URLs', async () => {
const sampleM3u8 = [
'#EXTM3U',
'#EXT-X-VERSION:7',
'#EXT-X-TARGETDURATION:4',
'#EXT-X-MEDIA-SEQUENCE:0',
'#EXT-X-PLAYLIST-TYPE:VOD',
'#EXT-X-MAP:URI="init.mp4"',
'#EXTINF:4.000000,',
'segment_000.m4s',
'#EXTINF:4.000000,',
'segment_001.m4s',
'#EXT-X-ENDLIST',
].join('\n')
const rewritten = await rewriteM3u8WithPresignedUrls(
sampleM3u8,
'files/asset1/hls/1080p',
'shumai',
)
expect(rewritten).toContain(
'#EXT-X-MAP:URI="https://presigned.example.com/files/asset1/hls/1080p/init.mp4"',
)
expect(rewritten).toContain(
'https://presigned.example.com/files/asset1/hls/1080p/segment_000.m4s',
)
expect(rewritten).toContain(
'https://presigned.example.com/files/asset1/hls/1080p/segment_001.m4s',
)
expect(rewritten).toContain('#EXTINF:4.000000,')
expect(rewritten).toContain('#EXT-X-ENDLIST')
})
it('executes transcodeHlsRendition with correct fMP4 CMAF parameters', async () => {
let executedArgs: string[] = []
// child_process.execFile mock signature
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(child_process.execFile as any).mockImplementation(
(
_file: string,
args: string[],
cb: (err: unknown, result: { stdout: string; stderr: string }) => void,
) => {
executedArgs = args
cb(null, { stdout: '', stderr: '' })
},
)
const outputDir = path.join(tempDir, 'hls_output')
fs.mkdirSync(outputDir, { recursive: true })
await transcodeService.transcodeHlsRendition({
inputFile: 'input.mp4',
outputDir,
width: 1920,
height: 1080,
frameRate: 30,
hardwareAcceleration: 'off',
threads: 4,
})
expect(executedArgs).toContain('-f')
expect(executedArgs).toContain('hls')
expect(executedArgs).toContain('-hls_segment_type')
expect(executedArgs).toContain('fmp4')
expect(executedArgs).toContain('-hls_fmp4_init_filename')
expect(executedArgs).toContain('init.mp4')
expect(executedArgs).toContain('-flags')
expect(executedArgs).toContain('+cgop')
expect(executedArgs).toContain('-threads')
expect(executedArgs).toContain('4')
})
})
})
+639 -77
View File
@@ -12,7 +12,9 @@ import sharp from 'sharp'
import { ulid } from 'ulid'
import { promisify } from 'util'
import { mapConcurrent } from '../utils/async'
import { isRawImage } from '../utils/raw'
import { dataFormatNames } from './dataFormatNames'
import { extractAndValidateRawPreview, EXIF_ORIENTATION_TO_ROTATION } from './raw-extract'
import { logger } from '@shumai/core/src/logger'
const execFileAsync = promisify(execFile)
@@ -95,6 +97,54 @@ export interface GeneratePosterOptions {
hdrType?: HdrType
colorTransfer?: string
signal?: AbortSignal
streamIndex?: number
}
export interface FfprobeStreamLike {
index: number
codec_type?: string
codec_name?: string
bit_rate?: string
avg_frame_rate?: string
r_frame_rate?: string
nb_frames?: string
width?: number
height?: number
channels?: number
sample_rate?: string | number
bits_per_raw_sample?: string | number
bits_per_sample?: string | number
color_transfer?: string
color_primaries?: string
color_space?: string
disposition?: Record<string, number | undefined>
tags?: Record<string, string | undefined>
side_data_list?: Array<Record<string, unknown>>
mime_codec_string?: string
[key: string]: unknown
}
/**
* Immich-compatible stream comparator:
* 1. Streams with disposition.default: 1 come first.
* 2. Ties are broken by higher bit_rate.
*/
export function compareStreams<T extends FfprobeStreamLike>(a: T, b: T): number {
const defDiff = (b.disposition?.default ?? 0) - (a.disposition?.default ?? 0)
if (defDiff !== 0) return defDiff
const aBitrate = parseInt(a.bit_rate || '0', 10) || 0
const bBitrate = parseInt(b.bit_rate || '0', 10) || 0
return bBitrate - aBitrate
}
export function selectPrimaryVideoStream<T extends FfprobeStreamLike>(streams: T[]): T | undefined {
return streams
.filter((s) => s.codec_type === 'video' && !s.disposition?.attached_pic)
.sort(compareStreams)[0]
}
export function selectPrimaryAudioStream<T extends FfprobeStreamLike>(streams: T[]): T | undefined {
return streams.filter((s) => s.codec_type === 'audio').sort(compareStreams)[0]
}
export interface MediaMetadata {
@@ -121,6 +171,8 @@ export interface MediaMetadata {
dvProfile?: number
dvCompatibilityId?: number
rotation?: number
videoStreamIndex?: number
audioStreamIndex?: number
}
export interface TranscodeVideoParams {
@@ -142,6 +194,32 @@ export interface TranscodeVideoParams {
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
streamIndex?: number
audioStreamIndex?: number
}
export interface TranscodeHlsRenditionParams {
inputFile: string
outputDir: string
width: number
height: number
frameRate?: number | string
disableAudio?: boolean
overlayFile?: string
hardwareAcceleration?: 'off' | 'auto'
videoBitrate?: string
sourceVideoBitrate?: number
threads?: number
signal?: AbortSignal
hdr?: boolean
sourceIsHdr?: boolean
sourceHdrType?: HdrType
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
segmentDuration?: number
streamIndex?: number
audioStreamIndex?: number
}
export interface EncoderConfig {
@@ -239,9 +317,11 @@ export function getDefaultBitrateBps(height: number, width?: number): number {
const effectiveHeight = shortSide > 0 ? shortSide : height
if (effectiveHeight >= 2160 || longSide >= 3840) return 12_000_000
if (effectiveHeight >= 1440 || longSide >= 2560) return 8_000_000
if (effectiveHeight >= 1080 || longSide >= 1920) return 4_500_000
if (effectiveHeight >= 720 || longSide >= 1280) return 2_500_000
if (effectiveHeight >= 540 || longSide >= 960) return 1_200_000
if (effectiveHeight >= 480 || longSide >= 854) return 1_400_000
if (effectiveHeight >= 360 || longSide >= 640) return 800_000
return 100_000
}
@@ -449,10 +529,15 @@ export class TranscodeService {
return stream.codec_name
}
private safeParseInt(value: string | number | undefined): number | undefined {
private safeParseInt(value: unknown): number | undefined {
if (value === undefined || value === null) return undefined
const parsed = typeof value === 'string' ? parseInt(value, 10) : value
return Number.isFinite(parsed) ? (parsed as number) : undefined
const parsed =
typeof value === 'string'
? parseInt(value, 10)
: typeof value === 'number'
? value
: undefined
return parsed !== undefined && Number.isFinite(parsed) ? parsed : undefined
}
async getVideoInfo(inputFile: string): Promise<MediaMetadata> {
@@ -466,10 +551,8 @@ export class TranscodeService {
inputFile,
])
const info = JSON.parse(stdout)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const videoStream = info.streams.find((s: any) => s.codec_type === 'video')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const audioStream = info.streams.find((s: any) => s.codec_type === 'audio')
const videoStream = selectPrimaryVideoStream(info.streams || [])
const audioStream = selectPrimaryAudioStream(info.streams || [])
if (!videoStream) {
throw new Error('No video stream found')
@@ -652,6 +735,8 @@ export class TranscodeService {
hdrType,
dvProfile,
dvCompatibilityId,
videoStreamIndex: videoStream.index,
audioStreamIndex: audioStream ? audioStream.index : undefined,
}
}
@@ -699,6 +784,48 @@ export class TranscodeService {
}
}
// RAW branch — extract embedded JPEG for metadata
if (typeof input === 'string' && isRawImage(input)) {
const extracted = await extractAndValidateRawPreview(input)
if (extracted) {
try {
const isSwapped =
extracted.orientation !== undefined &&
extracted.orientation >= 5 &&
extracted.orientation <= 8
const originalWidth = isSwapped ? extracted.height : extracted.width
const originalHeight = isSwapped ? extracted.width : extracted.height
return {
originalWidth,
originalHeight,
duration: 0,
bitRate: 0,
frameRate: 0,
totalFrames: 0,
startTimecode: undefined,
hasAudio: false,
mimeType: 'jpeg',
}
} finally {
extracted.cleanup()
}
}
// No extractable preview — return zero dimensions
// The workflow will still mark the asset as processed
return {
originalWidth: 0,
originalHeight: 0,
duration: 0,
bitRate: 0,
frameRate: 0,
totalFrames: 0,
startTimecode: undefined,
hasAudio: false,
mimeType: '',
}
}
const metadata = await sharp(input, { limitInputPixels: false }).metadata()
return {
originalWidth: metadata.width || 0,
@@ -952,11 +1079,13 @@ export class TranscodeService {
args.push('-i', params.inputFile)
const baseScale = `scale=w=${params.width}:h=${params.height}:force_original_aspect_ratio=decrease,scale=w='trunc(iw/2)*2':h='trunc(ih/2)*2'`
const vPad = params.streamIndex !== undefined ? `0:${params.streamIndex}` : '0:V'
if (params.overlayFile) {
args.push('-i', params.overlayFile)
filterComplex = `[0:v]scale=${params.width}:${params.height}[vscaled];[vscaled][1:v]overlay=0:0`
filterComplex = `[${vPad}]scale=${params.width}:${params.height}[vscaled];[vscaled][1:v]overlay=0:0`
} else {
filterComplex = `[0:v]${baseScale}`
filterComplex = `[${vPad}]${baseScale}`
}
if (params.frameRate) {
@@ -995,7 +1124,8 @@ export class TranscodeService {
}
if (!params.disableAudio) {
args.push('-map', '0:a?')
const aMap = params.audioStreamIndex !== undefined ? `0:${params.audioStreamIndex}` : '0:a:0?'
args.push('-map', aMap)
}
args.push('-c:v', encoder.name)
@@ -1087,6 +1217,304 @@ export class TranscodeService {
}
}
async transcodeHlsRendition(params: TranscodeHlsRenditionParams): Promise<void> {
const encoder = await this.selectH264Encoder(params.hardwareAcceleration)
if (encoder.name !== 'libx264') {
try {
await this.executeFfmpegHlsTranscode(params, encoder)
return
} catch (err) {
if (this.isAbortError(err, params.signal)) {
throw err
}
logger.warn(
{
err,
encoder: encoder.name,
inputFile: params.inputFile,
outputDir: params.outputDir,
width: params.width,
height: params.height,
},
'Hardware HLS transcoding failed; falling back to software transcode (libx264)',
)
if (fs.existsSync(params.outputDir)) {
try {
fs.rmSync(params.outputDir, { recursive: true, force: true })
} catch {
// ignore unlink errors
}
}
await this.executeFfmpegHlsTranscode(params, H264_ENCODER_CONFIGS.libx264)
return
}
}
await this.executeFfmpegHlsTranscode(params, encoder)
}
private async executeFfmpegHlsTranscode(
params: TranscodeHlsRenditionParams,
encoder: EncoderConfig,
): Promise<void> {
fs.mkdirSync(params.outputDir, { recursive: true })
const isSourceHdr =
params.sourceIsHdr ||
params.sourceHdrType === 'pq' ||
params.sourceHdrType === 'hlg' ||
params.sourceHdrType === 'dovi_p5'
const isHdrOutput = Boolean(params.hdr)
const isVaapi = encoder.name === 'h264_vaapi'
const vaapiDevice = isVaapi ? (this.getVaapiDevice() ?? '/dev/dri/renderD128') : undefined
const segmentDuration = params.segmentDuration || 4
let calculatedFps = 30
if (params.frameRate) {
if (typeof params.frameRate === 'number') {
calculatedFps = params.frameRate
} else {
const parts = params.frameRate.split('/')
if (parts.length === 2) {
calculatedFps = parseFloat(parts[0]) / parseFloat(parts[1])
} else {
calculatedFps = parseFloat(params.frameRate)
}
}
}
if (!Number.isFinite(calculatedFps) || calculatedFps <= 0) {
calculatedFps = 30
}
const gopSize = Math.max(1, Math.round(calculatedFps * segmentDuration))
logger.info(
{
encoder: encoder.name,
hardwareAcceleration: params.hardwareAcceleration ?? 'off',
inputFile: params.inputFile,
outputDir: params.outputDir,
width: params.width,
height: params.height,
segmentDuration,
gopSize,
},
'Starting HLS rendition transcoding',
)
let filterComplex: string
const args: string[] = []
if (isVaapi && vaapiDevice) {
args.push('-init_hw_device', `vaapi=accel:${vaapiDevice}`, '-filter_hw_device', 'accel')
}
args.push('-i', params.inputFile)
const baseScale = `scale=w=${params.width}:h=${params.height}:force_original_aspect_ratio=decrease,scale=w='trunc(iw/2)*2':h='trunc(ih/2)*2'`
const vPad = params.streamIndex !== undefined ? `0:${params.streamIndex}` : '0:V'
if (params.overlayFile) {
args.push('-i', params.overlayFile)
filterComplex = `[${vPad}]scale=${params.width}:${params.height}[vscaled];[vscaled][1:v]overlay=0:0`
} else {
filterComplex = `[${vPad}]${baseScale}`
}
filterComplex += `,fps=${calculatedFps}`
if (isVaapi) {
filterComplex += ',format=nv12,hwupload=extra_hw_frames=64'
}
filterComplex += '[vout]'
args.push('-filter_complex', filterComplex, '-map', '[vout]')
if (!params.disableAudio) {
const aMap = params.audioStreamIndex !== undefined ? `0:${params.audioStreamIndex}` : '0:a:0?'
args.push('-map', aMap)
}
args.push('-c:v', encoder.name)
if (encoder.presetArgs.length > 0) {
args.push(...encoder.presetArgs)
}
if (encoder.name === 'h264_videotoolbox') {
if (params.videoBitrate) {
args.push('-b:v', params.videoBitrate)
} else {
const targetBps = calculateEffectiveBitrateBps(
params.height,
params.width,
params.sourceVideoBitrate,
calculatedFps,
)
const targetKbps = Math.max(50, Math.round(targetBps / 1000))
args.push('-b:v', `${targetKbps}k`)
}
} else if (isVaapi) {
let maxKbps: number
if (params.videoBitrate) {
maxKbps = parseBitrateKbps(params.videoBitrate)
} else {
const { maxrate } = calculateMaxBitrate(
params.height,
params.width,
params.sourceVideoBitrate,
calculatedFps,
)
maxKbps = parseInt(maxrate, 10)
}
const targetKbps = Math.max(25, Math.ceil(maxKbps / 1.45))
const minKbps = Math.max(10, Math.round(targetKbps / 2))
args.push(
'-rc_mode',
'3',
'-b:v',
`${targetKbps}k`,
'-minrate',
`${minKbps}k`,
'-maxrate',
`${maxKbps}k`,
)
} else if (encoder.name === 'h264_qsv') {
let maxKbps: number
if (params.videoBitrate) {
maxKbps = parseBitrateKbps(params.videoBitrate)
} else {
const { maxrate } = calculateMaxBitrate(
params.height,
params.width,
params.sourceVideoBitrate,
calculatedFps,
)
maxKbps = parseInt(maxrate, 10)
}
const targetKbps = Math.max(25, Math.ceil(maxKbps / 1.45))
args.push('-b:v', `${targetKbps}k`, '-maxrate', `${maxKbps}k`)
} else if (encoder.name === 'h264_nvenc') {
let maxrateStr: string
let bufsizeStr: string
if (params.videoBitrate) {
const kbps = parseBitrateKbps(params.videoBitrate)
maxrateStr = `${kbps}k`
bufsizeStr = `${kbps * 2}k`
} else {
const res = calculateMaxBitrate(
params.height,
params.width,
params.sourceVideoBitrate,
calculatedFps,
)
maxrateStr = res.maxrate
bufsizeStr = res.bufsize
}
args.push(
'-rc',
'vbr',
'-cq',
'23',
'-maxrate',
maxrateStr,
'-bufsize',
bufsizeStr,
'-spatial-aq',
'1',
'-temporal-aq',
'1',
)
} else {
let maxrateStr: string
let bufsizeStr: string
if (params.videoBitrate) {
const kbps = parseBitrateKbps(params.videoBitrate)
maxrateStr = `${kbps}k`
bufsizeStr = `${kbps * 2}k`
} else {
const res = calculateMaxBitrate(
params.height,
params.width,
params.sourceVideoBitrate,
calculatedFps,
)
maxrateStr = res.maxrate
bufsizeStr = res.bufsize
}
args.push('-crf', '23', '-maxrate', maxrateStr, '-bufsize', bufsizeStr)
}
if (!isVaapi) {
args.push('-pix_fmt', 'yuv420p')
}
if (isSourceHdr && isHdrOutput) {
const isHlg = params.sourceHdrType === 'hlg' || params.sourceColorTransfer === 'arib-std-b67'
const trc = isHlg ? 'arib-std-b67' : 'smpte2084'
args.push('-color_primaries', 'bt2020', '-color_trc', trc, '-colorspace', 'bt2020nc')
if (encoder.name === 'libx264') {
args.push('-x264-params', `colorprim=bt2020:transfer=${trc}:colormatrix=bt2020nc`)
}
}
args.push(
'-r',
calculatedFps.toString(),
'-g',
gopSize.toString(),
'-keyint_min',
gopSize.toString(),
'-sc_threshold',
'0',
'-flags',
'+cgop',
'-force_key_frames',
`expr:gte(t,n_forced*${segmentDuration})`,
)
if (!params.disableAudio) {
args.push('-c:a', 'aac', '-b:a', '128k')
}
if (params.threads && params.threads > 0) {
args.push('-threads', params.threads.toString())
}
const initFilename = 'init.mp4'
const segmentPattern = path.join(params.outputDir, 'segment_%03d.m4s')
const playlistFile = path.join(params.outputDir, 'index.m3u8')
args.push(
'-f',
'hls',
'-hls_time',
segmentDuration.toString(),
'-hls_playlist_type',
'vod',
'-hls_segment_type',
'fmp4',
'-hls_fmp4_init_filename',
initFilename,
'-hls_segment_filename',
segmentPattern,
'-max_muxing_queue_size',
'1024',
playlistFile,
)
if (params.signal) {
await execFileAsync('ffmpeg', ['-y', '-loglevel', 'warning', ...args], {
signal: params.signal,
})
} else {
await execFileAsync('ffmpeg', ['-y', '-loglevel', 'warning', ...args])
}
}
async transcodeImage(
inputFile: string | Buffer,
outputFile: string,
@@ -1117,73 +1545,106 @@ export class TranscodeService {
input = Buffer.from(await resp.arrayBuffer())
}
const WEBP_MAX_DIMENSION = 7680
let targetW = width > 0 ? Math.min(width, WEBP_MAX_DIMENSION) : WEBP_MAX_DIMENSION
let targetH = height && height > 0 ? Math.min(height, WEBP_MAX_DIMENSION) : WEBP_MAX_DIMENSION
const sharpInstance = sharp(input, { limitInputPixels: false })
if (isPreview) {
try {
const meta = await sharpInstance.metadata()
if (meta.width && meta.height) {
// Fallback shim: If legacy 480 caller passed width=480, map targetShort to 300
const targetShort = width === 480 ? 300 : width
const maxLong = Math.round((targetShort * 16) / 9)
const dims = calculatePreviewDimensions(meta.width, meta.height, targetShort, maxLong)
targetW = dims.width
targetH = dims.height
}
} catch {
// Fallback to targetW/targetH as calculated above
// RAW branch — extract embedded JPEG to temporary file + orientation, then treat as normal image
let rawOrientation: number | undefined
let rawCleanup: (() => void) | null = null
if (typeof input === 'string' && isRawImage(input)) {
const extracted = await extractAndValidateRawPreview(input)
if (!extracted) {
throw new Error(`Cannot generate preview for RAW file: no usable embedded JPEG in ${input}`)
}
input = extracted.previewPath
rawOrientation = extracted.orientation
rawCleanup = extracted.cleanup
}
// WEBP_MAX_DIMENSION (7680) safety cap is ALWAYS enforced
targetW = Math.min(targetW, WEBP_MAX_DIMENSION)
targetH = Math.min(targetH, WEBP_MAX_DIMENSION)
try {
const WEBP_MAX_DIMENSION = 7680
let targetW = width > 0 ? Math.min(width, WEBP_MAX_DIMENSION) : WEBP_MAX_DIMENSION
let targetH = height && height > 0 ? Math.min(height, WEBP_MAX_DIMENSION) : WEBP_MAX_DIMENSION
if (isPsdInput(input)) {
let psdPath: string
let tempDirToCleanup: string | null = null
if (typeof input === 'string') {
psdPath = input
} else {
tempDirToCleanup = this.createTempDir('psd-transcode-')
psdPath = path.join(tempDirToCleanup, 'input.psd')
fs.writeFileSync(psdPath, input)
}
const sharpInstance = sharp(input, { limitInputPixels: false })
try {
const resizeGeometry =
targetH < WEBP_MAX_DIMENSION
? `${targetW}x${targetH}>`
: `${targetW}x${WEBP_MAX_DIMENSION}>`
await this.execImageMagick([
`${psdPath}[0]`,
'-colorspace',
'sRGB',
'-resize',
resizeGeometry,
'-quality',
quality.toString(),
outputFile,
])
return
} finally {
if (tempDirToCleanup) {
this.removeDir(tempDirToCleanup)
if (isPreview) {
try {
const meta = await sharpInstance.metadata()
if (meta.width && meta.height) {
const isSwapped =
rawOrientation !== undefined && rawOrientation >= 5 && rawOrientation <= 8
const srcW = isSwapped ? meta.height : meta.width
const srcH = isSwapped ? meta.width : meta.height
// Fallback shim: If legacy 480 caller passed width=480, map targetShort to 300
const targetShort = width === 480 ? 300 : width
const maxLong = Math.round((targetShort * 16) / 9)
const dims = calculatePreviewDimensions(srcW, srcH, targetShort, maxLong)
targetW = dims.width
targetH = dims.height
}
} catch {
// Fallback to targetW/targetH as calculated above
}
}
// WEBP_MAX_DIMENSION (7680) safety cap is ALWAYS enforced
targetW = Math.min(targetW, WEBP_MAX_DIMENSION)
targetH = Math.min(targetH, WEBP_MAX_DIMENSION)
if (isPsdInput(input)) {
let psdPath: string
let tempDirToCleanup: string | null = null
if (typeof input === 'string') {
psdPath = input
} else {
tempDirToCleanup = this.createTempDir('psd-transcode-')
psdPath = path.join(tempDirToCleanup, 'input.psd')
fs.writeFileSync(psdPath, input)
}
try {
const resizeGeometry =
targetH < WEBP_MAX_DIMENSION
? `${targetW}x${targetH}>`
: `${targetW}x${WEBP_MAX_DIMENSION}>`
await this.execImageMagick([
`${psdPath}[0]`,
'-colorspace',
'sRGB',
'-resize',
resizeGeometry,
'-quality',
quality.toString(),
outputFile,
])
return
} finally {
if (tempDirToCleanup) {
this.removeDir(tempDirToCleanup)
}
}
}
// Apply EXIF orientation from the RAW container to the extracted buffer.
// The buffer itself often lacks orientation EXIF, so Sharp won't auto-rotate.
// For non-RAW images, Sharp auto-rotates from the image's own EXIF.
if (rawOrientation && EXIF_ORIENTATION_TO_ROTATION[rawOrientation]) {
const { angle, flip, flop } = EXIF_ORIENTATION_TO_ROTATION[rawOrientation]
if (angle) sharpInstance.rotate(angle)
if (flip) sharpInstance.flip()
if (flop) sharpInstance.flop()
}
sharpInstance.toColorspace('srgb').resize(targetW, targetH, {
withoutEnlargement: true,
fit: 'inside',
})
await sharpInstance.webp({ quality }).toFile(outputFile)
} finally {
if (rawCleanup) {
rawCleanup()
}
}
sharpInstance.toColorspace('srgb').resize(targetW, targetH, {
withoutEnlargement: true,
fit: 'inside',
})
await sharpInstance.webp({ quality }).toFile(outputFile)
}
async generatePoster(
@@ -1228,6 +1689,8 @@ export class TranscodeService {
filters.push('scale=-2:300:force_original_aspect_ratio=decrease')
const vMap = options.streamIndex !== undefined ? `0:${options.streamIndex}` : '0:V:0'
const args = [
...(isRemote
? ['-reconnect', '1', '-reconnect_streamed', '1', '-reconnect_delay_max', '5']
@@ -1235,6 +1698,8 @@ export class TranscodeService {
...(isMpegTs ? [] : ['-skip_frame', 'nointra']),
'-i',
inputFile,
'-map',
vMap,
'-vf',
filters.join(','),
'-fps_mode',
@@ -1264,13 +1729,14 @@ export class TranscodeService {
duration: number,
signal?: AbortSignal,
hdrOptions?: { isHdr?: boolean; hdrType?: HdrType; colorTransfer?: string },
streamIndex?: number,
): Promise<void> {
if (signal?.aborted) {
throw new Error('Sprite generation cancelled')
}
// 1. Generate poster first (fast, smart frame selection)
await this.generatePoster(inputFile, outputPoster, { ...hdrOptions, signal })
await this.generatePoster(inputFile, outputPoster, { ...hdrOptions, signal, streamIndex })
// 2. Generate sprite
let fileSize = Infinity
@@ -1283,9 +1749,23 @@ export class TranscodeService {
const isSmallAndShort = fileSize <= 50 * 1024 * 1024 && duration <= 30
if (isSmallAndShort) {
await this.generateSpriteSinglePass(inputFile, outputSprite, duration, signal, hdrOptions)
await this.generateSpriteSinglePass(
inputFile,
outputSprite,
duration,
signal,
hdrOptions,
streamIndex,
)
} else {
await this.generateSpriteSeekPool(inputFile, outputSprite, duration, signal, hdrOptions)
await this.generateSpriteSeekPool(
inputFile,
outputSprite,
duration,
signal,
hdrOptions,
streamIndex,
)
}
}
@@ -1295,9 +1775,11 @@ export class TranscodeService {
duration: number,
signal?: AbortSignal,
hdrOptions?: { isHdr?: boolean; hdrType?: HdrType; colorTransfer?: string },
streamIndex?: number,
): Promise<void> {
const spriteFps = 100 / duration
let filterComplex: string
const vPad = streamIndex !== undefined ? `0:${streamIndex}` : '0:V'
if (hdrOptions?.isHdr) {
const availableFilters = await this.getAvailableFilters()
@@ -1306,9 +1788,9 @@ export class TranscodeService {
colorTransfer: hdrOptions.colorTransfer,
availableFilters,
})
filterComplex = `[0:v]${tonemap},fps=${spriteFps},scale=w=300:h=-2,tile=10x10[sprite_out]`
filterComplex = `[${vPad}]${tonemap},fps=${spriteFps},scale=w=300:h=-2,tile=10x10[sprite_out]`
} else {
filterComplex = `[0:v]fps=${spriteFps},scale=w=300:h=-2,tile=10x10[sprite_out]`
filterComplex = `[${vPad}]fps=${spriteFps},scale=w=300:h=-2,tile=10x10[sprite_out]`
}
const args = [
@@ -1339,6 +1821,7 @@ export class TranscodeService {
duration: number,
signal?: AbortSignal,
hdrOptions?: { isHdr?: boolean; hdrType?: HdrType; colorTransfer?: string },
streamIndex?: number,
): Promise<void> {
if (signal?.aborted) {
throw new Error('Sprite generation cancelled')
@@ -1382,11 +1865,14 @@ export class TranscodeService {
const ts = timestamps[idx]
const framePath = path.join(tmpDir, `frame_${idx.toString().padStart(3, '0')}.webp`)
const vMap = streamIndex !== undefined ? `0:${streamIndex}` : '0:V:0'
const args = [
'-ss',
ts.toFixed(3),
'-i',
inputFile,
'-map',
vMap,
'-vframes',
'1',
'-vf',
@@ -1778,8 +2264,7 @@ export class TranscodeService {
inputFile,
])
const info = JSON.parse(stdout)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const audioStream = info.streams.find((s: any) => s.codec_type === 'audio')
const audioStream = selectPrimaryAudioStream(info.streams || [])
if (!audioStream) {
throw new Error('No audio stream found')
@@ -1803,6 +2288,7 @@ export class TranscodeService {
audioBitDepth:
this.safeParseInt(audioStream?.bits_per_raw_sample) ??
this.safeParseInt(audioStream?.bits_per_sample),
audioStreamIndex: audioStream.index,
mimeType: '',
}
}
@@ -1813,9 +2299,23 @@ export class TranscodeService {
bitrate?: string
threads?: number
signal?: AbortSignal
audioStreamIndex?: number
}): Promise<void> {
const bitrate = params.bitrate || '128k'
const args = ['-i', params.inputFile, '-vn', '-c:a', 'aac', '-b:a', bitrate, '-ac', '2']
const aMap = params.audioStreamIndex !== undefined ? `0:${params.audioStreamIndex}` : '0:a:0?'
const args = [
'-i',
params.inputFile,
'-vn',
'-map',
aMap,
'-c:a',
'aac',
'-b:a',
bitrate,
'-ac',
'2',
]
if (params.threads && params.threads > 0) {
args.push('-threads', params.threads.toString())
}
@@ -1844,6 +2344,7 @@ export class TranscodeService {
const numFrames = Math.max(1, params.numFrames)
const step = duration / numFrames
const timestamps = Array.from({ length: numFrames }, (_, i) => i * step)
const vMap = meta.videoStreamIndex !== undefined ? `0:${meta.videoStreamIndex}` : '0:V:0'
await mapConcurrent(timestamps, 10, async (t, idx) => {
const outputFile = path.join(params.outputDir, `${idx + 1}.webp`)
@@ -1855,6 +2356,8 @@ export class TranscodeService {
t.toFixed(4),
'-i',
params.inputFile,
'-map',
vMap,
'-vframes',
'1',
'-vf',
@@ -1979,6 +2482,8 @@ export class TranscodeService {
t.toFixed(4),
'-i',
inputSource,
'-map',
'0:V:0',
'-vframes',
'1',
'-vf',
@@ -2308,3 +2813,60 @@ function renderAnnotationsToSvg(
}
export const transcodeService = new TranscodeService()
export function buildHlsMasterPlaylist(
renditions: Array<{
resolution: string
width: number
height: number
bitrateBps: number
isHdr?: boolean
}>,
): string {
let content = '#EXTM3U\n#EXT-X-VERSION:7\n#EXT-X-INDEPENDENT-SEGMENTS\n'
for (const r of renditions) {
const bandwidth = r.bitrateBps + 128_000
const codecs = r.isHdr ? 'avc1.640028,mp4a.40.2' : 'avc1.64001f,mp4a.40.2'
content += `#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},AVERAGE-BANDWIDTH=${bandwidth},RESOLUTION=${r.width}x${r.height},CODECS="${codecs}"\n`
content += `${r.resolution}/index.m3u8\n`
}
return content
}
export async function rewriteM3u8WithPresignedUrls(
m3u8Content: string,
s3Prefix: string,
bucket: string,
): Promise<string> {
const lines = m3u8Content.split(/\r?\n/)
const rewrittenLines: string[] = []
for (const line of lines) {
const trimmed = line.trim()
if (!trimmed) {
rewrittenLines.push(line)
continue
}
const mapMatch = trimmed.match(/^#EXT-X-MAP:URI="([^"]+)"(.*)$/)
if (mapMatch) {
const initRelPath = mapMatch[1]
const extra = mapMatch[2] || ''
const initKey = path.posix.join(s3Prefix, initRelPath)
const presignedUrl = await s3Service.presign(bucket, initKey, 'GET')
rewrittenLines.push(`#EXT-X-MAP:URI="${presignedUrl}"${extra}`)
continue
}
if (trimmed.endsWith('.m4s') || trimmed.endsWith('.mp4') || trimmed.endsWith('.ts')) {
const segmentKey = path.posix.join(s3Prefix, trimmed)
const presignedUrl = await s3Service.presign(bucket, segmentKey, 'GET')
rewrittenLines.push(presignedUrl)
continue
}
rewrittenLines.push(line)
}
return rewrittenLines.join('\n')
}
+7 -3
View File
@@ -198,17 +198,19 @@ describe('UploadService', () => {
threads: 0,
sprite: true,
poster: true,
hlsEnabled: false,
},
})
})
it('should pass custom transcode.threads from team settings to transcode task', async () => {
it('should pass custom transcode.threads and videoResolutions from team settings to transcode task', async () => {
await prisma.team.update({
where: { id: teamId },
data: {
settings: {
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 12,
},
@@ -244,11 +246,13 @@ describe('UploadService', () => {
expect(workflowTask?.payload).toEqual({
projectId: projectId,
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 12,
sprite: true,
poster: true,
hlsEnabled: false,
},
})
})
+5
View File
@@ -334,12 +334,17 @@ export class UploadService {
const threads = settings?.transcode?.threads ?? 0
if (isVideo) {
const strategy = settings?.transcode?.videoStrategy || 'best_match'
const videoResolutions = settings?.transcode?.videoResolutions
const hardwareAcceleration = settings?.transcode?.hardwareAcceleration || 'off'
const hlsEnabled = settings?.transcode?.hlsEnabled ?? false
const hlsResolutions = settings?.transcode?.hlsResolutions
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await new VideoTranscoder(tx as any, asset.id, team.id, projectId)
.setStrategy(strategy)
.setVideoResolutions(videoResolutions)
.setHardwareAcceleration(hardwareAcceleration)
.setThreads(threads)
.setHls(hlsEnabled, hlsResolutions)
.withSprite()
.withPoster()
.submit()
+24
View File
@@ -15,6 +15,30 @@ const EXTENSION_MIME_TYPES: Record<string, string> = {
'.txt': 'text/plain',
'.md': 'text/markdown',
'.markdown': 'text/markdown',
// Camera RAW formats
'.3fr': 'image/x-hasselblad-3fr',
'.arw': 'image/x-sony-arw',
'.cr2': 'image/x-canon-cr2',
'.cr3': 'image/x-canon-cr3',
'.crw': 'image/x-canon-crw',
'.dcr': 'image/x-kodak-dcr',
'.dng': 'image/x-adobe-dng',
'.erf': 'image/x-epson-erf',
'.fff': 'image/x-hasselblad-fff',
'.iiq': 'image/x-phaseone-iiq',
'.kdc': 'image/x-kodak-kdc',
'.nef': 'image/x-nikon-nef',
'.nrw': 'image/x-nikon-nrw',
'.orf': 'image/x-olympus-orf',
'.pef': 'image/x-pentax-pef',
'.raf': 'image/x-fuji-raf',
'.raw': 'image/x-panasonic-raw',
'.rw2': 'image/x-panasonic-rw2',
'.rwl': 'image/x-leica-rwl',
'.sr2': 'image/x-sony-sr2',
'.srf': 'image/x-sony-srf',
'.srw': 'image/x-samsung-srw',
'.x3f': 'image/x-sigma-x3f',
}
/**
+14
View File
@@ -92,6 +92,20 @@ describe('getProxyType', () => {
expect(getProxyType(null, 'design.psd')).toBe('image')
})
it('should detect image proxyType for RAW camera files', () => {
// RAW files typically upload with application/octet-stream or null mediaType
expect(getProxyType('application/octet-stream', 'photo.cr2')).toBe('image')
expect(getProxyType('application/octet-stream', 'photo.arw')).toBe('image')
expect(getProxyType(null, 'photo.dng')).toBe('image')
expect(getProxyType(null, 'photo.nef')).toBe('image')
expect(getProxyType(null, 'photo.raf')).toBe('image')
expect(getProxyType(null, 'photo.cr3')).toBe('image')
expect(getProxyType(null, 'photo.rw2')).toBe('image')
// Case-insensitive
expect(getProxyType(null, 'PHOTO.ARW')).toBe('image')
expect(getProxyType(null, 'Photo.DNG')).toBe('image')
})
it('should detect video proxyType', () => {
expect(getProxyType('video/mp4', 'clip.mp4')).toBe('video')
})
+8 -1
View File
@@ -1,3 +1,5 @@
import { isRawImage } from './raw'
export type ProxyType = 'image' | 'video' | 'audio' | 'pdf'
export function isOfficeDocument(mediaType?: string | null, filename?: string | null): boolean {
@@ -73,7 +75,12 @@ export function getProxyType(
const lowerMediaType = mediaType?.toLowerCase() || ''
const lowerFilename = filename?.toLowerCase() || ''
if (lowerMediaType.startsWith('image/') || lowerFilename.endsWith('.psd')) return 'image'
if (
lowerMediaType.startsWith('image/') ||
lowerFilename.endsWith('.psd') ||
isRawImage(lowerFilename)
)
return 'image'
if (lowerMediaType.startsWith('video/')) return 'video'
if (lowerMediaType.startsWith('audio/')) return 'audio'
+68
View File
@@ -0,0 +1,68 @@
import { describe, it, expect } from 'vitest'
import { isRawImage } from './raw'
describe('isRawImage', () => {
it('recognizes common RAW extensions (lowercase)', () => {
expect(isRawImage('photo.cr2')).toBe(true)
expect(isRawImage('photo.arw')).toBe(true)
expect(isRawImage('photo.dng')).toBe(true)
expect(isRawImage('photo.nef')).toBe(true)
expect(isRawImage('photo.raf')).toBe(true)
expect(isRawImage('photo.rw2')).toBe(true)
expect(isRawImage('photo.orf')).toBe(true)
expect(isRawImage('photo.pef')).toBe(true)
expect(isRawImage('photo.cr3')).toBe(true)
expect(isRawImage('photo.x3f')).toBe(true)
expect(isRawImage('photo.raw')).toBe(true)
expect(isRawImage('photo.3fr')).toBe(true)
expect(isRawImage('photo.iiq')).toBe(true)
expect(isRawImage('photo.kdc')).toBe(true)
expect(isRawImage('photo.dcr')).toBe(true)
expect(isRawImage('photo.erf')).toBe(true)
expect(isRawImage('photo.fff')).toBe(true)
expect(isRawImage('photo.crw')).toBe(true)
expect(isRawImage('photo.nrw')).toBe(true)
expect(isRawImage('photo.rwl')).toBe(true)
expect(isRawImage('photo.sr2')).toBe(true)
expect(isRawImage('photo.srf')).toBe(true)
expect(isRawImage('photo.srw')).toBe(true)
})
it('recognizes RAW extensions case-insensitively', () => {
expect(isRawImage('photo.CR2')).toBe(true)
expect(isRawImage('IMG_1234.ARW')).toBe(true)
expect(isRawImage('test.DNG')).toBe(true)
expect(isRawImage('photo.Nef')).toBe(true)
})
it('rejects non-RAW image files', () => {
expect(isRawImage('photo.jpg')).toBe(false)
expect(isRawImage('photo.png')).toBe(false)
expect(isRawImage('photo.webp')).toBe(false)
expect(isRawImage('photo.gif')).toBe(false)
expect(isRawImage('photo.svg')).toBe(false)
})
it('rejects non-image files', () => {
expect(isRawImage('design.psd')).toBe(false)
expect(isRawImage('video.mp4')).toBe(false)
expect(isRawImage('doc.pdf')).toBe(false)
expect(isRawImage('readme.txt')).toBe(false)
})
it('rejects files without extensions', () => {
expect(isRawImage('photo')).toBe(false)
expect(isRawImage('rawfile')).toBe(false)
})
it('handles filenames with multiple dots', () => {
expect(isRawImage('my.photo.cr2')).toBe(true)
expect(isRawImage('backup.2024.dng')).toBe(true)
expect(isRawImage('my.photo.jpg')).toBe(false)
})
it('handles paths with directories', () => {
expect(isRawImage('/home/user/photos/photo.cr2')).toBe(true)
expect(isRawImage('C:\\Users\\photos\\photo.arw')).toBe(true)
})
})
+37
View File
@@ -0,0 +1,37 @@
/**
* Centralized set of recognized camera RAW file extensions.
* Used only for routing files into the ExifTool extraction path.
*/
const RAW_EXTENSIONS: ReadonlySet<string> = new Set([
'.3fr',
'.arw',
'.cr2',
'.cr3',
'.crw',
'.dcr',
'.dng',
'.erf',
'.fff',
'.iiq',
'.kdc',
'.nef',
'.nrw',
'.orf',
'.pef',
'.raf',
'.raw',
'.rw2',
'.rwl',
'.sr2',
'.srf',
'.srw',
'.x3f',
])
/** Check whether a filename has a recognized camera RAW extension. */
export function isRawImage(filename: string): boolean {
const dotIndex = filename.lastIndexOf('.')
if (dotIndex === -1) return false
const ext = filename.slice(dotIndex).toLowerCase()
return RAW_EXTENSIONS.has(ext)
}
+1 -1
View File
@@ -323,7 +323,7 @@ export class WatermarkService {
if (isVideo || isImage) {
mediaAssetIds.push(asset.id)
}
} else if (asset.type === 'folder') {
} else if (asset.type === 'folder' || asset.type === 'version_stack') {
folderIds.push(asset.id)
} else if (asset.type === 'symlink' && asset.targetId) {
// Resolve nested symlinks (e.g. folders that contain their own symlinks)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/db",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -0,0 +1,16 @@
-- Migrate videoStrategy 'all' / 'full' to 'multi'
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoStrategy}', '"multi"')
WHERE "settings"->'transcode'->>'videoStrategy' IN ('all', 'full');
-- Normalize legacy videoStrategy values 'single' / 'disable' to 'best_match'
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoStrategy}', '"best_match"')
WHERE "settings"->'transcode'->>'videoStrategy' IN ('single', 'disable');
-- Backfill default videoResolutions where transcode settings exist without videoResolutions
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoResolutions}', '["480p", "720p", "1080p", "1440p", "2160p"]'::jsonb)
WHERE "settings" IS NOT NULL
AND jsonb_typeof("settings"->'transcode') = 'object'
AND NOT ("settings"->'transcode' ? 'videoResolutions');
@@ -0,0 +1,2 @@
-- AlterEnum
ALTER TYPE "AssetStatus" ADD VALUE 'failed';
+1
View File
@@ -40,6 +40,7 @@ enum AssetStatus {
uploaded
processing
processed
failed
trashed
pending_purge
}
+1
View File
@@ -36,6 +36,7 @@ export const AssetStatus = {
uploaded: 'uploaded',
processing: 'processing',
processed: 'processed',
failed: 'failed',
trashed: 'trashed',
pending_purge: 'pending_purge'
} as const
File diff suppressed because one or more lines are too long
+21 -1
View File
@@ -48,7 +48,8 @@ declare global {
// ----------------------------------------------------------------------
// Team Settings
// ----------------------------------------------------------------------
export type VideoTranscodeStrategy = 'best_match' | 'all'
export type VideoTranscodeStrategy = 'best_match' | 'multi' | 'all'
export type VideoResolutionLadder = '480p' | '720p' | '1080p' | '1440p' | '2160p'
export type HardwareAcceleration = 'off' | 'auto'
export type HdrType = 'pq' | 'hlg' | 'dovi_p5' | 'dovi_p8' | 'sdr'
@@ -57,10 +58,15 @@ declare global {
// ----------------------------------------------------------------------
export type CollectionFilter = import('@shumai/dtos').CollectionFilter
export type HlsResolutionLadder = '480p' | '720p' | '1080p' | '1440p' | '2160p'
export interface TranscodeSettings {
videoStrategy: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
export interface MediaProviderConfig {
@@ -164,6 +170,8 @@ declare global {
isHdr?: boolean
hdrType?: HdrType
dvProfile?: number
videoStreamIndex?: number
audioStreamIndex?: number
format: unknown
}
@@ -178,6 +186,12 @@ declare global {
url?: string
}
export interface HlsInfo {
key: string
url?: string
resolutions: VideoTranscode[]
}
export interface MediaInfo {
duration: number
filesize: number
@@ -193,6 +207,9 @@ declare global {
finishedAt: string
metadata: Metadata | null
original: OriginalInfo | null
isHls?: boolean
hls?: HlsInfo
error?: string
}
// ----------------------------------------------------------------------
@@ -336,11 +353,14 @@ declare global {
// ----------------------------------------------------------------------
export interface TaskSpec {
videoStrategy?: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
thumbnail?: boolean
sprite?: boolean
poster?: boolean
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
export interface AgentTaskPayload {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/dtos",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
+22
View File
@@ -105,6 +105,7 @@ export const assetInfoSchema = z.object({
width: z.number(),
height: z.number(),
size: z.number(),
resolution: z.string().optional(),
hdr: z.boolean().optional(),
}),
)
@@ -123,8 +124,28 @@ export const assetInfoSchema = z.object({
.optional(),
videoPreview: z.object({ url: z.string(), key: z.string().optional() }).optional(),
pdfTranscode: z.object({ url: z.string(), key: z.string().optional() }).optional(),
isHls: z.boolean().optional(),
hls: z
.object({
key: z.string(),
url: z.string().optional(),
resolutions: z
.array(
z.object({
width: z.number(),
height: z.number(),
resolution: z.string().optional(),
hdr: z.boolean().optional(),
key: z.string().optional(),
url: z.string().optional(),
}),
)
.optional(),
})
.optional(),
proxyType: z.enum(['image', 'video', 'audio', 'pdf']).nullable().optional(),
metadata: mediaMetadataSchema.optional(),
error: z.string().optional(),
})
.optional(),
ancestorFolders: z.array(ancestorFolderSchema).optional(),
@@ -338,6 +359,7 @@ export interface VideoTranscode {
width: number
height: number
size: number
resolution?: string
hdr?: boolean
}
+45
View File
@@ -69,22 +69,56 @@ export const listMembersQuerySchema = z.object({
export const VideoTranscodeStrategy = {
// eslint-disable-next-line @typescript-eslint/naming-convention
best_match: 'best_match',
multi: 'multi',
all: 'all',
} as const
export type VideoTranscodeStrategy =
(typeof VideoTranscodeStrategy)[keyof typeof VideoTranscodeStrategy]
export const VideoResolutionLadder = {
r480p: '480p',
r720p: '720p',
r1080p: '1080p',
r1440p: '1440p',
r2160p: '2160p',
} as const
export type VideoResolutionLadder =
(typeof VideoResolutionLadder)[keyof typeof VideoResolutionLadder]
export const DEFAULT_VIDEO_RESOLUTIONS: VideoResolutionLadder[] = [
'480p',
'720p',
'1080p',
'1440p',
'2160p',
]
export const HardwareAcceleration = {
off: 'off',
auto: 'auto',
} as const
export type HardwareAcceleration = (typeof HardwareAcceleration)[keyof typeof HardwareAcceleration]
export const HlsResolutionLadder = {
r480p: '480p',
r720p: '720p',
r1080p: '1080p',
r1440p: '1440p',
r2160p: '2160p',
} as const
export type HlsResolutionLadder = (typeof HlsResolutionLadder)[keyof typeof HlsResolutionLadder]
export const DEFAULT_HLS_RESOLUTIONS: HlsResolutionLadder[] = ['480p', '720p', '1080p']
export const updateTeamSettingsRequestSchema = z.union([
z.object({
key: z.literal('transcode.videoStrategy'),
value: z.nativeEnum(VideoTranscodeStrategy),
}),
z.object({
key: z.literal('transcode.videoResolutions'),
value: z.array(z.nativeEnum(VideoResolutionLadder)),
}),
z.object({
key: z.literal('transcode.hardwareAcceleration'),
value: z.nativeEnum(HardwareAcceleration),
@@ -93,6 +127,14 @@ export const updateTeamSettingsRequestSchema = z.union([
key: z.literal('transcode.threads'),
value: z.number().int().min(0).max(32),
}),
z.object({
key: z.literal('transcode.hlsEnabled'),
value: z.boolean(),
}),
z.object({
key: z.literal('transcode.hlsResolutions'),
value: z.array(z.nativeEnum(HlsResolutionLadder)),
}),
z.object({
key: z.literal('appearance.hideAgent'),
value: z.boolean(),
@@ -107,8 +149,11 @@ export interface TeamAppearanceSettings {
export interface TeamSettingsResponse {
transcode?: {
videoStrategy?: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
appearance?: TeamAppearanceSettings
semanticSearchEnabled?: boolean
Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/e2e",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"dependencies": {
@@ -214,5 +214,86 @@ describe.each(['local', 'temporal'] as const)(
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
it('should run transcodeMedia workflow for a RAW image asset (.dng) successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E RAW Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E RAW Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.dng',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.dng',
type: 'file',
status: 'uploaded',
mediaType: 'image/x-adobe-dng',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.dng
const dngPath = path.join(fixturesDir, 'test.dng')
const dngBuffer = fs.readFileSync(dngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.dng',
dngBuffer,
dngBuffer.length,
'image/x-adobe-dng',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E RAW Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
},
)
+957 -1
View File
@@ -1,17 +1,63 @@
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest'
import { prisma, AssetStatus } from '@shumai/db'
import { prisma, AssetStatus, AssetType } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { workflowService, TaskQueueTranscode } from '@shumai/workflow-core'
import { initTranscodeWorkflows } from '@shumai/transcode'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { fileURLToPath } from 'url'
import * as path from 'path'
import * as fs from 'fs'
import * as os from 'os'
import { execFile } from 'child_process'
import { promisify } from 'util'
const execFileAsync = promisify(execFile)
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
async function sampleFirstFrameRgb(
buffer: Buffer,
ext: string = 'mp4',
): Promise<{ r: number; g: number; b: number }> {
const tmpFile = path.join(
os.tmpdir(),
`sample-${Date.now()}-${Math.random().toString(36).slice(2)}.${ext}`,
)
try {
fs.writeFileSync(tmpFile, buffer)
const proc = await execFileAsync(
'ffmpeg',
[
'-y',
'-i',
tmpFile,
'-vframes',
'1',
'-s',
'1x1',
'-f',
'rawvideo',
'-pix_fmt',
'rgb24',
'pipe:1',
],
{ encoding: 'buffer' },
)
return {
r: proc.stdout[0],
g: proc.stdout[1],
b: proc.stdout[2],
}
} finally {
if (fs.existsSync(tmpFile)) {
fs.unlinkSync(tmpFile)
}
}
}
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - transcodeVideoWorkflow (executor: %s)',
(mode) => {
@@ -221,5 +267,915 @@ describe.each(['local', 'temporal'] as const)(
// Audio proxy key should end with -audio-proxy.mp4
expect(mediaInfo.videoTranscodes[0].key).toContain('-audio-proxy.mp4')
}, 50000)
it('should run transcodeMedia workflow with HLS enabled and serve playlists for version stacks and symlinks', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: {
name: 'E2E HLS Video Transcode Team',
settings: {
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
const project = await prisma.project.create({
data: { name: 'E2E HLS Video Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-hls.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-hls.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture (small-480p.mp4)
const mp4Path = path.join(fixturesDir, 'small-480p.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-hls.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E HLS Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
// 5. Verification of HLS generation
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as PrismaJson.MediaInfo
expect(mediaInfo.isHls).toBe(true)
expect(mediaInfo.hls).toBeDefined()
expect(mediaInfo.hls?.key).toBe('projects/e2e/hls/master.m3u8')
expect(mediaInfo.hls?.resolutions).toBeDefined()
expect(mediaInfo.hls?.resolutions?.length).toBeGreaterThan(0)
expect(mediaInfo.hls?.resolutions?.[0]?.resolution).toBe('480p')
// Verify master playlist in S3
const masterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/hls/master.m3u8',
)
const masterPlaylist = masterObj.buffer.toString('utf-8')
expect(masterPlaylist).toContain('#EXTM3U')
expect(masterPlaylist).toContain('480p/index.m3u8')
// Verify variant playlist in S3
const variantObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/hls/480p/index.m3u8',
)
expect(variantObj.buffer.toString('utf-8')).toContain('#EXTM3U')
// 6. Test Version Stack & Symlink Resolution
const stack = await prisma.asset.create({
data: {
name: 'video-hls-stack',
type: AssetType.version_stack,
status: AssetStatus.processed,
projectId: project.id,
},
})
await prisma.asset.update({
where: { id: asset.id },
data: {
parentId: stack.id,
sortIndex: 'a',
},
})
const symlink = await prisma.asset.create({
data: {
name: 'video-hls-symlink',
type: AssetType.symlink,
status: AssetStatus.processed,
projectId: project.id,
targetId: stack.id,
},
})
// Calling getHlsMasterPlaylist with stack ID
const stackMaster = await assetService.getHlsMasterPlaylist({ assetId: stack.id })
expect(stackMaster).toContain('#EXTM3U')
expect(stackMaster).toContain('480p/index.m3u8')
// Calling getHlsMasterPlaylist with symlink ID
const symlinkMaster = await assetService.getHlsMasterPlaylist({ assetId: symlink.id })
expect(symlinkMaster).toContain('#EXTM3U')
expect(symlinkMaster).toContain('480p/index.m3u8')
// Calling getHlsVariantPlaylist with symlink ID
const symlinkVariant = await assetService.getHlsVariantPlaylist({
assetId: symlink.id,
resolution: '480p',
})
expect(symlinkVariant).toContain('#EXTM3U')
}, 60000)
it('Case 1: should ignore attached picture cover art (Red) and select primary video stream (Green)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Cover Art Video Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Cover Art Video Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case1/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case1-'))
const videoPath = path.join(tmpDir, 'case1.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=green:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-pix_fmt:v:0',
'yuv420p',
'-c:v:1',
'mjpeg',
'-pix_fmt:v:1',
'yuvj420p',
'-c:a',
'aac',
'-disposition:v:1',
'attached_pic',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case1/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
sprite: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
sprite: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
expect(mediaInfo.metadata.videoStreamIndex).toBe(0)
// Verify poster is Green (not Red attached pic)
expect(mediaInfo.poster?.key).toBeDefined()
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.g).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.b).toBeLessThan(50)
// Verify MP4 proxy is Green
expect(mediaInfo.videoTranscodes.length).toBeGreaterThan(0)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.g).toBeGreaterThan(100)
expect(videoColor.r).toBeLessThan(50)
expect(videoColor.b).toBeLessThan(50)
// Verify sprite sheet exists
expect(mediaInfo.sprite?.key).toBeDefined()
await s3Service.headObject('shumai-e2e-test-bucket-transcode', mediaInfo.sprite.key)
}, 60000)
it('Case 2: should select higher bitrate stream (Blue) when both streams have default flag', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Bitrate Comparison Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Bitrate Comparison Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case2/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case2-'))
const videoPath = path.join(tmpDir, 'case2.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-b:v:0',
'200k',
'-minrate:v:0',
'200k',
'-maxrate:v:0',
'200k',
'-bufsize:v:0',
'200k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'2000k',
'-minrate:v:1',
'2000k',
'-maxrate:v:1',
'2000k',
'-bufsize:v:1',
'2000k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'default',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case2/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 1 (Blue, 2000k) should be selected over Stream 0 (Red, 200k)
expect(mediaInfo.metadata.videoStreamIndex).toBe(1)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.b).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.g).toBeLessThan(50)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.b).toBeGreaterThan(100)
expect(videoColor.r).toBeLessThan(50)
expect(videoColor.g).toBeLessThan(50)
}, 60000)
it('Case 3: should select stream with default flag (Yellow) over higher bitrate stream without default flag (Blue)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Default Flag Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Default Flag Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case3/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case3-'))
const videoPath = path.join(tmpDir, 'case3.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=yellow:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-b:v:0',
'200k',
'-minrate:v:0',
'200k',
'-maxrate:v:0',
'200k',
'-bufsize:v:0',
'200k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'2000k',
'-minrate:v:1',
'2000k',
'-maxrate:v:1',
'2000k',
'-bufsize:v:1',
'2000k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'0',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case3/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 0 (Yellow, default: 1) should win over Stream 1 (Blue, 10x higher bitrate but default: 0)
expect(mediaInfo.metadata.videoStreamIndex).toBe(0)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.r).toBeGreaterThan(100)
expect(posterColor.g).toBeGreaterThan(100)
expect(posterColor.b).toBeLessThan(50)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.r).toBeGreaterThan(100)
expect(videoColor.g).toBeGreaterThan(100)
expect(videoColor.b).toBeLessThan(50)
}, 60000)
it('Case 4: should handle audio-only MP4 with cover art by downgrading to audio proxy (catches audio bug)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Audio Cover Art Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Audio Cover Art Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case4/audio-cover.mp4' },
})
// Uploaded as video/mp4 because of .mp4 container, even though it only has audio + cover art
const asset = await prisma.asset.create({
data: {
name: 'audio-cover.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case4-'))
const videoPath = path.join(tmpDir, 'case4.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-f',
'lavfi',
'-i',
'color=c=magenta:s=320x240:d=2',
'-map',
'0:a',
'-map',
'1:v',
'-c:a',
'aac',
'-c:v',
'mjpeg',
'-pix_fmt:v:0',
'yuvj420p',
'-disposition:v:0',
'attached_pic',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case4/audio-cover.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
// Spec requested poster, sprite, videoStrategy as video upload would
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
sprite: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
// Must complete without error (previously failed trying to generate poster/sprite for audio)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
poster?: unknown
sprite?: unknown
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('audio')
expect(mediaInfo.poster).toBeUndefined()
expect(mediaInfo.sprite).toBeUndefined()
expect(mediaInfo.videoTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.videoTranscodes[0].key).toContain('-audio-proxy.mp4')
await s3Service.headObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
}, 60000)
it('Case 5: should rank 3 video streams with HLS enabled and select highest bitrate stream (Blue)', async () => {
const team = await prisma.team.create({
data: {
name: 'E2E Multi Stream HLS Team',
settings: {
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
const project = await prisma.project.create({
data: { name: 'E2E Multi Stream HLS Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case5/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case5-'))
const videoPath = path.join(tmpDir, 'case5.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=green:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:v',
'-map',
'3:a',
'-c:v:0',
'libx264',
'-b:v:0',
'100k',
'-minrate:v:0',
'100k',
'-maxrate:v:0',
'100k',
'-bufsize:v:0',
'100k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'500k',
'-minrate:v:1',
'500k',
'-maxrate:v:1',
'500k',
'-bufsize:v:1',
'500k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'default',
'-c:v:2',
'libx264',
'-b:v:2',
'1500k',
'-minrate:v:2',
'1500k',
'-maxrate:v:2',
'1500k',
'-bufsize:v:2',
'1500k',
'-nal-hrd',
'cbr',
'-disposition:v:2',
'default',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case5/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
isHls: boolean
hls: { key: string; resolutions: { resolution: string }[] }
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 2 (Blue, 1500k) should be selected
expect(mediaInfo.metadata.videoStreamIndex).toBe(2)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.b).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.g).toBeLessThan(50)
expect(mediaInfo.isHls).toBe(true)
const masterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.hls.key,
)
expect(masterObj.buffer.toString('utf-8')).toContain('480p/index.m3u8')
}, 60000)
it('should transition both task and asset to failed status with media.error when transcoding fails', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Failed Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Failed Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/corrupt-video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'corrupt-video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const corruptBuffer = Buffer.from('not a valid mp4 media stream at all')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/corrupt-video.mp4',
corruptBuffer,
corruptBuffer.length,
'video/mp4',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
},
},
},
})
await expect(workflowService.executeWait(task, 45000)).rejects.toThrow()
const failedTask = await prisma.workflowTask.findUnique({
where: { id: task.id },
})
expect(failedTask?.status).toBe('failed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.failed)
const media = updatedAsset?.media as PrismaJson.MediaInfo
expect(media?.error).toBeDefined()
expect(media?.error?.length).toBeGreaterThan(0)
}, 45000)
},
)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/tableprinter",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -217,6 +217,86 @@ describe('Transcode Activities', () => {
)
})
it('should downgrade proxyType to audio when getVideoInfo throws No video stream found but audio exists', async () => {
const asset = await prisma.asset.create({
data: { name: 'audio-with-art.mp4', type: 'file', status: 'uploaded' },
})
vi.mocked(transcodeService.getVideoInfo).mockRejectedValueOnce(
new Error('No video stream found'),
)
vi.mocked(transcodeService.getAudioInfo).mockResolvedValueOnce({
originalWidth: 0,
originalHeight: 0,
duration: 42,
bitRate: 192000,
frameRate: 0,
totalFrames: 0,
hasAudio: true,
audioCodec: 'aac',
audioChannels: 2,
audioSampleRate: 44100,
audioBitDepth: 16,
audioStreamIndex: 0,
mimeType: 'audio/mp4',
})
const result = await getMediaInfoActivity({
assetId: asset.id,
filePath: '/tmp/audio-with-art.mp4',
proxyType: 'video',
mediaType: 'video/mp4',
})
expect(transcodeService.getVideoInfo).toHaveBeenCalledWith('/tmp/audio-with-art.mp4')
expect(transcodeService.getAudioInfo).toHaveBeenCalledWith('/tmp/audio-with-art.mp4')
expect(result.proxyType).toBe('audio')
expect(result.duration).toBe(42)
expect(result.metadata?.hasAudio).toBe(true)
expect(result.metadata?.audioStreamIndex).toBe(0)
expect(result.metadata?.videoStreamIndex).toBeUndefined()
expect(metadataService.updateAssetMetadata).toHaveBeenCalledWith(
asset.id,
expect.arrayContaining([
{ key: 'file_type', value: 'audio' },
{ key: 'proxy_type', value: 'audio' },
{ key: 'duration', value: 42 },
]),
true,
)
})
it('should populate videoStreamIndex and audioStreamIndex in mediaInfo.metadata', async () => {
const asset = await prisma.asset.create({
data: { name: 'dual-stream.mp4', type: 'file', status: 'uploaded' },
})
vi.mocked(transcodeService.getVideoInfo).mockResolvedValueOnce({
originalWidth: 1920,
originalHeight: 1080,
duration: 15,
bitRate: 2000000,
frameRate: 30,
totalFrames: 450,
hasAudio: true,
videoCodec: 'h264',
audioCodec: 'aac',
videoStreamIndex: 1,
audioStreamIndex: 2,
mimeType: 'video/mp4',
})
const result = await getMediaInfoActivity({
assetId: asset.id,
filePath: '/tmp/dual-stream.mp4',
proxyType: 'video',
mediaType: 'video/mp4',
})
expect(result.metadata?.videoStreamIndex).toBe(1)
expect(result.metadata?.audioStreamIndex).toBe(2)
})
it('should call getImageInfo and set file_type to image', async () => {
const asset = await prisma.asset.create({
data: { name: 'i.png', type: 'file', status: 'uploaded' },
@@ -703,6 +783,28 @@ describe('Transcode Activities', () => {
)
})
it('transcodeVideoActivity forwards streamIndex and audioStreamIndex to transcodeService', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
await transcodeVideoActivity({
assetKey: 'v.mp4',
filePath: '/tmp/v.mp4',
videoSpec: { resolution: '720p', width: 1280, height: 720 },
duration: 10,
originalFps: 30,
streamIndex: 2,
audioStreamIndex: 1,
})
expect(transcodeService.transcodeVideo).toHaveBeenCalledWith(
expect.objectContaining({
inputFile: '/tmp/v.mp4',
streamIndex: 2,
audioStreamIndex: 1,
}),
)
})
it('transcodeVideoActivity appends -hdr.mp4 suffix and forwards HDR metadata', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
@@ -946,6 +1048,44 @@ describe('Transcode Activities', () => {
hdrType: 'pq',
colorTransfer: 'smpte2084',
},
undefined,
)
})
it('generateSpriteActivity forwards videoStreamIndex to transcodeService.generateSprite', async () => {
vi.mocked(transcodeService.generateSprite).mockImplementation(
async (_in, spriteOut, posterOut) => {
const fs = await import('fs')
fs.writeFileSync(spriteOut, 'fake-sprite')
fs.writeFileSync(posterOut, 'fake-poster')
},
)
await generateSpriteActivity({
assetKey: 'files/asset-1/video.mp4',
filePath: '/tmp/video.mp4',
mediaInfo: {
duration: 10,
metadata: {
videoStreamIndex: 1,
},
} as unknown as PrismaJson.MediaInfo,
spriteSpec: { key: 'files/asset-1/sprite.webp', frames: 100, tileX: 10, tileY: 10 },
posterSpec: { key: 'files/asset-1/poster.webp' },
})
expect(transcodeService.generateSprite).toHaveBeenCalledWith(
'/tmp/video.mp4',
expect.any(String),
expect.any(String),
10,
undefined,
{
isHdr: undefined,
hdrType: undefined,
colorTransfer: undefined,
},
1,
)
})
+307 -48
View File
@@ -1,10 +1,16 @@
import { AssetStatus, prisma, WorkflowTaskType, WorkflowTaskStatus } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService, type HdrType } from '@shumai/core/src/transcode/transcode'
import {
transcodeService,
buildHlsMasterPlaylist,
calculateEffectiveBitrateBps,
type HdrType,
} from '@shumai/core/src/transcode/transcode'
import { metadataService } from '@shumai/core/src/metadata/metadata'
import { getDerivedArtifactDirectory, stemFromKey } from '@shumai/core/src/utils/filename'
import { gotenbergService } from '@shumai/core/src/gotenberg/gotenberg'
import { parseCsvContent } from '@shumai/core/src/transcode/transcode'
import { resolutionToDimensions } from '../workflows/transcode-utils'
import {
getProxyType,
isCsvDocument,
@@ -124,54 +130,110 @@ export async function getMediaInfoActivity(params: {
}
if (isVideo) {
const info = await transcodeService.getVideoInfo(params.filePath)
mediaInfo.duration = info.duration
mediaInfo.frames = info.totalFrames
mediaInfo.metadata = {
originalWidth: info.originalWidth,
originalHeight: info.originalHeight,
duration: info.duration,
bitRate: info.bitRate,
videoBitRate: info.videoBitRate,
frameRate: info.frameRate,
totalFrames: info.totalFrames,
startTimecode: info.startTimecode || '00:00:00:00',
hasAudio: info.hasAudio,
videoCodec: info.videoCodec,
audioCodec: info.audioCodec,
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
colorTransfer: info.colorTransfer,
colorPrimaries: info.colorPrimaries,
colorSpace: info.colorSpace,
isHdr: info.isHdr,
hdrType: info.hdrType,
dvProfile: info.dvProfile,
rotation: info.rotation,
format: {},
let isDowngradedToAudio = false
let videoInfo: Awaited<ReturnType<typeof transcodeService.getVideoInfo>> | null = null
try {
videoInfo = await transcodeService.getVideoInfo(params.filePath)
} catch (err) {
if (err instanceof Error && err.message.includes('No video stream found')) {
try {
const audioInfo = await transcodeService.getAudioInfo(params.filePath)
isDowngradedToAudio = true
mediaInfo.proxyType = 'audio'
mediaInfo.duration = audioInfo.duration
mediaInfo.metadata = {
originalWidth: 0,
originalHeight: 0,
duration: audioInfo.duration,
bitRate: audioInfo.bitRate,
frameRate: 0,
totalFrames: 0,
startTimecode: '00:00:00:00',
hasAudio: true,
videoCodec: undefined,
audioCodec: audioInfo.audioCodec,
audioChannels: audioInfo.audioChannels,
audioSampleRate: audioInfo.audioSampleRate,
audioBitDepth: audioInfo.audioBitDepth,
audioStreamIndex: audioInfo.audioStreamIndex,
format: {},
}
const ftIdx = metadataUpdates.findIndex((u) => u.key === 'file_type')
if (ftIdx !== -1) metadataUpdates[ftIdx].value = 'audio'
const ptIdx = metadataUpdates.findIndex((u) => u.key === 'proxy_type')
if (ptIdx !== -1) metadataUpdates[ptIdx].value = 'audio'
metadataUpdates.push(
{ key: 'duration', value: audioInfo.duration },
{ key: 'bitRate', value: audioInfo.bitRate / 1000 },
)
if (audioInfo.audioCodec)
metadataUpdates.push({ key: 'audio_codec', value: audioInfo.audioCodec })
if (audioInfo.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: audioInfo.audioChannels })
if (audioInfo.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: audioInfo.audioSampleRate })
if (audioInfo.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: audioInfo.audioBitDepth })
} catch {
throw err
}
} else {
throw err
}
}
if (!isDowngradedToAudio && videoInfo) {
const info = videoInfo
mediaInfo.duration = info.duration
mediaInfo.frames = info.totalFrames
mediaInfo.metadata = {
originalWidth: info.originalWidth,
originalHeight: info.originalHeight,
duration: info.duration,
bitRate: info.bitRate,
videoBitRate: info.videoBitRate,
frameRate: info.frameRate,
totalFrames: info.totalFrames,
startTimecode: info.startTimecode || '00:00:00:00',
hasAudio: info.hasAudio,
videoCodec: info.videoCodec,
audioCodec: info.audioCodec,
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
colorTransfer: info.colorTransfer,
colorPrimaries: info.colorPrimaries,
colorSpace: info.colorSpace,
isHdr: info.isHdr,
hdrType: info.hdrType,
dvProfile: info.dvProfile,
rotation: info.rotation,
videoStreamIndex: info.videoStreamIndex,
audioStreamIndex: info.audioStreamIndex,
format: {},
}
metadataUpdates.push(
{ key: 'resolution_width', value: info.originalWidth },
{ key: 'resolution_height', value: info.originalHeight },
{ key: 'duration', value: info.duration },
{ key: 'bitRate', value: info.bitRate / 1000 },
{ key: 'frame_rate', value: info.frameRate },
)
if (info.rotation !== undefined)
metadataUpdates.push({ key: 'rotation', value: info.rotation })
if (info.isHdr !== undefined)
metadataUpdates.push({ key: 'is_hdr', value: info.isHdr ? 1 : 0 })
if (info.colorTransfer)
metadataUpdates.push({ key: 'color_transfer', value: info.colorTransfer })
if (info.videoCodec) metadataUpdates.push({ key: 'video_codec', value: info.videoCodec })
if (info.audioCodec) metadataUpdates.push({ key: 'audio_codec', value: info.audioCodec })
if (info.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: info.audioChannels })
if (info.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: info.audioSampleRate })
if (info.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: info.audioBitDepth })
}
metadataUpdates.push(
{ key: 'resolution_width', value: info.originalWidth },
{ key: 'resolution_height', value: info.originalHeight },
{ key: 'duration', value: info.duration },
{ key: 'bitRate', value: info.bitRate / 1000 },
{ key: 'frame_rate', value: info.frameRate },
)
if (info.rotation !== undefined)
metadataUpdates.push({ key: 'rotation', value: info.rotation })
if (info.isHdr !== undefined)
metadataUpdates.push({ key: 'is_hdr', value: info.isHdr ? 1 : 0 })
if (info.colorTransfer)
metadataUpdates.push({ key: 'color_transfer', value: info.colorTransfer })
if (info.videoCodec) metadataUpdates.push({ key: 'video_codec', value: info.videoCodec })
if (info.audioCodec) metadataUpdates.push({ key: 'audio_codec', value: info.audioCodec })
if (info.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: info.audioChannels })
if (info.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: info.audioSampleRate })
if (info.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: info.audioBitDepth })
} else if (isAudio) {
const info = await transcodeService.getAudioInfo(params.filePath)
mediaInfo.duration = info.duration
@@ -189,6 +251,7 @@ export async function getMediaInfoActivity(params: {
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
audioStreamIndex: info.audioStreamIndex,
format: {},
}
metadataUpdates.push(
@@ -271,6 +334,8 @@ export interface VideoActivityParams {
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
streamIndex?: number
audioStreamIndex?: number
}
export async function transcodeVideoActivity(
@@ -332,6 +397,8 @@ export async function transcodeVideoActivity(
sourceColorTransfer: params.sourceColorTransfer,
sourceColorPrimaries: params.sourceColorPrimaries,
sourceColorSpace: params.sourceColorSpace,
streamIndex: params.streamIndex,
audioStreamIndex: params.audioStreamIndex,
})
const stat = fs.statSync(outputFile)
@@ -380,11 +447,197 @@ export async function transcodeVideoActivity(
}
}
export interface HlsActivityParams {
taskId: string
assetKey: string
filePath: string
ladders: PrismaJson.HlsResolutionLadder[]
originalWidth: number
originalHeight: number
duration: number
originalFps: number
hardwareAcceleration?: 'off' | 'auto'
sourceVideoBitrate?: number
threads?: number
sourceIsHdr?: boolean
sourceHdrType?: HdrType
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
streamIndex?: number
audioStreamIndex?: number
}
export async function transcodeHlsActivity(params: HlsActivityParams): Promise<PrismaJson.HlsInfo> {
const bucket = process.env.S3_BUCKET || 'shumai'
const assetDir = path.posix.dirname(params.assetKey)
const hlsDirKey = path.posix.join(assetDir, 'hls')
const masterKey = path.posix.join(hlsDirKey, 'master.m3u8')
const signal = getActivityCancellationSignal(params.taskId)
if (signal?.aborted) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
})
}
const hlsTmpDir = path.join(path.dirname(params.filePath), `hls-${ulid()}`)
try {
const renditionSpecs: Array<{
resolution: string
width: number
height: number
bitrateBps: number
isHdr?: boolean
}> = []
const mediaResolutions: PrismaJson.VideoTranscode[] = []
for (const ladder of params.ladders) {
if (signal?.aborted) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
})
}
const [width, height] = resolutionToDimensions(
ladder,
params.originalWidth,
params.originalHeight,
)
const targetBps = calculateEffectiveBitrateBps(
height,
width,
params.sourceVideoBitrate,
params.originalFps,
)
const ladderDir = path.join(hlsTmpDir, ladder)
await transcodeService.transcodeHlsRendition({
inputFile: params.filePath,
outputDir: ladderDir,
width,
height,
frameRate: params.originalFps,
hardwareAcceleration: params.hardwareAcceleration,
sourceVideoBitrate: params.sourceVideoBitrate,
threads: params.threads,
signal,
hdr: Boolean(params.sourceIsHdr),
sourceIsHdr: params.sourceIsHdr,
sourceHdrType: params.sourceHdrType,
sourceColorTransfer: params.sourceColorTransfer,
sourceColorPrimaries: params.sourceColorPrimaries,
sourceColorSpace: params.sourceColorSpace,
streamIndex: params.streamIndex,
audioStreamIndex: params.audioStreamIndex,
})
renditionSpecs.push({
resolution: ladder,
width,
height,
bitrateBps: targetBps,
isHdr: params.sourceIsHdr,
})
mediaResolutions.push({
resolution: ladder,
width,
height,
hdr: Boolean(params.sourceIsHdr),
})
await ensureAssetNotPurging(params.assetKey)
const files = fs.readdirSync(ladderDir)
for (const file of files) {
const localFilePath = path.join(ladderDir, file)
const stat = fs.statSync(localFilePath)
const s3Key = path.posix.join(hlsDirKey, ladder, file)
let contentType = 'application/octet-stream'
if (file.endsWith('.m3u8')) {
contentType = 'application/vnd.apple.mpegurl'
} else if (file.endsWith('.mp4')) {
contentType = 'video/mp4'
} else if (file.endsWith('.m4s')) {
contentType = 'video/iso.segment'
}
const stream = Bun.file(localFilePath).stream()
await s3Service.putObject(bucket, s3Key, stream, stat.size, contentType)
}
}
const masterContent = buildHlsMasterPlaylist(renditionSpecs)
const masterBuffer = Buffer.from(masterContent, 'utf-8')
await ensureAssetNotPurging(params.assetKey)
await s3Service.putObject(
bucket,
masterKey,
masterBuffer,
masterBuffer.length,
'application/vnd.apple.mpegurl',
)
return {
key: masterKey,
resolutions: mediaResolutions,
}
} catch (err) {
const { code, message } = getErrorDetails(err)
const lowerMsg = message.toLowerCase()
if (
signal?.aborted ||
code === 'ABORT_ERR' ||
message.includes('aborted') ||
lowerMsg.includes('abort')
) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
cause: err instanceof Error ? err : undefined,
})
}
if (
code === 'ENOENT' ||
lowerMsg.includes('enoent') ||
lowerMsg.includes('ffmpeg') ||
lowerMsg.includes('ffprobe') ||
lowerMsg.includes('spawn') ||
lowerMsg.includes('format') ||
lowerMsg.includes('no video stream found') ||
lowerMsg.includes('zscale') ||
lowerMsg.includes('tonemap')
) {
throw ApplicationFailure.create({
message: `HLS transcoding failed: ${message}`,
nonRetryable: true,
cause: err instanceof Error ? err : undefined,
})
}
throw err
} finally {
if (fs.existsSync(hlsTmpDir)) {
try {
fs.rmSync(hlsTmpDir, { recursive: true, force: true })
} catch {
// ignore cleanup error
}
}
}
}
export interface AudioActivityParams {
taskId?: string
assetKey: string
filePath: string
threads?: number
audioStreamIndex?: number
}
export async function transcodeAudioActivity(
@@ -419,6 +672,7 @@ export async function transcodeAudioActivity(
bitrate: '128k',
threads: params.threads,
signal,
audioStreamIndex: params.audioStreamIndex,
})
const stat = fs.statSync(outputFile)
@@ -590,6 +844,7 @@ export async function generateSpriteActivity(params: GenerateSpriteActivityParam
hdrType: params.mediaInfo.metadata?.hdrType,
colorTransfer: params.mediaInfo.metadata?.colorTransfer,
},
params.mediaInfo.metadata?.videoStreamIndex,
)
}
@@ -894,6 +1149,7 @@ export interface ExtractPosterActivityParams {
isHdr?: boolean
hdrType?: HdrType
colorTransfer?: string
streamIndex?: number
}
export async function extractPosterActivity(
@@ -925,6 +1181,7 @@ export async function extractPosterActivity(
hdrType: params.hdrType,
colorTransfer: params.colorTransfer,
signal,
streamIndex: params.streamIndex,
})
const posterBuffer = fs.readFileSync(posterFile)
@@ -1241,6 +1498,8 @@ export async function transcodeVideoChunkActivity(
params.startTime.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-t',
(params.endTime - params.startTime).toString(),
'-vf',
@@ -45,26 +45,37 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
},
}))
vi.mock('@shumai/core/src/transcode/transcode', () => ({
transcodeService: {
getVideoInfo: vi.fn(),
getImageInfo: vi.fn(),
createTempDir: vi.fn().mockReturnValue('/tmp'),
removeDir: vi.fn(),
renderSvgToPng: vi.fn().mockResolvedValue(Buffer.from('fake-overlay-png')),
downscaleImageToPng: vi.fn().mockResolvedValue({
buffer: Buffer.from('fake-block-png'),
width: 32,
height: 32,
}),
compositeOverlayToWebpFile: vi.fn().mockImplementation(async (_in, _overlay, out) => {
fs.writeFileSync(out, 'fake-webp')
}),
transcodeVideo: vi.fn().mockImplementation(async (params) => {
fs.writeFileSync(params.outputFile, 'fake-mp4')
}),
},
}))
vi.mock('@shumai/core/src/transcode/transcode', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/core/src/transcode/transcode')>()
return {
...actual,
transcodeService: {
...actual.transcodeService,
getVideoInfo: vi.fn(),
getImageInfo: vi.fn(),
createTempDir: vi.fn().mockReturnValue('/tmp'),
removeDir: vi.fn(),
renderSvgToPng: vi.fn().mockResolvedValue(Buffer.from('fake-overlay-png')),
downscaleImageToPng: vi.fn().mockResolvedValue({
buffer: Buffer.from('fake-block-png'),
width: 32,
height: 32,
}),
compositeOverlayToWebpFile: vi.fn().mockImplementation(async (_in, _overlay, out) => {
fs.writeFileSync(out, 'fake-webp')
}),
transcodeVideo: vi.fn().mockImplementation(async (params) => {
fs.writeFileSync(params.outputFile, 'fake-mp4')
}),
transcodeHlsRendition: vi.fn().mockImplementation(async (params) => {
fs.mkdirSync(params.outputDir, { recursive: true })
fs.writeFileSync(path.join(params.outputDir, 'init.mp4'), 'fake-init')
fs.writeFileSync(path.join(params.outputDir, 'segment_000.m4s'), 'fake-segment')
fs.writeFileSync(path.join(params.outputDir, 'index.m3u8'), 'fake-index')
}),
},
}
})
describe('Watermark Activities', () => {
setupTestDbHooks()
@@ -524,6 +535,89 @@ describe('Watermark Activities', () => {
fs.rmSync(sdrOut, { force: true })
fs.rmSync(hdrOut, { force: true })
})
it('produces watermarked HLS renditions and updates mediaInfo.hls when original is HLS', async () => {
const originalMedia: PrismaJson.MediaInfo = {
proxyType: 'video',
duration: 10,
filesize: 1000,
frames: 300,
videoTranscodes: [
{ key: 'files/asset/video-1080p.mp4', width: 1920, height: 1080, resolution: '1080p' },
],
imageTranscodes: [],
videoPreview: { width: 1920, height: 1080 },
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
hasAudio: false,
format: {},
bitRate: 1000,
},
original: { key: 'files/asset/raw.mp4', filesizeInBytes: 1000, codec: 'h264' },
finishedAt: new Date().toISOString(),
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
}
const storageKey = await prisma.storageKey.create({
data: { key: 'files/asset/raw.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'wm-hls.mp4',
type: 'file',
status: 'processed',
storageKeyId: storageKey.id,
mediaType: 'video/mp4',
media: originalMedia,
},
})
const config = await prisma.watermarkConfig.create({
data: {
hash: 'hls-hash-' + Math.random(),
config: {
blocks: [
{
id: 'b1',
type: 'text',
text: 'HLS Confidential',
opacity: 0.5,
size: 0.05,
color: '#FFFFFF',
x: 0.5,
y: 0.5,
rotation: 0,
},
],
},
},
})
const media = await transcodeWatermarkMediaActivity({
assetId: asset.id,
watermarkConfigId: config.id,
})
expect(media.isHls).toBe(true)
expect(media.hls).toBeDefined()
expect(media.hls?.key).toContain(`hls-watermark-${config.id}/master.m3u8`)
expect(media.hls?.resolutions).toHaveLength(1)
expect(media.hls?.resolutions?.[0]?.resolution).toBe('1080p')
expect(transcodeService.transcodeHlsRendition).toHaveBeenCalledWith(
expect.objectContaining({
width: 1920,
height: 1080,
}),
)
})
})
describe('completeWatermarkFileActivity', () => {
+120 -1
View File
@@ -1,7 +1,11 @@
import { prisma, WatermarkFileStatus } from '@shumai/db'
import '@shumai/db/src/prisma-json-types'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService } from '@shumai/core/src/transcode/transcode'
import {
transcodeService,
buildHlsMasterPlaylist,
calculateEffectiveBitrateBps,
} from '@shumai/core/src/transcode/transcode'
import {
generateWatermarkSvg,
RenderBlockImageData,
@@ -366,6 +370,8 @@ export async function transcodeWatermarkMediaActivity(
sourceColorTransfer: colorTransfer ?? originalMedia?.metadata?.colorTransfer,
sourceColorPrimaries: colorPrimaries ?? originalMedia?.metadata?.colorPrimaries,
sourceColorSpace: colorSpace ?? originalMedia?.metadata?.colorSpace,
streamIndex: originalMedia?.metadata?.videoStreamIndex,
audioStreamIndex: originalMedia?.metadata?.audioStreamIndex,
})
const stat = fs.statSync(outFilePath)
@@ -388,6 +394,119 @@ export async function transcodeWatermarkMediaActivity(
hdr: isVtHdr,
})
}
if (originalMedia?.isHls && originalMedia?.hls?.resolutions?.length) {
const hlsWatermarkDirKey = path.posix.join(
path.posix.dirname(assetKey),
`hls-watermark-${params.watermarkConfigId}`,
)
const masterKey = path.posix.join(hlsWatermarkDirKey, 'master.m3u8')
const hlsTmpDir = path.join(tmpDir, `hls-watermark-${params.watermarkConfigId}`)
fs.mkdirSync(hlsTmpDir, { recursive: true })
const renditionSpecs: Array<{
resolution: string
width: number
height: number
bitrateBps: number
isHdr?: boolean
}> = []
const hlsResolutions: PrismaJson.VideoTranscode[] = []
for (const vt of originalMedia.hls.resolutions) {
const ladder = vt.resolution || `${vt.height}p`
const targetWidth = vt.width || originalWidth
const targetHeight = vt.height || originalHeight
const isVtHdr = Boolean(vt.hdr)
const svgString = generateWatermarkSvg(config, targetWidth, targetHeight, blockImagesMap)
const overlayPngBuffer = await transcodeService.renderSvgToPng(svgString)
const overlayPngPath = path.join(tmpDir, `watermarkOverlay-hls-${ladder}.png`)
fs.writeFileSync(overlayPngPath, overlayPngBuffer)
const ladderDir = path.join(hlsTmpDir, ladder)
await transcodeService.transcodeHlsRendition({
inputFile: rawFilePath,
outputDir: ladderDir,
width: targetWidth,
height: targetHeight,
overlayFile: overlayPngPath,
frameRate,
hardwareAcceleration,
sourceVideoBitrate,
threads,
hdr: isVtHdr,
disableAudio: !hasAudio,
sourceIsHdr: isHdr ?? originalMedia?.metadata?.isHdr,
sourceHdrType: hdrType ?? originalMedia?.metadata?.hdrType,
sourceColorTransfer: colorTransfer ?? originalMedia?.metadata?.colorTransfer,
sourceColorPrimaries: colorPrimaries ?? originalMedia?.metadata?.colorPrimaries,
sourceColorSpace: colorSpace ?? originalMedia?.metadata?.colorSpace,
streamIndex: originalMedia?.metadata?.videoStreamIndex,
audioStreamIndex: originalMedia?.metadata?.audioStreamIndex,
})
const targetBps = calculateEffectiveBitrateBps(
targetHeight,
targetWidth,
sourceVideoBitrate,
frameRate,
)
renditionSpecs.push({
resolution: ladder,
width: targetWidth,
height: targetHeight,
bitrateBps: targetBps,
isHdr: isVtHdr,
})
const ladderFiles = fs.readdirSync(ladderDir)
for (const file of ladderFiles) {
const filePath = path.join(ladderDir, file)
const stat = fs.statSync(filePath)
totalSize += stat.size
const contentType = file.endsWith('.m3u8')
? 'application/vnd.apple.mpegurl'
: file.endsWith('.m4s')
? 'video/iso.segment'
: 'video/mp4'
const s3Key = path.posix.join(hlsWatermarkDirKey, ladder, file)
await s3Service.putObject(
bucket,
s3Key,
Bun.file(filePath).stream(),
stat.size,
contentType,
)
}
hlsResolutions.push({
key: path.posix.join(hlsWatermarkDirKey, ladder, 'index.m3u8'),
width: targetWidth,
height: targetHeight,
resolution: ladder,
hdr: isVtHdr,
})
}
const masterM3u8Content = buildHlsMasterPlaylist(renditionSpecs)
const masterM3u8Buffer = Buffer.from(masterM3u8Content, 'utf-8')
await s3Service.putObject(
bucket,
masterKey,
masterM3u8Buffer,
masterM3u8Buffer.length,
'application/vnd.apple.mpegurl',
)
mediaInfo.isHls = true
mediaInfo.hls = {
key: masterKey,
resolutions: hlsResolutions,
}
}
mediaInfo.filesize = totalSize
}
+17
View File
@@ -22,6 +22,13 @@ export class VideoTranscoder {
return this
}
setVideoResolutions(resolutions?: PrismaJson.VideoResolutionLadder[]): this {
if (resolutions) {
this.spec.videoResolutions = resolutions
}
return this
}
setHardwareAcceleration(hardwareAcceleration?: PrismaJson.HardwareAcceleration): this {
if (hardwareAcceleration) {
this.spec.hardwareAcceleration = hardwareAcceleration
@@ -46,6 +53,16 @@ export class VideoTranscoder {
return this
}
setHls(enabled?: boolean, resolutions?: PrismaJson.HlsResolutionLadder[]): this {
if (enabled !== undefined) {
this.spec.hlsEnabled = enabled
}
if (resolutions) {
this.spec.hlsResolutions = resolutions
}
return this
}
async submit(): Promise<string> {
const task = await this.db.workflowTask.create({
data: {
+20 -3
View File
@@ -38,14 +38,31 @@ export async function completeTask(
})
}
export async function failTask(workerQueue: string, taskId: string, err: unknown) {
const { updateTaskStatusActivity } = getActivities()
export async function failTask(
workerQueue: string,
taskId: string,
err: unknown,
assetId?: string,
) {
const { updateTaskStatusActivity, updateAssetStatusActivity } = getActivities()
if (workerQueue) {
const errorMsg = err instanceof Error ? err.message : String(err)
await executeActivity(workerQueue, updateTaskStatusActivity, {
taskId,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
output: { error: errorMsg },
})
if (assetId) {
try {
await executeActivity(workerQueue, updateAssetStatusActivity, {
assetId,
status: 'failed',
error: errorMsg,
})
} catch (assetErr) {
console.error('Failed to update asset status to failed:', assetErr)
}
}
}
}
@@ -155,4 +155,73 @@ describe('transcodeImageWorkflow', () => {
projectId: 'proj-1',
})
})
it('should handle RAW image without usable preview by skipping transcodes and marking processed', async () => {
const task: WorkflowTask = {
id: 'task-raw-nopreview',
assetId: 'asset-raw',
type: WorkflowTaskType.transcode_image,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
thumbnail: true,
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-raw',
storageKey: { key: 'photo.cr2' },
mediaType: 'image/x-canon-cr2',
})
// RAW file without usable preview returns originalWidth: 0, originalHeight: 0
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'image',
metadata: {
originalWidth: 0,
originalHeight: 0,
duration: 0,
frameRate: 0,
totalFrames: 0,
startTimecode: undefined,
bitRate: 0,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
await transcodeImageWorkflow(task)
// transcodeImageActivity should NOT be called since hasUsablePreview is false
expect(mockActivities.transcodeImageActivity).not.toHaveBeenCalled()
// Asset media should still be updated
expect(mockActivities.updateAssetMediaActivity).toHaveBeenCalledWith({
assetId: 'asset-raw',
mediaInfo: expect.objectContaining({
imageTranscodes: [],
}),
})
// Asset should still be marked as processed
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-raw',
status: AssetStatus.processed,
})
})
})
@@ -12,6 +12,7 @@ import {
export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -54,9 +55,10 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
}
const metadata = mediaInfo.metadata
const hasUsablePreview = metadata && metadata.originalWidth > 0 && metadata.originalHeight > 0
const isImage = mediaInfo.proxyType === 'image'
if (isImage && metadata) {
if (isImage && hasUsablePreview && metadata) {
const imageSpec: PrismaJson.ImageTranscode = {
width: metadata.originalWidth,
height: metadata.originalHeight,
@@ -71,7 +73,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
mediaInfo.imageTranscodes.push(imageTranscode)
}
if (spec.thumbnail) {
if (spec.thumbnail && hasUsablePreview) {
const thumbTranscode = await executeActivity(workerQueue, transcodeImageActivity, {
assetKey: key,
filePath,
@@ -89,6 +91,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -105,7 +108,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodeImageWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
@@ -12,6 +12,7 @@ import {
export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -101,6 +102,7 @@ export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -117,7 +119,7 @@ export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodePdfWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
@@ -0,0 +1,173 @@
import { describe, it, expect } from 'vitest'
import {
getTargetHlsResolutions,
resolutionToDimensions,
getTargetVideoResolutions,
isMimePsd,
} from './transcode-utils'
describe('transcode-utils', () => {
describe('getTargetHlsResolutions', () => {
it('uses default ladders when configuredLadders is undefined or empty', () => {
// 1080p source video (1920x1080)
const res1 = getTargetHlsResolutions(undefined, 1920, 1080)
expect(res1).toEqual(['1080p', '720p', '480p'])
const res2 = getTargetHlsResolutions([], 1920, 1080)
expect(res2).toEqual(['1080p', '720p', '480p'])
})
it('caps ladders up to source resolution (no upscaling)', () => {
// 1080p source with 4K configured ladders -> should cap to 1080p
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 1920, 1080)
expect(res).toEqual(['1080p', '720p', '480p'])
})
it('includes higher ladders when source resolution supports them', () => {
// 4K source (3840x2160)
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 3840, 2160)
expect(res).toEqual(['2160p', '1440p', '1080p', '720p', '480p'])
// 1440p source (2560x1440)
const res1440 = getTargetHlsResolutions(
['480p', '720p', '1080p', '1440p', '2160p'],
2560,
1440,
)
expect(res1440).toEqual(['1440p', '1080p', '720p', '480p'])
})
it('handles vertical videos correctly by using the longest side', () => {
// 1080x1920 (vertical 1080p)
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 1080, 1920)
expect(res).toEqual(['1080p', '720p', '480p'])
})
it('falls back to lowest selected ladder when source is smaller than smallest ladder', () => {
// 640x360 source is smaller than 480p (854 long side) -> falls back to lowest selected (480p)
const res = getTargetHlsResolutions(undefined, 640, 360)
expect(res).toEqual(['480p'])
// When user selected 720p and 1440p, and source is 480p (854 long side) -> falls back to lowest selected (720p)
const resCustom = getTargetHlsResolutions(['720p', '1440p'], 854, 480)
expect(resCustom).toEqual(['720p'])
})
it('respects selective subset of configured ladders', () => {
// User only selected 720p and 1440p
const res = getTargetHlsResolutions(['720p', '1440p'], 2560, 1440)
expect(res).toEqual(['1440p', '720p'])
})
})
describe('resolutionToDimensions', () => {
it('calculates dimensions correctly for 1440p and 480p', () => {
const [w1440, h1440] = resolutionToDimensions('1440p', 3840, 2160)
expect(w1440).toBe(2560)
expect(h1440).toBe(1440)
const [w480, h480] = resolutionToDimensions('480p', 1920, 1080)
expect(w480).toBe(854)
expect(h480).toBe(480)
})
it('ensures even dimensions for odd calculations', () => {
const [w, h] = resolutionToDimensions('720p', 1921, 1081)
expect(w % 2).toBe(0)
expect(h % 2).toBe(0)
})
it('returns [0, 0] for unknown resolution', () => {
expect(resolutionToDimensions('invalid', 1920, 1080)).toEqual([0, 0])
})
})
describe('getTargetVideoResolutions', () => {
describe('best_match strategy', () => {
it('returns best match resolution for 4K video', () => {
const resolutions = getTargetVideoResolutions('best_match', 3840, 2160)
expect(resolutions).toEqual(['180p', '2160p'])
})
it('returns best match resolution for 1080p video', () => {
const resolutions = getTargetVideoResolutions('best_match', 1920, 1080)
expect(resolutions).toEqual(['180p', '1080p'])
})
it('returns best match resolution for 720p video', () => {
const resolutions = getTargetVideoResolutions('best_match', 1280, 720)
expect(resolutions).toEqual(['180p', '720p'])
})
it('falls back to 480p for smaller videos (460p and 230p)', () => {
const res460 = getTargetVideoResolutions('best_match', 818, 460)
expect(res460).toEqual(['180p', '480p'])
const res230 = getTargetVideoResolutions('best_match', 408, 230)
expect(res230).toEqual(['180p', '480p'])
})
})
describe('multi strategy', () => {
it('returns all supported resolutions up to source for 4K video', () => {
const resolutions = getTargetVideoResolutions('multi', 3840, 2160)
expect(resolutions).toEqual(['180p', '2160p', '1440p', '1080p', '720p', '480p'])
})
it('returns resolutions up to 720p for 1280x720 video', () => {
const resolutions = getTargetVideoResolutions('multi', 1280, 720)
expect(resolutions).toEqual(['180p', '720p', '480p'])
})
it('returns resolutions up to 1080p for 2276x1280 video', () => {
const resolutions = getTargetVideoResolutions('multi', 2276, 1280)
expect(resolutions).toEqual(['180p', '1080p', '720p', '480p'])
})
it('falls back to lowest selected resolution (480p) for 460p and 230p videos', () => {
const res460 = getTargetVideoResolutions('multi', 818, 460)
expect(res460).toEqual(['180p', '480p'])
const res230 = getTargetVideoResolutions('multi', 408, 230)
expect(res230).toEqual(['180p', '480p'])
})
it('respects user-selected ladders and falls back to lowest selected ladder', () => {
const customLadders = ['720p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[]
// 4K video: matches 2160p, 1440p, 720p (1080p was not selected)
const res4k = getTargetVideoResolutions('multi', 3840, 2160, customLadders)
expect(res4k).toEqual(['180p', '2160p', '1440p', '720p'])
// 230p video: nothing matches <= 408 -> falls back to lowest selected (720p)
const res230 = getTargetVideoResolutions('multi', 408, 230, customLadders)
expect(res230).toEqual(['180p', '720p'])
})
it('normalizes legacy "all" and "full" strategy values to multi', () => {
const resAll = getTargetVideoResolutions(
'all' as unknown as PrismaJson.VideoTranscodeStrategy,
1280,
720,
)
expect(resAll).toEqual(['180p', '720p', '480p'])
const resFull = getTargetVideoResolutions(
'full' as unknown as PrismaJson.VideoTranscodeStrategy,
1280,
720,
)
expect(resFull).toEqual(['180p', '720p', '480p'])
})
})
})
describe('isMimePsd', () => {
it('identifies PSD mime types', () => {
expect(isMimePsd('image/vnd.adobe.photoshop')).toBe(true)
expect(isMimePsd('image/x-photoshop')).toBe(true)
expect(isMimePsd('image/psd')).toBe(true)
expect(isMimePsd('image/png')).toBe(false)
})
})
})
@@ -13,13 +13,67 @@ export function isMimePsd(mimeType: string): boolean {
const RESOLUTION_LONG_SIDES: Record<string, number> = {
'2160p': 3840,
'1440p': 2560,
'1080p': 1920,
'720p': 1280,
'540p': 960,
'480p': 854,
'360p': 640,
'180p': 320,
}
export const HLS_LADDER_ORDER: PrismaJson.HlsResolutionLadder[] = [
'2160p',
'1440p',
'1080p',
'720p',
'480p',
]
export const VIDEO_LADDER_ORDER: PrismaJson.VideoResolutionLadder[] = [
'2160p',
'1440p',
'1080p',
'720p',
'480p',
]
export function matchResolutionsByLongSide<T extends string>(
selectedLadders: T[],
ladderOrder: T[],
rawLongSide: number,
): T[] {
const orderedSelected = ladderOrder.filter((ladder) => selectedLadders.includes(ladder))
if (orderedSelected.length === 0) return []
const matched = orderedSelected.filter((ladder) => {
const targetLongSide = RESOLUTION_LONG_SIDES[ladder]
return targetLongSide !== undefined && targetLongSide <= rawLongSide
})
if (matched.length === 0) {
// Falls back to the lowest resolution selected if nothing matched
return [orderedSelected[orderedSelected.length - 1]]
}
return matched
}
export function getTargetHlsResolutions(
configuredLadders: PrismaJson.HlsResolutionLadder[] | undefined,
originalWidth: number,
originalHeight: number,
): PrismaJson.HlsResolutionLadder[] {
const selectedLadders =
configuredLadders && configuredLadders.length > 0
? configuredLadders
: (['480p', '720p', '1080p'] as PrismaJson.HlsResolutionLadder[])
const rawLongSide = Math.max(originalWidth, originalHeight)
return matchResolutionsByLongSide(selectedLadders, HLS_LADDER_ORDER, rawLongSide)
}
export function resolutionToDimensions(
resolution: string,
originalWidth: number,
@@ -45,31 +99,11 @@ export function resolutionToDimensions(
return [width, height]
}
const TARGET_RESOLUTIONS = [
{ name: '2160p', longSide: 3840 },
{ name: '1080p', longSide: 1920 },
{ name: '720p', longSide: 1280 },
{ name: '540p', longSide: 960 },
{ name: '360p', longSide: 640 },
]
function getBestMatchResolution(
originalWidth: number,
originalHeight: number,
): { name: string; longSide: number } {
const rawLongSide = Math.max(originalWidth, originalHeight)
const lower = TARGET_RESOLUTIONS.filter((r) => r.longSide <= rawLongSide)
if (lower.length > 0) {
lower.sort((a, b) => b.longSide - a.longSide)
return lower[0]
}
return TARGET_RESOLUTIONS[TARGET_RESOLUTIONS.length - 1]
}
export function getTargetVideoResolutions(
strategy: PrismaJson.VideoTranscodeStrategy,
originalWidth: number,
originalHeight: number,
configuredResolutions?: PrismaJson.VideoResolutionLadder[],
): string[] {
const resolutions = ['180p']
@@ -77,21 +111,27 @@ export function getTargetVideoResolutions(
const stratStr = strategy as string
if (stratStr === 'single' || stratStr === 'disable') {
normalizedStrategy = 'best_match'
} else if (stratStr === 'full') {
normalizedStrategy = 'all'
} else if (stratStr === 'full' || stratStr === 'all' || stratStr === 'multi') {
normalizedStrategy = 'multi'
}
const bestMatch = getBestMatchResolution(originalWidth, originalHeight)
const selectedLadders =
configuredResolutions && configuredResolutions.length > 0
? configuredResolutions
: (['480p', '720p', '1080p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[])
const rawLongSide = Math.max(originalWidth, originalHeight)
if (normalizedStrategy === 'best_match') {
resolutions.push(bestMatch.name)
} else if (normalizedStrategy === 'all') {
const bestMatchIndex = TARGET_RESOLUTIONS.findIndex((r) => r.name === bestMatch.name)
if (bestMatchIndex !== -1) {
for (let i = bestMatchIndex; i < TARGET_RESOLUTIONS.length; i++) {
resolutions.push(TARGET_RESOLUTIONS[i].name)
}
}
const matched = matchResolutionsByLongSide(
['480p', '720p', '1080p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[],
VIDEO_LADDER_ORDER,
rawLongSide,
)
resolutions.push(matched[0])
} else if (normalizedStrategy === 'multi') {
const matched = matchResolutionsByLongSide(selectedLadders, VIDEO_LADDER_ORDER, rawLongSide)
resolutions.push(...matched)
}
return resolutions
@@ -25,6 +25,7 @@ describe('transcodeVideoWorkflow', () => {
getAssetActivity: Object.assign(vi.fn(), { _activityName: 'getAssetActivity' }),
getMediaInfoActivity: Object.assign(vi.fn(), { _activityName: 'getMediaInfoActivity' }),
transcodeVideoActivity: Object.assign(vi.fn(), { _activityName: 'transcodeVideoActivity' }),
transcodeHlsActivity: Object.assign(vi.fn(), { _activityName: 'transcodeHlsActivity' }),
transcodeAudioActivity: Object.assign(vi.fn(), { _activityName: 'transcodeAudioActivity' }),
transcodeImageActivity: Object.assign(vi.fn(), { _activityName: 'transcodeImageActivity' }),
generateSpriteActivity: Object.assign(vi.fn(), { _activityName: 'generateSpriteActivity' }),
@@ -702,4 +703,300 @@ describe('transcodeVideoWorkflow', () => {
status: WorkflowTaskStatus.completed,
})
})
it('should trigger HLS transcoding when hlsEnabled is true', async () => {
const task: WorkflowTask = {
id: 'task-hls',
assetId: 'asset-hls',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p', '720p', '1080p'],
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid-hls',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-hls',
storageKey: { key: 'files/asset-hls/video.mp4' },
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
videoBitRate: 850000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockResolvedValue({
key: 'files/asset-hls/video-1080p.mp4',
width: 1920,
height: 1080,
resolution: '1080p',
})
mockActivities.transcodeHlsActivity.mockResolvedValue({
key: 'files/asset-hls/hls/master.m3u8',
resolutions: ['1080p', '720p', '480p'],
})
await transcodeVideoWorkflow(task)
expect(mockActivities.transcodeHlsActivity).toHaveBeenCalledWith(
expect.objectContaining({
taskId: 'task-hls',
assetKey: 'files/asset-hls/video.mp4',
filePath: '/tmp/video.mp4',
ladders: ['1080p', '720p', '480p'],
originalWidth: 1920,
originalHeight: 1080,
}),
)
expect(mockActivities.updateAssetMediaActivity).toHaveBeenCalledWith({
assetId: 'asset-hls',
mediaInfo: expect.objectContaining({
isHls: true,
hls: {
key: 'files/asset-hls/hls/master.m3u8',
resolutions: ['1080p', '720p', '480p'],
},
}),
})
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-hls',
status: WorkflowTaskStatus.completed,
})
})
it('should transcode selected multi resolutions when videoStrategy is multi', () => {
const task: WorkflowTask = {
id: 'task-multi',
assetId: 'asset-multi',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p', '2160p'],
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-multi',
storageKey: { key: 'files/asset-multi/video.mp4' },
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockImplementation(async (params) => ({
key: `files/asset-multi/${params.videoSpec.resolution}.mp4`,
width: params.videoSpec.width,
height: params.videoSpec.height,
resolution: params.videoSpec.resolution,
}))
return transcodeVideoWorkflow(task).then(() => {
// 180p preview + 1080p + 720p (2160p is skipped since > 1920)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledTimes(3)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '180p' }),
}),
)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '1080p' }),
}),
)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '720p' }),
}),
)
})
})
it('should mark task and asset as failed when an activity fails', async () => {
const task: WorkflowTask = {
id: 'task-fail',
assetId: 'asset-fail',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: { videoStrategy: 'best_match' },
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-fail',
name: 'fail.mp4',
storageKey: { key: 'files/asset-fail/fail.mp4' },
status: AssetStatus.uploading,
})
mockActivities.getMediaInfoActivity.mockRejectedValue(new Error('FFprobe failed to read media'))
await expect(transcodeVideoWorkflow(task)).rejects.toThrow('FFprobe failed to read media')
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-fail',
status: 'failed',
output: { error: 'FFprobe failed to read media' },
})
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-fail',
status: 'failed',
error: 'FFprobe failed to read media',
})
})
it('should mark task as failed but keep asset as processed when post-transcode activity fails', async () => {
const task: WorkflowTask = {
id: 'task-post-fail',
assetId: 'asset-post-fail',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: { videoStrategy: 'best_match' },
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-post-fail',
name: 'video.mp4',
storageKey: { key: 'files/asset-post-fail/video.mp4' },
status: AssetStatus.uploading,
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockResolvedValue({
key: 'v.mp4',
width: 1920,
height: 1080,
})
mockActivities.createEmbeddingTaskIfEnabledActivity.mockRejectedValue(
new Error('Embedding creation failed'),
)
await expect(transcodeVideoWorkflow(task)).rejects.toThrow('Embedding creation failed')
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-post-fail',
status: 'failed',
output: { error: 'Embedding creation failed' },
})
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-post-fail',
status: 'processed',
})
expect(mockActivities.updateAssetStatusActivity).not.toHaveBeenCalledWith(
expect.objectContaining({
status: 'failed',
}),
)
})
})
@@ -8,11 +8,16 @@ import {
failTask,
cleanupTmpDir,
} from './common'
import { getTargetVideoResolutions, resolutionToDimensions } from './transcode-utils'
import {
getTargetVideoResolutions,
getTargetHlsResolutions,
resolutionToDimensions,
} from './transcode-utils'
export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -21,6 +26,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
updateAssetStatusActivity,
getMediaInfoActivity,
transcodeVideoActivity,
transcodeHlsActivity,
transcodeAudioActivity,
transcodeImageActivity,
extractPosterActivity,
@@ -59,7 +65,9 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
codec: '',
}
if (spec.poster) {
const isVideo = mediaInfo.proxyType === 'video'
if (spec.poster && isVideo) {
const lastSlashIndex = key.lastIndexOf('/')
const assetDir = lastSlashIndex === -1 ? '' : key.substring(0, lastSlashIndex)
const posterSpec: PrismaJson.PosterInfo = {
@@ -72,6 +80,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
isHdr: mediaInfo.metadata?.isHdr,
hdrType: mediaInfo.metadata?.hdrType,
colorTransfer: mediaInfo.metadata?.colorTransfer,
streamIndex: mediaInfo.metadata?.videoStreamIndex,
})
mediaInfo.poster = posterResult.poster
await executeActivity(workerQueue, updateAssetMediaActivity, {
@@ -82,7 +91,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
// Generate the sprite (and its poster fallback) before the expensive proxy transcodes so the
// file list can show a poster/sprite preview while the asset is still processing.
if (spec.sprite || (spec.poster && !mediaInfo.poster)) {
if ((spec.sprite || (spec.poster && !mediaInfo.poster)) && isVideo) {
const lastSlashIndex = key.lastIndexOf('/')
const assetDir = lastSlashIndex === -1 ? '' : key.substring(0, lastSlashIndex)
@@ -119,11 +128,12 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
const metadata = mediaInfo.metadata
if (mediaInfo.proxyType === 'video' && metadata) {
if (isVideo && metadata) {
const videoResolutions = getTargetVideoResolutions(
spec.videoStrategy || 'best_match',
metadata.originalWidth,
metadata.originalHeight,
spec.videoResolutions,
)
for (const res of videoResolutions) {
const [width, height] = resolutionToDimensions(
@@ -157,6 +167,8 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoPreview = videoTranscode
@@ -185,10 +197,43 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoTranscodes.push(videoTranscode)
}
if (spec.hlsEnabled) {
const hlsLadders = getTargetHlsResolutions(
spec.hlsResolutions,
metadata.originalWidth,
metadata.originalHeight,
)
if (hlsLadders.length > 0) {
const hlsResult = await executeActivity(workerQueue, transcodeHlsActivity, {
taskId: task.id,
assetKey: key,
filePath,
ladders: hlsLadders,
originalWidth: metadata.originalWidth,
originalHeight: metadata.originalHeight,
originalFps: metadata.frameRate,
sourceVideoBitrate: metadata.videoBitRate || metadata.bitRate,
hardwareAcceleration: spec.hardwareAcceleration,
threads: spec.threads,
sourceIsHdr: metadata.isHdr,
sourceHdrType: metadata.hdrType,
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.isHls = true
mediaInfo.hls = hlsResult
}
}
}
const isAudio = mediaInfo.proxyType === 'audio'
@@ -198,11 +243,12 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
assetKey: key,
filePath,
threads: spec.threads,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoTranscodes.push(audioTranscode)
}
if (spec.thumbnail) {
if (spec.thumbnail && isVideo) {
const thumbTranscode = await executeActivity(workerQueue, transcodeImageActivity, {
assetKey: key,
filePath,
@@ -220,6 +266,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -236,7 +283,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodeVideoWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
+7 -2
View File
@@ -51,9 +51,14 @@ export function AuthLayout({ children }: { children: ReactNode }) {
{/* Footer Label */}
<div className="relative pt-4 border-t border-zinc-200/30 dark:border-zinc-800/30">
<p className="text-xs text-zinc-500 dark:text-zinc-400 font-medium">
<a
href="https://discord.gg/sUzhujK6DD"
target="_blank"
rel="noreferrer"
className="text-xs text-zinc-500 hover:text-zinc-800 dark:text-zinc-400 dark:hover:text-zinc-200 font-medium transition-colors hover:underline inline-flex items-center gap-1"
>
{m.auth_community_cta()}
</p>
</a>
</div>
</div>
@@ -1,6 +1,6 @@
// @vitest-environment happy-dom
import React from 'react'
import { cleanup, render, screen } from '@testing-library/react'
import { cleanup, fireEvent, render, screen } from '@testing-library/react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { BreadcrumbNav } from './breadcrumb-nav'
@@ -160,4 +160,107 @@ describe('BreadcrumbNav component', () => {
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
expect(triggerButton).toBeDefined()
})
it('displays 480p instead of 360p for 480p video transcode in download menu', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-1"
currentAsset={{ id: 'file-1', name: 'sample.mov', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'sample-480p.mp4',
width: 854,
height: 480,
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('displays explicit resolution when provided in download menu', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-1"
currentAsset={{ id: 'file-1', name: 'sample.mov', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'sample-480p.mp4',
width: 854,
height: 480,
resolution: '480p',
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('correctly labels legacy portrait video transcode in download menu using long side fallback', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-portrait"
currentAsset={{ id: 'file-portrait', name: 'portrait.mp4', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mp4',
videoTranscodes: [
{
key: 'portrait-legacy.mp4',
width: 1080,
height: 1920,
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'portrait.mp4' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('1080p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('1920p'))).toBe(false)
})
})
+3 -8
View File
@@ -48,6 +48,7 @@ import { useState } from 'react'
import { useDualSidebarStore } from '@/ui/stores/dual-sidebar'
import { ManageVersionsDialog } from './manage-versions-dialog'
import { AssetLinkedTasksDialog } from './kanban/asset-linked-tasks-dialog'
import { getVideoResolutionLabel } from '@/ui/lib/media'
interface BreadcrumbNavProps {
teamId: string
@@ -84,6 +85,7 @@ interface BreadcrumbNavProps {
key: string
width: number
height: number
resolution?: string
hdr?: boolean
}>
}
@@ -287,14 +289,7 @@ export function BreadcrumbNav({
<DropdownMenuSubContent className="w-48">
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
{downloadInfo?.videoTranscodes?.map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const resolution = getVideoResolutionLabel(t)
const label = t.hdr ? `${resolution} (HDR)` : resolution
return (
<DropdownMenuItem
@@ -66,6 +66,9 @@ export function CompareControlBar({
showFrames: false,
currentResolution: v.currentResolution,
isCurrentHdr: v.isCurrentHdr,
activeAutoResolution: v.activeAutoResolution,
isHlsManualSupported: v.isHlsManualSupported,
isLoading: v.isLoading,
}
// The active-side asset carries the metadata used for timecode display and
+13 -3
View File
@@ -1,10 +1,17 @@
import type { VideoTranscode } from '@shumai/dtos'
export type PaneKind = 'image' | 'video' | 'unsupported'
export type CompareSide = 'left' | 'right'
export type DisplayTranscode = VideoTranscode & { resolution: string }
export interface DisplayTranscode {
id?: string
url?: string
key?: string
width: number
height: number
size?: number
hdr?: boolean
resolution: string
}
/**
* Reactive state a pane reports upward so the shared control bar can render.
@@ -26,6 +33,9 @@ export interface PaneReportedState {
isCurrentHdr?: boolean
resolutions: DisplayTranscode[]
buffered: number
activeAutoResolution?: string
isHlsManualSupported?: boolean
isLoading?: boolean
}
}
@@ -1258,7 +1258,8 @@ export function FileBrowser({
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={() => {
onClick={(e) => {
e.preventDefault()
emptyTrash()
}}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
@@ -382,4 +382,63 @@ describe('FileCard', () => {
// The duration badge appears once processing is done.
expect(screen.getByText('00:10')).toBeTruthy()
})
it('renders failed state with alert icon and shows creator at bottom when status is failed without a preview', () => {
const failedItem: AssetInfo = {
...fileItem,
status: 'failed',
creator: { id: 'u1', name: 'Alice' },
} as AssetInfo
renderComponent({ item: failedItem })
expect(screen.queryByTestId('file-card-preparing-circle')).toBeNull()
const failedState = screen.getByTestId('file-card-failed-state')
expect(failedState).toBeTruthy()
expect(screen.getByText(/Processing failed|处理失败/i)).toBeTruthy()
expect(screen.getByText(/Alice/i)).toBeTruthy()
})
it('renders thumbnail with failed badge in top-right corner, shows error in tooltip on hover, and shows creator at bottom', () => {
vi.useFakeTimers()
const failedItemWithPreview: AssetInfo = {
...fileItem,
status: 'failed',
creator: { id: 'u1', name: 'Alice' },
media: {
error: 'FFmpeg transcode failed: exit code 1',
},
preview: {
proxyType: 'video',
thumbnailUrl: 'https://example.com/poster.webp',
duration: 10,
},
} as AssetInfo
renderComponent({ item: failedItemWithPreview })
const media = screen.getByTestId('file-card-preview-media')
expect(media.className).not.toContain('animate-pulse')
const badge = screen.getByTestId('file-card-failed-badge')
expect(badge).toBeTruthy()
expect(badge.textContent).toMatch(/Transcoding Failed|转码失败/i)
expect(badge.querySelector('svg')).toBeNull()
// Failed badge is in the top-right container
const rightTopContainer = badge.closest('.absolute.right-2.top-2')
expect(rightTopContainer).toBeTruthy()
// Creator is still shown at the bottom of the card
expect(screen.getByText(/Alice/i)).toBeTruthy()
// Hovering the badge reveals the error message in tooltip
fireEvent.pointerMove(badge, { pointerType: 'mouse' })
act(() => {
vi.advanceTimersByTime(200)
})
const tooltip = screen.getByRole('tooltip')
expect(tooltip).toBeTruthy()
expect(tooltip.textContent).toContain('FFmpeg transcode failed: exit code 1')
vi.useRealTimers()
})
})
@@ -27,6 +27,7 @@ import { cn } from '@/ui/lib/utils'
import { useUploadStore } from '@/ui/stores/upload'
import { useDraggable, useDroppable } from '@dnd-kit/react'
import {
AlertCircle,
Clock,
Download,
Edit,
@@ -131,15 +132,21 @@ export function FileCard({
enabled: shouldPoll,
refetchInterval: (query: unknown) => {
const data = (query as { state: { data: { status?: string } } }).state.data
return data?.status === 'processed' ? false : 1000
return data?.status === 'processed' || data?.status === 'failed' || data?.status === 'error'
? false
: 1000
},
})
const displayItem = useMemo(() => {
if (item.status === 'processed' || item.status === 'error') {
if (item.status === 'processed' || item.status === 'error' || item.status === 'failed') {
return item
}
if (polledItem?.status === 'processed' || polledItem?.status === 'error') {
if (
polledItem?.status === 'processed' ||
polledItem?.status === 'error' ||
polledItem?.status === 'failed'
) {
return polledItem
}
return polledItem || item
@@ -263,10 +270,11 @@ export function FileCard({
// A preview (poster thumbnail and/or sprite) can be available before transcoding finishes,
// since the poster/sprite are generated and persisted ahead of the proxy transcodes.
const hasPreview = Boolean(displayItem.preview?.thumbnailUrl || displayItem.preview?.spriteUrl)
const isFailed = displayItem.status === 'failed'
const isProcessing = displayItem.status === 'processing' || displayItem.status === 'uploaded'
// While uploading/transcoding, the creator row is replaced by a short status label so the real
// date/author only appears once the asset is ready.
// date/author only appears once the asset is ready or failed.
const statusText =
displayItem.status === 'uploading'
? m.uploading()
@@ -274,6 +282,8 @@ export function FileCard({
? m.preparing()
: null
const failedErrorMessage = displayItem.media?.error || m.processing_failed()
const previewBadges =
daysLeft !== null ||
(typeof displayItem.commentsCount === 'number' && displayItem.commentsCount > 0) ? (
@@ -322,7 +332,7 @@ export function FileCard({
showDropFeedback && 'border-primary outline-1 outline-primary',
)}
>
<div className="absolute left-2 top-2 z-10">
<div className="absolute left-2 top-2 z-10 flex items-center gap-1.5">
<Checkbox
checked={isChecked}
onCheckedChange={() => {}}
@@ -351,6 +361,27 @@ export function FileCard({
displayItem.versionStack.versions.length}
</Badge>
)}
{isFailed && (
<TooltipProvider delayDuration={100}>
<Tooltip>
<TooltipTrigger asChild>
<span
data-testid="file-card-failed-badge"
className="select-none rounded bg-destructive/90 px-1.5 py-0.5 text-xs font-medium text-destructive-foreground shadow-sm cursor-help"
>
{m.transcoding_failed()}
</span>
</TooltipTrigger>
<TooltipContent
side="bottom"
data-testid="file-card-failed-tooltip"
className="max-w-xs break-words text-xs"
>
{failedErrorMessage}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)}
</div>
<div className="relative aspect-square overflow-hidden bg-muted/30">
@@ -364,7 +395,7 @@ export function FileCard({
data-testid="file-card-preview-media"
className={cn('h-full w-full', isProcessing && 'animate-pulse')}
>
<FilePreview item={displayItem} showDuration={!isProcessing} />
<FilePreview item={displayItem} showDuration={!isProcessing && !isFailed} />
</div>
{previewBadges}
</>
@@ -374,7 +405,17 @@ export function FileCard({
{m.failed_to_upload()}
</span>
</div>
) : displayItem.status === 'processing' || displayItem.status === 'uploaded' ? (
) : isFailed ? (
<div
data-testid="file-card-failed-state"
className="flex flex-col h-full w-full items-center justify-center gap-2 p-3 text-center bg-muted/40"
>
<AlertCircle className="h-10 w-10 text-destructive/80 shrink-0" />
<span className="z-10 text-xs font-semibold text-destructive px-2">
{m.processing_failed()}
</span>
</div>
) : isProcessing ? (
<div className="flex h-full w-full items-center justify-center">
{/* Same geometry as the upload progress ring, but empty inside and no percentage. */}
<svg
@@ -431,7 +472,7 @@ export function FileCard({
}}
>
<TooltipTrigger asChild>
<p ref={creatorRef} className="text-sm text-muted-foreground line-clamp-2 h-[2lh]">
<p ref={creatorRef} className="text-sm line-clamp-2 h-[2lh] text-muted-foreground">
{statusText ?? creatorText}
</p>
</TooltipTrigger>
@@ -134,4 +134,17 @@ describe('FileListItem', () => {
expect(onDoubleClick).toHaveBeenCalled()
})
it('renders (Processing failed) label when status is failed', () => {
const failedItem: AssetInfo = {
...mockItem,
status: 'failed',
} as AssetInfo
renderComponent({ item: failedItem })
const failedLabel = screen.getByTestId('file-list-item-failed')
expect(failedLabel).toBeTruthy()
expect(failedLabel.textContent).toMatch(/Processing failed|处理失败/i)
})
})
@@ -1,9 +1,10 @@
import type { AssetInfo, FieldValueInfo } from '@shumai/dtos'
import { client } from '@/ui/api/client'
import { m } from '@/ui/paraglide/messages.js'
import { useQuery } from '@tanstack/react-query'
import { useDraggable } from '@dnd-kit/react'
import { File, Folder, MoreVertical, AudioLines } from 'lucide-react'
import { AlertCircle, File, Folder, MoreVertical, AudioLines } from 'lucide-react'
import React, { useEffect, useMemo, useRef, useState } from 'react'
import { cn } from '@/ui/lib/utils'
import { formatSize } from '@/ui/lib/format'
@@ -90,15 +91,21 @@ export function FileListItem({
enabled: shouldPoll,
refetchInterval: (query: unknown) => {
const data = (query as { state: { data: { status?: string } } }).state.data
return data?.status === 'processed' ? false : 1000
return data?.status === 'processed' || data?.status === 'failed' || data?.status === 'error'
? false
: 1000
},
})
const displayItem = useMemo(() => {
if (item.status === 'processed' || item.status === 'error') {
if (item.status === 'processed' || item.status === 'error' || item.status === 'failed') {
return item
}
if (polledItem?.status === 'processed' || polledItem?.status === 'error') {
if (
polledItem?.status === 'processed' ||
polledItem?.status === 'error' ||
polledItem?.status === 'failed'
) {
return polledItem
}
return polledItem || item
@@ -215,6 +222,8 @@ export function FileListItem({
<div className="w-8 h-8 shrink-0 flex items-center justify-center bg-muted rounded overflow-hidden">
{displayItem.status === 'uploading' ? (
<ProgressCircle progress={uploadPercent} className="w-7 h-7" />
) : displayItem.status === 'failed' && !displayItem.preview?.thumbnailUrl ? (
<AlertCircle className="h-4 w-4 text-destructive" />
) : displayItem.preview?.thumbnailUrl ? (
<img
src={displayItem.preview.thumbnailUrl}
@@ -240,7 +249,15 @@ export function FileListItem({
/>
{displayItem.status === 'error' && (
<span className="text-xs text-destructive font-semibold shrink-0">
(Failed to upload)
{m.failed_to_upload_parenthesized()}
</span>
)}
{displayItem.status === 'failed' && (
<span
data-testid="file-list-item-failed"
className="text-xs text-destructive font-semibold shrink-0"
>
{m.processing_failed_parenthesized()}
</span>
)}
@@ -654,7 +654,10 @@ export function MobileFileBrowser({
<AlertDialogFooter>
<AlertDialogCancel>{m.cancel()}</AlertDialogCancel>
<AlertDialogAction
onClick={() => emptyTrash()}
onClick={(e) => {
e.preventDefault()
emptyTrash()
}}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
disabled={isEmptyingTrash}
>
@@ -145,6 +145,7 @@ export function MobileFileDetail({
key: t.key,
width: t.width,
height: t.height,
resolution: t.resolution,
hdr: t.hdr,
})),
}
@@ -119,4 +119,69 @@ describe('MobileFileHeader', () => {
expect(menuBtn).toBeDefined()
fireEvent.click(menuBtn)
})
it('displays 480p instead of 360p for 480p video transcode in download menu', () => {
render(
<MobileFileHeader
fileName="scene1.mp4"
activeFileId="file-1"
allowDownload={true}
onBack={vi.fn()}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'scene1-480p.mp4',
width: 854,
height: 480,
resolution: '480p',
hdr: false,
},
],
}}
/>,
)
const menuBtn = screen.getByLabelText(/More options/i)
fireEvent.pointerDown(menuBtn, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('correctly labels legacy portrait video transcode in download menu using long side fallback', () => {
render(
<MobileFileHeader
fileName="portrait.mp4"
activeFileId="file-portrait"
allowDownload={true}
onBack={vi.fn()}
downloadInfo={{
originalKey: 'raw.mp4',
videoTranscodes: [
{
key: 'portrait-legacy.mp4',
width: 1080,
height: 1920,
hdr: false,
},
],
}}
/>,
)
const menuBtn = screen.getByLabelText(/More options/i)
fireEvent.pointerDown(menuBtn, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('1080p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('1920p'))).toBe(false)
})
})
@@ -26,6 +26,7 @@ import {
Trash2,
} from 'lucide-react'
import React from 'react'
import { getVideoResolutionLabel } from '@/ui/lib/media'
export interface MobileFileHeaderProps {
fileName: string
@@ -46,6 +47,7 @@ export interface MobileFileHeaderProps {
key: string
width: number
height: number
resolution?: string
hdr?: boolean
}>
}
@@ -238,14 +240,7 @@ export function MobileFileHeader({
<DropdownMenuSubContent className="w-48 z-50">
<DropdownMenuLabel className="text-xs">{m.download()}</DropdownMenuLabel>
{downloadInfo?.videoTranscodes?.map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const resolution = getVideoResolutionLabel(t)
const label = t.hdr ? `${resolution} (HDR)` : resolution
return (
<DropdownMenuItem
@@ -440,6 +440,7 @@ export function PublicShareManager({
key: t.key,
width: t.width,
height: t.height,
resolution: t.resolution,
hdr: t.hdr,
})),
}
@@ -0,0 +1,46 @@
// @vitest-environment happy-dom
import { cleanup, render, screen } from '@testing-library/react'
import React from 'react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { DefaultViewer } from './default-viewer'
import type { AssetInfo } from '@shumai/dtos'
vi.mock('@/ui/paraglide/messages.js', () => ({
m: new Proxy({}, { get: () => () => '' }),
}))
describe('DefaultViewer', () => {
afterEach(() => {
cleanup()
})
it('renders failed state when file status is failed', () => {
const file = {
id: 'failed-doc',
name: 'failed.docx',
status: 'failed',
media: {
error: 'PDF generation failed',
},
} as unknown as AssetInfo
render(<DefaultViewer file={file} />)
expect(screen.getByTestId('default-viewer-failed-state')).toBeDefined()
expect(screen.getByText('PDF generation failed')).toBeDefined()
})
it('renders standard download view when file is not failed', () => {
const file = {
id: 'file-1',
name: 'archive.zip',
status: 'processed',
sizeByte: 1024,
media: {
original: { key: 'raw/archive.zip' },
},
} as unknown as AssetInfo
render(<DefaultViewer file={file} />)
expect(screen.queryByTestId('default-viewer-failed-state')).toBeNull()
expect(screen.getByText('Download')).toBeDefined()
})
})
@@ -1,5 +1,6 @@
import { client } from '@/ui/api/client'
import { Download } from 'lucide-react'
import { m } from '@/ui/paraglide/messages.js'
import { Download, AlertCircle } from 'lucide-react'
import React, { useImperativeHandle } from 'react'
import { FileViewerProps, MediaController } from '../types'
@@ -37,12 +38,40 @@ export const DefaultViewer = React.forwardRef<MediaController, FileViewerProps>(
}
}
const isFailed = file.status === 'failed'
const errorMessage = file.media?.error
return (
<div className="flex flex-col flex-1 h-full overflow-hidden bg-gray-100 dark:bg-gray-950 relative">
<div className="flex-1 flex flex-col-reverse md:flex-row min-h-0 relative">
{children}
<div className="flex-1 flex items-center justify-center">
<p className="text-muted-foreground">Preview unavailable</p>
<div className="flex-1 flex items-center justify-center p-6 text-center">
{isFailed ? (
<div
data-testid="default-viewer-failed-state"
className="flex flex-col items-center justify-center gap-3 max-w-md"
>
<AlertCircle className="h-12 w-12 text-destructive/80" />
<div className="space-y-1">
<h3 className="text-base font-semibold text-foreground">
{m.media_processing_failed()}
</h3>
<p className="text-sm text-muted-foreground">
{m.media_processing_failed_desc()}
</p>
{errorMessage && (
<div
title={errorMessage}
className="mt-3 rounded-lg border border-destructive/20 bg-destructive/10 p-3 text-xs text-destructive text-left line-clamp-3 break-words"
>
{errorMessage}
</div>
)}
</div>
</div>
) : (
<p className="text-muted-foreground">Preview unavailable</p>
)}
</div>
</div>
<div className="relative px-4 py-3 bg-card border-t border-gray-200 dark:border-gray-700 z-10 flex items-center justify-end gap-2 transition-colors duration-200">
@@ -0,0 +1,502 @@
// @vitest-environment happy-dom
import { act, cleanup, render } from '@testing-library/react'
import React, { createRef } from 'react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { CompareVideoPane } from './compare-video-pane'
import type { ComparePaneHandle, PaneReportedState } from '../../compare/types'
import type { AssetInfo } from '@shumai/dtos'
const { mockHlsInstance, mockHlsConstructor } = vi.hoisted(() => {
const instance = {
attachMedia: vi.fn(),
loadSource: vi.fn(),
on: vi.fn(),
off: vi.fn(),
once: vi.fn(),
destroy: vi.fn(),
levels: [
{ height: 1080, width: 1920, bitrate: 5000000 },
{ height: 720, width: 1280, bitrate: 2500000 },
],
currentLevel: -1,
startLoad: vi.fn(),
recoverMediaError: vi.fn(),
}
const constructor = vi.fn(function () {
return instance
})
return { mockHlsInstance: instance, mockHlsConstructor: constructor }
})
vi.mock('hls.js', () => {
const MockHls = mockHlsConstructor
// @ts-expect-error adding static mock property
MockHls.isSupported = vi.fn(() => true)
// @ts-expect-error adding static mock property
MockHls.Events = {
MANIFEST_PARSED: 'hlsManifestParsed',
LEVEL_SWITCHED: 'hlsLevelSwitched',
ERROR: 'hlsError',
}
// @ts-expect-error adding static mock property
MockHls.ErrorTypes = {
NETWORK_ERROR: 'networkError',
MEDIA_ERROR: 'mediaError',
OTHER_ERROR: 'otherError',
}
return {
default: MockHls,
}
})
vi.mock('@/ui/components/drawing-canvas', () => ({
default: () => <div data-testid="drawing-canvas" />,
}))
vi.mock('@/ui/paraglide/messages.js', () => ({
m: new Proxy({}, { get: () => () => '' }),
}))
describe('CompareVideoPane', () => {
beforeEach(async () => {
vi.clearAllMocks()
mockHlsInstance.currentLevel = -1
const Hls = (await import('hls.js')).default as unknown as {
isSupported: ReturnType<typeof vi.fn>
}
Hls.isSupported.mockReturnValue(true)
})
afterEach(() => {
cleanup()
})
it('safely handles native HLS fallback without clearing video.src when manual resolution is invoked', async () => {
const Hls = (await import('hls.js')).default as unknown as {
isSupported: ReturnType<typeof vi.fn>
}
Hls.isSupported.mockReturnValue(false)
// Mock HTMLMediaElement.prototype.canPlayType to support native HLS
const originalCanPlayType = HTMLMediaElement.prototype.canPlayType
HTMLMediaElement.prototype.canPlayType = vi.fn((type: string) =>
type === 'application/vnd.apple.mpegurl' ? 'maybe' : '',
)
try {
const hlsVideo: AssetInfo = {
id: 'hls-compare',
name: 'hls-compare.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
const ref = createRef<ComparePaneHandle>()
const { container } = render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onStateChange={(state) => reportedStates.push(state)}
onActivate={vi.fn()}
/>,
)
const video = container.querySelector('video') as HTMLVideoElement
expect(video.src).toBe('https://cdn.example.com/master.m3u8')
// Check reported state: isHlsManualSupported should be false
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.isHlsManualSupported).toBe(false)
// Invoke changeResolution('1080p')
act(() => {
ref.current?.changeResolution('1080p')
})
// In native fallback, changeResolution should NO-OP and NOT clear video.src to empty string
expect(video.src).toBe('https://cdn.example.com/master.m3u8')
} finally {
HTMLMediaElement.prototype.canPlayType = originalCanPlayType
}
})
it('switches HLS resolution immediately using currentLevel and does not hijack subsequent seeks', async () => {
vi.useFakeTimers()
try {
const hlsVideo: AssetInfo = {
id: 'hls-compare-switch',
name: 'hls-compare-switch.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
const ref = createRef<ComparePaneHandle>()
const { container } = render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onStateChange={(state) => reportedStates.push(state)}
onActivate={vi.fn()}
/>,
)
const video = container.querySelector('video') as HTMLVideoElement
Object.defineProperty(video, 'paused', { value: true, configurable: true })
video.currentTime = 0
// In Hls.js supported mode, isHlsManualSupported should be true
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.isHlsManualSupported).toBe(true)
// Switch to 720p (index 1)
act(() => {
ref.current?.changeResolution('720p')
})
expect(mockHlsInstance.currentLevel).toBe(1)
// Advance timers past the 1500ms safety timeout without seeked event
act(() => {
vi.advanceTimersByTime(2000)
})
// Now user performs an unrelated seek to frame 150 (currentTime = 5.0)
video.currentTime = 5.0
act(() => {
video.dispatchEvent(new Event('seeked'))
})
// video.currentTime must remain at frame 150, NOT rewound to frame 0
expect(Math.floor(video.currentTime * 30 + 0.001)).toBe(150)
expect(video.currentTime).toBeCloseTo(5.0167, 3)
} finally {
vi.useRealTimers()
}
})
it('reports 2160p instead of 3840p for 4K vertical video when HLS switches level on Auto', () => {
mockHlsInstance.levels = [{ height: 3840, width: 2160, bitrate: 10000000 }]
const hlsVideo: AssetInfo = {
id: 'video-hls-compare-vertical-4k',
name: 'vertical-4k.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [{ width: 2160, height: 3840, resolution: '2160p' }],
},
metadata: {
originalWidth: 2160,
originalHeight: 3840,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const ref = createRef<ComparePaneHandle>()
const reportedStates: PaneReportedState[] = []
render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={(s) => reportedStates.push(s)}
/>,
)
const levelSwitchedHandler = mockHlsInstance.on.mock.calls.find(
([event]) => event === 'hlsLevelSwitched',
)?.[1]
expect(levelSwitchedHandler).toBeDefined()
act(() => {
levelSwitchedHandler('hlsLevelSwitched', { level: 0 })
})
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.activeAutoResolution).toBe('2160p')
})
it('derives resolution from long side for legacy portrait MP4 transcode without explicit resolution', () => {
const legacyPortraitVideo: AssetInfo = {
id: 'legacy-portrait-compare-mp4',
name: 'portrait.mp4',
proxyType: 'video',
media: {
videoTranscodes: [
{
key: 'portrait-1080p.mp4',
url: 'https://cdn.example.com/portrait-1080p.mp4',
width: 1080,
height: 1920,
size: 5000000,
},
],
metadata: {
originalWidth: 1080,
originalHeight: 1920,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
render(
<CompareVideoPane
file={legacyPortraitVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={(s) => reportedStates.push(s)}
/>,
)
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.resolutions?.[0]?.resolution).toBe('1080p')
})
it('sets poster on video element from file.preview.thumbnailUrl', () => {
const videoWithThumb: AssetInfo = {
id: 'compare-thumb',
name: 'thumb.mp4',
proxyType: 'video',
preview: {
thumbnailUrl: 'https://cdn.example.com/compare-poster.jpg',
},
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(
<CompareVideoPane
file={videoWithThumb}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
/>,
)
const video = container.querySelector(
'[data-testid="compare-video-area"] video',
) as HTMLVideoElement
expect(video.getAttribute('poster')).toBe('https://cdn.example.com/compare-poster.jpg')
})
it('shows loading spinner initially, transitions on loadeddata, and blocks play while loading', () => {
const testVideo: AssetInfo = {
id: 'compare-loading-test',
name: 'loading.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const onRequestTogglePlay = vi.fn()
const { container } = render(
<CompareVideoPane
file={testVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
onRequestTogglePlay={onRequestTogglePlay}
/>,
)
const area = container.querySelector('[data-testid="compare-video-area"]') as HTMLDivElement
const video = area.querySelector('video') as HTMLVideoElement
// Initially loading spinner is visible
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).not.toBeNull()
expect(container.querySelector('[data-testid="compare-video-play-overlay"]')).toBeNull()
// Clicking while loading should not trigger play
act(() => {
area.click()
})
expect(onRequestTogglePlay).not.toHaveBeenCalled()
// Dispatch loadeddata
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
// Spinner gone, play overlay visible
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
expect(container.querySelector('[data-testid="compare-video-play-overlay"]')).not.toBeNull()
// Clicking now triggers toggle play
act(() => {
area.click()
})
expect(onRequestTogglePlay).toHaveBeenCalledTimes(1)
})
it('does not cancel waiting stall debounce when progress events fire during stall', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'compare-stall-progress-test',
name: 'compare-stall-progress.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(
<CompareVideoPane
file={testVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
/>,
)
const video = container.querySelector(
'[data-testid="compare-video-area"] video',
) as HTMLVideoElement
// 1. Initial ready state
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 2. Play starts (paused === false)
Object.defineProperty(video, 'paused', { value: false, configurable: true, writable: true })
act(() => {
video.dispatchEvent(new Event('playing'))
})
// 3. Stalls: dispatches waiting
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Spinner not visible immediately (within 200ms debounce)
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 4. Progress event fires at 100ms while paused is false (data still trickling in)
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('progress'))
})
// Should NOT clear debounce or mark loading false
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 5. Advance past 200ms debounce (advance remaining 110ms)
act(() => {
vi.advanceTimersByTime(110)
})
// Spinner should now appear despite the progress event!
expect(
container.querySelector('[data-testid="compare-video-loading-spinner"]'),
).not.toBeNull()
// 6. Playback resumes: dispatch playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
// Spinner disappears
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
})
@@ -4,14 +4,14 @@ import { cn } from '@/ui/lib/utils'
import { useAnnotationStore } from '@/ui/stores/annotation-store'
import type { Annotation } from '@/ui/types'
import type { AssetInfo } from '@shumai/dtos'
import { Play, AudioLines } from 'lucide-react'
import { Play, AudioLines, Loader2 } from 'lucide-react'
import { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react'
import videojs from 'video.js'
import type Player from 'video.js/dist/types/player'
import Hls from 'hls.js'
import { useFramePlayer } from './use-frame-player'
import { resolveTotalFrames } from './utils'
import { calculateFrameCenterTime, resolveTotalFrames } from './utils'
import { clampFrame as clampFrameUtil } from '../../compare/compare-utils'
import type { ComparePaneHandle, DisplayTranscode, PaneReportedState } from '../../compare/types'
import { getVideoResolutionLabel } from '@/ui/lib/media'
interface CompareVideoPaneProps {
file: AssetInfo
@@ -32,17 +32,15 @@ interface CompareVideoPaneProps {
function computeResolutions(file: AssetInfo): DisplayTranscode[] {
// Only transcoded proxy versions are ever displayed; the raw original file
// is never used as a playback source.
return (file.media?.videoTranscodes ?? []).map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
return { ...t, resolution }
})
const isHls = Boolean(file.media?.isHls && file.media.hls?.url)
const baseTranscodes = isHls
? (file.media?.hls?.resolutions ?? [])
: (file.media?.videoTranscodes ?? [])
return baseTranscodes.map((t) => ({
...t,
resolution: getVideoResolutionLabel(t),
}))
}
function getInitialResolution(resolutions: DisplayTranscode[]): DisplayTranscode | null {
@@ -84,8 +82,7 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
},
ref,
) {
const playerRef = useRef<Player | null>(null)
const videoContainerRef = useRef<HTMLDivElement>(null)
const hlsRef = useRef<Hls | null>(null)
const containerRef = useRef<HTMLDivElement>(null)
const videoRef = useRef<HTMLVideoElement | null>(null)
@@ -110,6 +107,8 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const [playerVolume, setPlayerVolume] = useState(volume)
const [playerMuted, setPlayerMuted] = useState(muted)
const [isPlayerReady, setIsPlayerReady] = useState(false)
const [isLoading, setIsLoading] = useState(true)
const waitingTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const metadata = file.media?.metadata
const frameRate = metadata?.frameRate || 30
@@ -117,11 +116,17 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const containerDuration = metadata?.duration || 0
const totalFrames = resolveTotalFrames({ dbTotalFrames, containerDuration, frameRate })
const isHls = Boolean(file.media?.isHls && file.media.hls?.url)
const resolutions = computeResolutions(file)
const initialRes = getInitialResolution(resolutions)
const [currentResolution, setCurrentResolution] = useState(initialRes?.resolution ?? '')
const initialRes = isHls ? null : getInitialResolution(resolutions)
const [currentResolution, setCurrentResolution] = useState(
isHls ? 'Auto' : (initialRes?.resolution ?? ''),
)
const [isCurrentHdr, setIsCurrentHdr] = useState(initialRes?.hdr)
const currentSrcRef = useRef(initialRes?.url)
const [activeAutoResolution, setActiveAutoResolution] = useState<string | undefined>(undefined)
const pendingResolutionCorrectionRef = useRef<(() => void) | null>(null)
const isHlsManualSupported = !isHls || Boolean(Hls.isSupported())
const currentSrcRef = useRef(isHls ? file.media!.hls!.url : initialRes?.url)
const isAudio = file.proxyType === 'audio'
const { currentFrame, seekToFrame } = useFramePlayer(videoRef, frameRate, totalFrames, isAudio)
@@ -160,78 +165,220 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
// if the parent renders this pane without a per-asset `key`. Runs before the
// video.js init effect below (declaration order) so the ref is fresh.
useEffect(() => {
const res = getInitialResolution(resolutions)
setCurrentResolution(res?.resolution ?? '')
const res = isHls ? null : getInitialResolution(resolutions)
setCurrentResolution(isHls ? 'Auto' : (res?.resolution ?? ''))
setIsCurrentHdr(res?.hdr)
currentSrcRef.current = res?.url
setActiveAutoResolution(undefined)
currentSrcRef.current = isHls ? file.media!.hls!.url : res?.url
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsLoading(true)
setIsPlayerReady(false)
setHasStartedPlaying(false)
}, [file.id])
}, [file.id, Boolean(isHls ? file.media?.hls?.url : initialRes?.url)])
// Initialize video.js
// Initialize player (Hls.js or native HTML5 video)
useEffect(() => {
if (!videoContainerRef.current) return
const videoElement = document.createElement('video-js')
videoElement.classList.add('vjs-big-play-centered', '!h-full', '!w-full')
videoElement.style.pointerEvents = 'none'
videoContainerRef.current.appendChild(videoElement)
const video = videoRef.current
if (!video) return
const player = (playerRef.current = videojs(videoElement, {
controls: false,
autoplay: false,
preload: 'auto',
playsinline: true,
sources: [{ src: currentSrcRef.current ?? '', type: 'video/mp4' }],
}))
const targetSrc = isHls ? file.media!.hls!.url! : (currentSrcRef.current ?? '')
if (!targetSrc) return
const htmlVid = videoElement.querySelector('video')
if (htmlVid) {
videoRef.current = htmlVid
} else {
player.ready(() => {
const techEl = player.tech({ iWillNotUseThisInPlugins: true })?.el() as HTMLVideoElement
if (techEl) {
videoRef.current = techEl
}
})
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
player.on('play', () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
let hls: Hls | null = null
if (isHls) {
if (Hls.isSupported()) {
hls = new Hls({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
})
hlsRef.current = hls
hls.attachMedia(video)
hls.loadSource(targetSrc)
hls.on(Hls.Events.LEVEL_SWITCHED, (_event, eventData) => {
const level = hls?.levels[eventData.level]
if (level) {
const matched = resolutions.find(
(r) => r.width === level.width && r.height === level.height,
)
if (matched?.resolution) {
setActiveAutoResolution(matched.resolution)
} else {
const shortSide =
level.width && level.height
? Math.min(level.width, level.height)
: level.height || level.width
setActiveAutoResolution(shortSide ? `${shortSide}p` : undefined)
}
}
})
hls.on(Hls.Events.ERROR, (_event, eventData) => {
if (eventData.fatal) {
switch (eventData.type) {
case Hls.ErrorTypes.NETWORK_ERROR:
hls?.startLoad()
break
case Hls.ErrorTypes.MEDIA_ERROR:
hls?.recoverMediaError()
break
default:
hls?.destroy()
break
}
}
})
} else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = targetSrc
}
} else {
video.src = targetSrc
}
const clearWaitingTimeout = () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
}
const handlePlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setIsPlaying(true)
setHasStartedPlaying(true)
onPlay?.()
})
player.on('pause', () => setIsPlaying(false))
player.on('ended', () => setIsPlaying(false))
player.on('timeupdate', () => {
setBuffered(player.bufferedPercent())
})
player.on('loadedmetadata', () => {
}
const handlePause = () => setIsPlaying(false)
const handleEnded = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlaying(false)
}
const handleLoadedData = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
})
player.on('loadstart', () => setIsPlayerReady(false))
player.on('volumechange', () => {
setPlayerVolume(player.volume() || 0)
setPlayerMuted(player.muted() || false)
})
player.on('ratechange', () => setPlaybackRate(player.playbackRate() || 1))
}
const handleCanPlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
}
const handlePlaying = () => {
clearWaitingTimeout()
setIsLoading(false)
}
const handleWaiting = () => {
clearWaitingTimeout()
waitingTimeoutRef.current = setTimeout(() => {
setIsLoading(true)
}, 200)
}
const handleLoadStart = () => {
clearWaitingTimeout()
setIsLoading(true)
setIsPlayerReady(false)
}
const handleProgress = () => {
const vidDuration = video.duration || containerDuration || 0
if (vidDuration > 0 && video.buffered.length > 0) {
let bufferedEnd = 0
for (let i = 0; i < video.buffered.length; i++) {
if (
video.buffered.start(i) <= video.currentTime &&
video.currentTime <= video.buffered.end(i)
) {
bufferedEnd = video.buffered.end(i)
break
}
}
if (bufferedEnd === 0) {
bufferedEnd = video.buffered.end(video.buffered.length - 1)
}
setBuffered((bufferedEnd / vidDuration) * 100)
}
}
const handleVolumeChange = () => {
setPlayerVolume(video.volume || 0)
setPlayerMuted(video.muted || false)
}
const handleRateChange = () => setPlaybackRate(video.playbackRate || 1)
if (video.readyState >= 2) {
setIsLoading(false)
setIsPlayerReady(true)
}
video.addEventListener('play', handlePlay)
video.addEventListener('pause', handlePause)
video.addEventListener('ended', handleEnded)
video.addEventListener('loadeddata', handleLoadedData)
video.addEventListener('canplay', handleCanPlay)
video.addEventListener('playing', handlePlaying)
video.addEventListener('waiting', handleWaiting)
video.addEventListener('loadstart', handleLoadStart)
video.addEventListener('timeupdate', handleProgress)
video.addEventListener('progress', handleProgress)
video.addEventListener('volumechange', handleVolumeChange)
video.addEventListener('ratechange', handleRateChange)
return () => {
if (player && !player.isDisposed()) {
player.dispose()
playerRef.current = null
videoRef.current = null
clearWaitingTimeout()
video.removeEventListener('play', handlePlay)
video.removeEventListener('pause', handlePause)
video.removeEventListener('ended', handleEnded)
video.removeEventListener('loadeddata', handleLoadedData)
video.removeEventListener('canplay', handleCanPlay)
video.removeEventListener('playing', handlePlaying)
video.removeEventListener('waiting', handleWaiting)
video.removeEventListener('loadstart', handleLoadStart)
video.removeEventListener('timeupdate', handleProgress)
video.removeEventListener('progress', handleProgress)
video.removeEventListener('volumechange', handleVolumeChange)
video.removeEventListener('ratechange', handleRateChange)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
}
}, [file.id])
// Apply muted / volume (audio routing: only active side unmuted)
useEffect(() => {
const player = playerRef.current
if (!player || !isPlayerReady) return
player.muted(muted)
player.volume(volume)
const video = videoRef.current
if (!video || !isPlayerReady) return
video.muted = muted
video.volume = volume
}, [muted, volume, isPlayerReady])
// Report state upward
@@ -252,6 +399,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
isCurrentHdr,
resolutions,
buffered,
activeAutoResolution,
isHlsManualSupported,
isLoading,
},
}
onStateChange(state)
@@ -268,6 +418,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
currentResolution,
isCurrentHdr,
buffered,
activeAutoResolution,
isHlsManualSupported,
isLoading,
])
// Propagate active playhead time
@@ -323,8 +476,81 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const changeResolution = useCallback(
(resolution: string, hdr?: boolean) => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (isHls) {
if (!hlsRef.current) {
// Safari native HLS fallback does not support manual rendition switching
return
}
const hls = hlsRef.current
const wasPaused = video.paused
const targetFrame = currentFrameRef.current
if (resolution === 'Auto') {
if (currentResolution === 'Auto' && hls.currentLevel === -1) {
return
}
hls.currentLevel = -1
setCurrentResolution('Auto')
} else {
const target = resolutions.find((r) => r.resolution === resolution)
if (target) {
const targetIndex = hls.levels.findIndex(
(lvl) =>
lvl.height === target.height ||
Math.max(lvl.width, lvl.height) === Math.max(target.width, target.height),
)
if (targetIndex !== -1) {
if (targetIndex === hls.currentLevel && currentResolution === resolution) {
return
}
hls.currentLevel = targetIndex
setCurrentResolution(resolution)
setIsCurrentHdr(target.hdr)
}
}
}
if (wasPaused) {
let timer: ReturnType<typeof setTimeout> | null = null
const cleanup = () => {
if (timer) {
clearTimeout(timer)
timer = null
}
video.removeEventListener('seeked', onSeeked)
hls.off?.(Hls.Events.ERROR, cleanup)
if (pendingResolutionCorrectionRef.current === cleanup) {
pendingResolutionCorrectionRef.current = null
}
}
const onSeeked = () => {
cleanup()
const safeCenterTime = calculateFrameCenterTime(targetFrame, frameRate)
const frameDuration = 1 / frameRate
if (Math.abs(video.currentTime - safeCenterTime) > frameDuration / 4) {
video.currentTime = safeCenterTime
}
}
// Safety timeout (1.5s): if no seeked event fires from rendition switch, cleanly detach
timer = setTimeout(cleanup, 1500)
pendingResolutionCorrectionRef.current = cleanup
video.addEventListener('seeked', onSeeked)
hls.once?.(Hls.Events.ERROR, cleanup)
}
return
}
const target =
hdr !== undefined
? resolutions.find(
@@ -332,21 +558,24 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
)
: resolutions.find((r) => r.resolution === resolution)
if (!target) return
const wasPlaying = !player.paused()
const currentT = player.currentTime()
const wasPlaying = !video.paused
const currentT = video.currentTime
setCurrentResolution(resolution)
setIsCurrentHdr(target.hdr)
currentSrcRef.current = target.url
player.src({ type: 'video/mp4', src: target.url })
player.one('loadedmetadata', () => {
player.currentTime(currentT)
video.src = target.url || ''
const onLoadedMetadata = () => {
video.removeEventListener('loadedmetadata', onLoadedMetadata)
video.currentTime = currentT
if (wasPlaying) {
const p = player.play()
const p = video.play()
if (p !== undefined) p.catch(() => {})
}
})
video.playbackRate = playbackRate
}
video.addEventListener('loadedmetadata', onLoadedMetadata)
},
[resolutions],
[resolutions, isHls, frameRate, playbackRate, currentResolution],
)
useImperativeHandle(
@@ -354,47 +583,71 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
(): ComparePaneHandle => ({
getKind: () => 'video',
play: () => {
const p = playerRef.current?.play()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
const p = videoRef.current?.play()
if (p !== undefined) p.catch(() => {})
},
pause: () => playerRef.current?.pause(),
pause: () => videoRef.current?.pause(),
togglePlay: () => {
const player = playerRef.current
if (!player) return
if (player.paused() || player.ended()) {
const p = player.play()
const video = videoRef.current
if (!video) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (video.paused || video.ended) {
const p = video.play()
if (p !== undefined) p.catch(() => {})
} else {
player.pause()
video.pause()
}
},
seekToFrame: (frame) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(clampFrame(frame))
},
seekToSecond: (second) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
const frame = Math.floor(second * frameRate + 0.45)
seekToFrame(clampFrame(frame))
},
stepFrame: (delta) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(clampFrame(currentFrameRef.current + delta))
},
setMuted: (m) => playerRef.current?.muted(m),
setMuted: (m) => {
if (videoRef.current) videoRef.current.muted = m
},
setVolume: (v) => {
const player = playerRef.current
if (!player) return
player.volume(v)
if (v > 0 && player.muted()) player.muted(false)
else if (v === 0 && !player.muted()) player.muted(true)
const video = videoRef.current
if (!video) return
video.volume = v
if (v > 0 && video.muted) video.muted = false
else if (v === 0 && !video.muted) video.muted = true
},
setPlaybackRate: (rate) => {
if (videoRef.current) videoRef.current.playbackRate = rate
},
setPlaybackRate: (rate) => playerRef.current?.playbackRate(rate),
toggleLoop: () => {
const player = playerRef.current
if (!player) return
const next = !player.loop()
player.loop(next)
const video = videoRef.current
if (!video) return
const next = !video.loop
video.loop = next
setIsLooping(next)
},
changeResolution,
@@ -426,7 +679,7 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
],
)
const displayAnnotations = [...annotations, ...(isActive ? draftAnnotations : [])]
const displayAnnotations = [...(annotations ?? []), ...(isActive ? draftAnnotations : [])]
const scale = zoom
const defaultPanX = (containerSize.width - vidW * scale) / 2
@@ -441,8 +694,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
onActivate?.()
return
}
if (isLoading) return
onRequestTogglePlay?.()
}, [isActive, onActivate, onRequestTogglePlay])
}, [isActive, isLoading, onActivate, onRequestTogglePlay])
if (!file.media?.metadata || resolutions.length === 0) {
return (
@@ -474,14 +728,22 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
transformOrigin: '0 0',
}}
>
<div
ref={videoContainerRef}
className="w-full h-full [&_.video-js]:!w-full [&_.video-js]:!h-full [&_video]:!w-full [&_video]:!h-full [&_video]:!block [&_video]:!object-contain"
<video
ref={videoRef}
className="w-full h-full block object-contain pointer-events-none"
playsInline
preload="auto"
poster={file.preview?.thumbnailUrl || undefined}
/>
</div>
)}
{isAudio && (
<div ref={videoContainerRef} className="absolute inset-0 pointer-events-none opacity-0" />
<video
ref={videoRef}
className="absolute inset-0 pointer-events-none opacity-0"
playsInline
preload="auto"
/>
)}
{isAudio ? (
@@ -512,8 +774,23 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
)
)}
{!hasStartedPlaying && !isDrawing && (
<div className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200">
{/* Loading Spinner Overlay */}
{isLoading && (
<div
className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200"
data-testid="compare-video-loading-spinner"
>
<div className="flex h-14 w-14 items-center justify-center rounded-full border-2 border-white/30 bg-white/10 backdrop-blur-sm">
<Loader2 className="h-7 w-7 text-white animate-spin" />
</div>
</div>
)}
{!isLoading && !hasStartedPlaying && !isDrawing && (
<div
className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200"
data-testid="compare-video-play-overlay"
>
<div className="flex h-16 w-16 items-center justify-center rounded-full border-2 border-white/30 bg-white/10 backdrop-blur-sm">
<Play className="ml-1 h-8 w-8 fill-white text-white" />
</div>
@@ -24,7 +24,7 @@ import {
} from '../../ui/dropdown-menu'
import ProgressBar from './progress-bar'
import { formatTime, formatTimecode } from './utils'
import type { ControlBarProps } from './video-control-bar'
import type { ControlBarProps, DisplayTranscode } from './video-control-bar'
export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
state,
@@ -45,7 +45,9 @@ export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
floatOverlayInFullScreen = true,
onMouseEnter,
onMouseLeave,
isLoading: propIsLoading,
}) => {
const isLoading = propIsLoading ?? state.isLoading ?? false
const { videoTimeDisplayMode, setVideoTimeDisplayMode } = useUiStore()
if (!data.media?.metadata) {
@@ -78,214 +80,248 @@ export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter={onMouseEnter}
onMouseLeave={onMouseLeave}
>
{/* Progress Bar */}
<div className="mb-2 px-0.5">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
<div
className={cn(
'transition-opacity duration-200',
isLoading && 'pointer-events-none opacity-60 select-none',
)}
>
{/* Progress Bar */}
<div className="mb-2 px-0.5">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
{/* Control Row */}
<div className="flex items-center justify-between text-foreground gap-1 select-none">
{/* Left Side: Play, Mute, Time */}
<div className="flex items-center gap-2 min-w-0">
<button
onClick={togglePlay}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-5 h-5 fill-current" />
) : (
<Play className="w-5 h-5 fill-current" />
)}
</button>
<button
onClick={toggleMute}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
aria-label={state.isMuted ? 'Unmute' : 'Mute'}
>
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-5 h-5" />
) : (
<Volume2 className="w-5 h-5" />
)}
</button>
<div className="flex items-center gap-0.5 text-xs font-medium tabular-nums min-w-0">
<span
className="text-muted-foreground font-mono truncate text-[11px] sm:text-xs"
data-testid="time-readout"
{/* Control Row */}
<div className="flex items-center justify-between text-foreground gap-1 select-none">
{/* Left Side: Play, Mute, Time */}
<div className="flex items-center gap-2 min-w-0">
<button
onClick={togglePlay}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{displayString}
</span>
{state.isPlaying ? (
<Pause className="w-5 h-5 fill-current" />
) : (
<Play className="w-5 h-5 fill-current" />
)}
</button>
<button
onClick={toggleMute}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
aria-label={state.isMuted ? 'Unmute' : 'Mute'}
>
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-5 h-5" />
) : (
<Volume2 className="w-5 h-5" />
)}
</button>
<div className="flex items-center gap-0.5 text-xs font-medium tabular-nums min-w-0">
<span
className="text-muted-foreground font-mono truncate text-[11px] sm:text-xs"
data-testid="time-readout"
>
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="h-5 w-5 inline-flex items-center justify-center rounded hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none shrink-0"
aria-label={m.change_time_format()}
>
<ChevronDown className="h-3 w-3" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.format()}
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.standard_time()}</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.frames()}</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.timecode()}</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Settings (Quality + Speed) & Fullscreen */}
<div className="flex items-center gap-1 shrink-0">
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="h-5 w-5 inline-flex items-center justify-center rounded hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none shrink-0"
aria-label={m.change_time_format()}
className="flex items-center gap-1 rounded border border-border px-1.5 py-1 text-xs font-semibold hover:bg-muted transition-colors cursor-pointer"
title={m.settings()}
aria-label={m.settings()}
>
<ChevronDown className="h-3 w-3" />
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1 text-[11px]">
<span>
{!isAudio && state.currentResolution
? state.currentResolution === 'Auto'
? state.activeAutoResolution
? `${m.quality_auto()} (${state.activeAutoResolution})`
: m.quality_auto()
: state.currentResolution === 'Original'
? m.original()
: state.currentResolution
: `${state.playbackRate}x`}
</span>
{!isAudio && state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuContent align="end" className="w-48 z-30">
{!isAudio && resolutions.length > 0 && (
<>
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.quality()}
</DropdownMenuLabel>
{data.media?.isHls && (
<DropdownMenuItem
onClick={() =>
changeResolution({
resolution: 'Auto',
width: data.media?.metadata?.originalWidth ?? 0,
height: data.media?.metadata?.originalHeight ?? 0,
} as DisplayTranscode)
}
className={cn(
'flex w-full items-center justify-between text-xs cursor-pointer',
state.currentResolution === 'Auto'
? 'text-primary font-medium'
: 'text-foreground',
)}
>
<span>{m.quality_auto()}</span>
</DropdownMenuItem>
)}
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
const isManualDisabled =
Boolean(data.media?.isHls) && state.isHlsManualSupported === false
return (
<DropdownMenuItem
key={itemKey}
disabled={isManualDisabled}
onClick={() => !isManualDisabled && changeResolution(res)}
className={cn(
'flex w-full items-center justify-between text-xs',
isSelected ? 'text-primary font-medium' : 'text-foreground',
isManualDisabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
)}
>
<div className="flex items-center gap-1.5">
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-[10px] text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
<DropdownMenuSeparator />
</>
)}
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.format()}
{m.playback_speed()}
</DropdownMenuLabel>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'flex items-center justify-between text-xs cursor-pointer',
state.playbackRate === rate ? 'text-primary font-medium' : 'text-foreground',
)}
>
<span>{rate}x</span>
{state.playbackRate === rate && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
))}
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
onClick={toggleLoop}
className="flex items-center justify-between text-xs cursor-pointer"
>
<span>{m.standard_time()}</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.frames()}</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.timecode()}</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
<span className="flex items-center gap-1.5">
<Repeat className="h-3.5 w-3.5" />
<span>{m.loop()}</span>
</span>
{state.isLooping && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Settings (Quality + Speed) & Fullscreen */}
<div className="flex items-center gap-1 shrink-0">
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="flex items-center gap-1 rounded border border-border px-1.5 py-1 text-xs font-semibold hover:bg-muted transition-colors cursor-pointer"
title={m.settings()}
aria-label={m.settings()}
>
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1 text-[11px]">
<span>
{!isAudio && state.currentResolution
? state.currentResolution === 'Original'
? m.original()
: state.currentResolution
: `${state.playbackRate}x`}
</span>
{!isAudio && state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-48 z-30">
{!isAudio && resolutions.length > 0 && (
<>
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.quality()}
</DropdownMenuLabel>
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
return (
<DropdownMenuItem
key={itemKey}
onClick={() => changeResolution(res)}
className={cn(
'flex w-full items-center justify-between text-xs cursor-pointer',
isSelected ? 'text-primary font-medium' : 'text-foreground',
)}
>
<div className="flex items-center gap-1.5">
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-[10px] text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
<DropdownMenuSeparator />
</>
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
title={m.fullscreen()}
aria-label={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-5 w-5" />
) : (
<Maximize className="h-5 w-5" />
)}
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.playback_speed()}
</DropdownMenuLabel>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'flex items-center justify-between text-xs cursor-pointer',
state.playbackRate === rate ? 'text-primary font-medium' : 'text-foreground',
)}
>
<span>{rate}x</span>
{state.playbackRate === rate && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
))}
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={toggleLoop}
className="flex items-center justify-between text-xs cursor-pointer"
>
<span className="flex items-center gap-1.5">
<Repeat className="h-3.5 w-3.5" />
<span>{m.loop()}</span>
</span>
{state.isLooping && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
title={m.fullscreen()}
aria-label={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-5 w-5" />
) : (
<Maximize className="h-5 w-5" />
)}
</button>
</button>
</div>
</div>
</div>
</div>
@@ -1,7 +1,7 @@
import { cn } from '@/ui/lib/utils'
import { m } from '@/ui/paraglide/messages.js'
import { useUiStore } from '@/ui/stores/ui'
import type { AssetInfo, VideoTranscode } from '@shumai/dtos'
import type { AssetInfo } from '@shumai/dtos'
import {
Check,
ChevronDown,
@@ -29,6 +29,7 @@ import {
import { Slider } from '../../ui/slider'
import ProgressBar from './progress-bar'
import { formatTime, formatTimecode } from './utils'
import { getVideoResolutionLabel } from '@/ui/lib/media'
export interface PlayerState {
isPlaying: boolean
@@ -44,9 +45,21 @@ export interface PlayerState {
currentResolution: string // 'Original' or '720p', '480p' etc.
currentSrc?: string
isCurrentHdr?: boolean
activeAutoResolution?: string
isHlsManualSupported?: boolean
isLoading?: boolean
}
export type DisplayTranscode = VideoTranscode & { resolution: string }
export interface DisplayTranscode {
id?: string
url?: string
key?: string
width: number
height: number
size?: number
hdr?: boolean
resolution: string
}
export interface ControlBarProps {
state: PlayerState
@@ -81,6 +94,7 @@ export interface ControlBarProps {
onMouseLeave?: () => void
/** When false, hides the download affordance. Defaults to true. */
allowDownload?: boolean
isLoading?: boolean
}
export const VideoControlBar: React.FC<ControlBarProps> = ({
@@ -108,7 +122,9 @@ export const VideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter,
onMouseLeave,
allowDownload = true,
isLoading: propIsLoading,
}) => {
const isLoading = propIsLoading ?? state.isLoading ?? false
const { videoTimeDisplayMode, setVideoTimeDisplayMode } = useUiStore()
if (!data.media?.metadata) {
@@ -144,301 +160,341 @@ export const VideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter={onMouseEnter}
onMouseLeave={onMouseLeave}
>
{/* Progress Bar */}
<div className="mb-4 px-1">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
<div
className={cn(
'transition-opacity duration-200',
isLoading && 'pointer-events-none opacity-60 select-none',
)}
>
{/* Progress Bar */}
<div className="mb-4 px-1">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
{/* Lower Controls Row */}
<div className="flex items-center justify-between text-foreground">
{/* Left Side: Play, Vol, Time */}
<div className="flex items-center gap-4">
<button
onClick={togglePlay}
className="hover:text-primary transition-colors"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-6 h-6 fill-current" />
) : (
<Play className="w-6 h-6 fill-current" />
)}
</button>
<button
onClick={toggleLoop}
className={cn(
'transition-colors',
state.isLooping ? 'text-primary' : 'text-muted-foreground',
)}
title="Loop"
>
<Repeat className="w-5 h-5" />
</button>
<div className="flex items-center gap-3 group/vol">
<button onClick={toggleMute} className="hover:text-primary">
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-6 h-6" />
{/* Lower Controls Row */}
<div className="flex items-center justify-between text-foreground">
{/* Left Side: Play, Vol, Time */}
<div className="flex items-center gap-4">
<button
onClick={togglePlay}
className="hover:text-primary transition-colors"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-6 h-6 fill-current" />
) : (
<Volume2 className="w-6 h-6" />
<Play className="w-6 h-6 fill-current" />
)}
</button>
<div className="w-0 overflow-hidden group-hover/vol:w-24 transition-all duration-300 ease-in-out">
<div className="w-24 px-1">
<Slider
value={state.isMuted ? [0] : [state.volume]}
onValueChange={(v: number[]) => handleVolumeChange(v[0])}
max={1}
step={0.05}
/>
<button
onClick={toggleLoop}
className={cn(
'transition-colors',
state.isLooping ? 'text-primary' : 'text-muted-foreground',
)}
title="Loop"
>
<Repeat className="w-5 h-5" />
</button>
<div className="flex items-center gap-3 group/vol">
<button onClick={toggleMute} className="hover:text-primary">
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-6 h-6" />
) : (
<Volume2 className="w-6 h-6" />
)}
</button>
<div className="w-0 overflow-hidden group-hover/vol:w-24 transition-all duration-300 ease-in-out">
<div className="w-24 px-1">
<Slider
value={state.isMuted ? [0] : [state.volume]}
onValueChange={(v: number[]) => handleVolumeChange(v[0])}
max={1}
step={0.05}
/>
</div>
</div>
</div>
</div>
<div className="flex items-center gap-1 text-sm font-medium tabular-nums select-none min-w-[100px]">
<span className="text-muted-foreground font-mono" data-testid="time-readout">
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button className="h-6 w-6 inline-flex items-center justify-center rounded-md hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none">
<ChevronDown className="h-3.5 w-3.5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
Format
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Standard Time</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Frames</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Timecode</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Zoom, Speed, Res, Download, Fullscreen */}
<div className="relative flex items-center gap-3">
{/* Zoom Controls */}
{!isAudio && (
<div className="flex items-center gap-1">
<div className="flex items-center gap-1 rounded-md bg-muted p-0.5">
<button
onClick={() => onZoomChange(zoom * 0.8)}
className="p-1 hover:text-primary rounded"
title={m.zoom_out()}
>
<Minus size={14} />
</button>
<span className="text-xs w-8 text-center tabular-nums">
{Math.round(zoom * 100)}%
</span>
<button
onClick={() => onZoomChange(zoom * 1.2)}
className="p-1 hover:text-primary rounded"
title={m.zoom_in()}
>
<Plus size={14} />
</button>
</div>
<button
onClick={onZoomReset}
className="text-xs font-medium px-2 py-1 rounded bg-muted hover:text-primary transition-colors"
>
{m.fit()}
</button>
</div>
)}
{/* Playback Rate */}
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="w-8 text-sm font-bold hover:text-primary">
{state.playbackRate}x
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'rounded py-1 px-2 text-left text-xs',
state.playbackRate === rate
? 'bg-primary text-primary-foreground focus:bg-primary focus:text-primary-foreground'
: 'text-foreground',
)}
>
{rate}x
</DropdownMenuItem>
))}
</DropdownMenuContent>
</DropdownMenu>
{/* Settings / Resolution */}
{!isAudio && resolutions.length > 0 && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="flex items-center gap-1.5 rounded border border-border px-2 py-0.5 text-sm font-semibold hover:bg-muted transition-colors">
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1">
<span>
{state.currentResolution === 'Original'
? m.original()
: state.currentResolution}
</span>
{state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.quality()}</DropdownMenuLabel>
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
return (
<DropdownMenuItem
key={itemKey}
onClick={() => changeResolution(res)}
className={cn(
'flex w-full items-center justify-between',
isSelected ? 'text-primary font-medium' : 'text-foreground',
)}
>
<div className="flex items-center gap-1.5">
<span>{res.resolution === 'Original' ? m.original() : res.resolution}</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="pl-5 text-xs text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
</DropdownMenuContent>
</DropdownMenu>
)}
{/* Download */}
{allowDownload &&
(isAudio ? (
<button
onClick={() => handleDownload(data.media?.original?.key || '')}
className="transition-colors hover:text-primary"
title={m.download()}
disabled={!data.media?.original?.key}
>
<Download className="h-5 w-5" />
</button>
) : (
<DropdownMenu>
<div className="flex items-center gap-1 text-sm font-medium tabular-nums select-none min-w-[100px]">
<span className="text-muted-foreground font-mono" data-testid="time-readout">
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button className="transition-colors hover:text-primary" title={m.download()}>
<Download className="h-5 w-5" />
<button className="h-6 w-6 inline-flex items-center justify-center rounded-md hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none">
<ChevronDown className="h-3.5 w-3.5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
Format
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Standard Time</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Frames</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Timecode</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Zoom, Speed, Res, Download, Fullscreen */}
<div className="relative flex items-center gap-3">
{/* Zoom Controls */}
{!isAudio && (
<div className="flex items-center gap-1">
<div className="flex items-center gap-1 rounded-md bg-muted p-0.5">
<button
onClick={() => onZoomChange(zoom * 0.8)}
className="p-1 hover:text-primary rounded"
title={m.zoom_out()}
>
<Minus size={14} />
</button>
<span className="text-xs w-8 text-center tabular-nums">
{Math.round(zoom * 100)}%
</span>
<button
onClick={() => onZoomChange(zoom * 1.2)}
className="p-1 hover:text-primary rounded"
title={m.zoom_in()}
>
<Plus size={14} />
</button>
</div>
<button
onClick={onZoomReset}
className="text-xs font-medium px-2 py-1 rounded bg-muted hover:text-primary transition-colors"
>
{m.fit()}
</button>
</div>
)}
{/* Playback Rate */}
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="w-8 text-sm font-bold hover:text-primary">
{state.playbackRate}x
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'rounded py-1 px-2 text-left text-xs',
state.playbackRate === rate
? 'bg-primary text-primary-foreground focus:bg-primary focus:text-primary-foreground'
: 'text-foreground',
)}
>
{rate}x
</DropdownMenuItem>
))}
</DropdownMenuContent>
</DropdownMenu>
{/* Settings / Resolution */}
{!isAudio && resolutions.length > 0 && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="flex items-center gap-1.5 rounded border border-border px-2 py-0.5 text-sm font-semibold hover:bg-muted transition-colors">
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1">
<span>
{state.currentResolution === 'Auto'
? state.activeAutoResolution
? `${m.quality_auto()} (${state.activeAutoResolution})`
: m.quality_auto()
: state.currentResolution === 'Original'
? m.original()
: state.currentResolution}
</span>
{state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
<DropdownMenuLabel>{m.quality()}</DropdownMenuLabel>
{data.media?.isHls && (
<DropdownMenuItem
onClick={() =>
changeResolution({
resolution: 'Auto',
width: data.media?.metadata?.originalWidth ?? 0,
height: data.media?.metadata?.originalHeight ?? 0,
} as DisplayTranscode)
}
className={cn(
'flex w-full items-center justify-between cursor-pointer',
state.currentResolution === 'Auto'
? 'text-primary font-medium'
: 'text-foreground',
)}
>
<span>{m.quality_auto()}</span>
</DropdownMenuItem>
)}
{resolutions.map((res) => {
const itemKey = res.key || `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
const isManualDisabled =
Boolean(data.media?.isHls) && state.isHlsManualSupported === false
return (
<DropdownMenuItem
key={itemKey}
onClick={() => handleDownload(res.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground cursor-pointer"
disabled={isManualDisabled}
onClick={() => !isManualDisabled && changeResolution(res)}
className={cn(
'flex w-full items-center justify-between',
isSelected ? 'text-primary font-medium' : 'text-foreground',
isManualDisabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
)}
>
<div className="flex items-center gap-1.5">
<span>{res.resolution}</span>
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-xs text-muted-foreground">MP4</span>
<span className="pl-5 text-xs text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
{data.media?.original?.key && (
<>
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => handleDownload(data.media?.original?.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground"
>
<span>{m.original()}</span>
<span className="text-xs text-muted-foreground">
{data.name?.split('.').pop()?.toUpperCase() || 'RAW'}
</span>
</DropdownMenuItem>
</>
)}
</DropdownMenuContent>
</DropdownMenu>
))}
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="transition-colors hover:text-primary"
title={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-6 w-6" />
) : (
<Maximize className="h-6 w-6" />
)}
</button>
{/* Download */}
{allowDownload &&
(isAudio ? (
<button
onClick={() => handleDownload(data.media?.original?.key || '')}
className="transition-colors hover:text-primary"
title={m.download()}
disabled={!data.media?.original?.key}
>
<Download className="h-5 w-5" />
</button>
) : (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="transition-colors hover:text-primary" title={m.download()}>
<Download className="h-5 w-5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
{(data.media?.videoTranscodes && data.media.videoTranscodes.length > 0
? data.media.videoTranscodes
: resolutions
).map((res) => {
const resLabel = getVideoResolutionLabel(res)
const itemKey = res.key || `${resLabel}-${res.hdr ? 'hdr' : 'sdr'}`
return (
<DropdownMenuItem
key={itemKey}
onClick={() => handleDownload(res.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground cursor-pointer"
>
<div className="flex items-center gap-1.5">
<span>{resLabel}</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-xs text-muted-foreground">MP4</span>
</DropdownMenuItem>
)
})}
{data.media?.original?.key && (
<>
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => handleDownload(data.media?.original?.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground"
>
<span>{m.original()}</span>
<span className="text-xs text-muted-foreground">
{data.name?.split('.').pop()?.toUpperCase() || 'RAW'}
</span>
</DropdownMenuItem>
</>
)}
</DropdownMenuContent>
</DropdownMenu>
))}
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="transition-colors hover:text-primary"
title={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-6 w-6" />
) : (
<Maximize className="h-6 w-6" />
)}
</button>
</div>
</div>
</div>
</div>
@@ -1,28 +1,52 @@
// @vitest-environment happy-dom
import { act, cleanup, render } from '@testing-library/react'
import { act, cleanup, fireEvent, render } from '@testing-library/react'
import React from 'react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import VideoViewer from './video-viewer'
import type { AssetInfo } from '@shumai/dtos'
const videojsMock = vi.fn((...args: unknown[]) => {
void args
return {
const { mockHlsInstance, mockHlsConstructor } = vi.hoisted(() => {
const instance = {
attachMedia: vi.fn(),
loadSource: vi.fn(),
on: vi.fn(),
volume: vi.fn(),
duration: vi.fn(() => 10),
currentTime: vi.fn(() => 0),
bufferedEnd: vi.fn(() => 0),
playbackRate: vi.fn(() => 1),
ready: vi.fn(),
dispose: vi.fn(),
isDisposed: vi.fn(() => false),
off: vi.fn(),
once: vi.fn(),
destroy: vi.fn(),
levels: [
{ height: 1080, width: 1920, bitrate: 5000000 },
{ height: 720, width: 1280, bitrate: 2500000 },
],
currentLevel: -1,
startLoad: vi.fn(),
recoverMediaError: vi.fn(),
}
const constructor = vi.fn(function () {
return instance
})
return { mockHlsInstance: instance, mockHlsConstructor: constructor }
})
vi.mock('video.js', () => ({
default: (...args: unknown[]) => videojsMock(...args),
}))
vi.mock('hls.js', () => {
const MockHls = mockHlsConstructor
// @ts-expect-error adding static mock property
MockHls.isSupported = vi.fn(() => true)
// @ts-expect-error adding static mock property
MockHls.Events = {
MANIFEST_PARSED: 'hlsManifestParsed',
LEVEL_SWITCHED: 'hlsLevelSwitched',
ERROR: 'hlsError',
}
// @ts-expect-error adding static mock property
MockHls.ErrorTypes = {
NETWORK_ERROR: 'networkError',
MEDIA_ERROR: 'mediaError',
OTHER_ERROR: 'otherError',
}
return {
default: MockHls,
}
})
vi.mock('@/ui/components/drawing-canvas', () => ({
default: () => <div data-testid="drawing-canvas" />,
@@ -34,7 +58,8 @@ vi.mock('@/ui/paraglide/messages.js', () => ({
describe('VideoViewer', () => {
beforeEach(() => {
videojsMock.mockClear()
vi.clearAllMocks()
mockHlsInstance.currentLevel = -1
})
afterEach(() => {
@@ -88,39 +113,20 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
const { rerender } = render(<VideoViewer file={videoA} />)
const { container, rerender } = render(<VideoViewer file={videoA} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Initial render should initialize videojs with video A URL
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/videoA-720p.mp4',
type: 'video/mp4',
},
],
}),
)
// Initial render should set video element src with video A URL
expect(video.src).toBe('https://cdn.example.com/videoA-720p.mp4')
// Rerender with video B (simulating selecting another video from the carousel)
rerender(<VideoViewer file={videoB} />)
// VideoJS should be re-initialized with video B's URL
expect(videojsMock).toHaveBeenLastCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/videoB-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
// Native video element should now have video B's URL
expect(video.src).toBe('https://cdn.example.com/videoB-1080p.mp4')
})
it('initializes VideoJS when transcode URL becomes available after initially empty with same ID', () => {
it('initializes video when transcode URL becomes available after initially empty with same ID', () => {
const initialVideo: AssetInfo = {
id: 'video-pending',
name: 'pending.mp4',
@@ -136,7 +142,6 @@ describe('VideoViewer', () => {
videoTranscodes: [
{
resolution: '1080p',
// URL not yet presigned (e.g. from file list payload)
url: undefined as unknown as string,
width: 1920,
height: 1080,
@@ -160,27 +165,17 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
const { rerender } = render(<VideoViewer file={initialVideo} autoPlay={true} />)
const { container, rerender } = render(<VideoViewer file={initialVideo} autoPlay={true} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Initially targetSrc is empty, videojs should not be initialized
expect(videojsMock).not.toHaveBeenCalled()
// Initially targetSrc is empty, video.src should be empty
expect(video.src).toBe('')
// Rerender with detailedVideo (same id, but signed URL now available)
rerender(<VideoViewer file={detailedVideo} autoPlay={true} />)
// VideoJS should now be initialized with the new URL
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
autoplay: false,
sources: [
{
src: 'https://cdn.example.com/signed-pending-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
// Video element should now have the newly available signed URL
expect(video.src).toBe('https://cdn.example.com/signed-pending-1080p.mp4')
})
it('auto-selects HDR proxy when display supports HDR (dynamic-range: high)', () => {
@@ -227,19 +222,10 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
render(<VideoViewer file={hdrVideo} />)
const { container } = render(<VideoViewer file={hdrVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/video-1080p-hdr.mp4',
type: 'video/mp4',
},
],
}),
)
expect(video.src).toBe('https://cdn.example.com/video-1080p-hdr.mp4')
window.matchMedia = originalMatchMedia
})
@@ -288,39 +274,15 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
render(<VideoViewer file={hdrVideo} />)
const { container } = render(<VideoViewer file={hdrVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/video-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
expect(video.src).toBe('https://cdn.example.com/video-1080p.mp4')
window.matchMedia = originalMatchMedia
})
it('shows play button and darkened overlay initially, but never again after playing (even when paused or scrubbed back to start)', () => {
const listeners: Record<string, () => void> = {}
videojsMock.mockImplementation(() => ({
on: vi.fn((event: string, cb: () => void) => {
listeners[event] = cb
}),
volume: vi.fn(),
duration: vi.fn(() => 10),
currentTime: vi.fn(() => 0),
bufferedEnd: vi.fn(() => 0),
playbackRate: vi.fn(() => 1),
ready: vi.fn(),
dispose: vi.fn(),
isDisposed: vi.fn(() => false),
}))
const testVideo: AssetInfo = {
id: 'test-video-lifecycle',
name: 'test.mp4',
@@ -346,27 +308,503 @@ describe('VideoViewer', () => {
const { container } = render(<VideoViewer file={testVideo} />)
const videoArea = container.querySelector('[data-testid="video-area"]')
const video = videoArea?.querySelector('video') as HTMLVideoElement
expect(videoArea).toBeTruthy()
expect(video).toBeTruthy()
// 1. Initially (before playing): play button & 20% black tint are present
expect(videoArea?.querySelector('.bg-black\\/20')).not.toBeNull()
// 2. Start playback
act(() => {
listeners['play']?.()
video.dispatchEvent(new Event('play'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
// 3. Pause video: overlay must NOT show again
act(() => {
listeners['pause']?.()
video.dispatchEvent(new Event('pause'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
// 4. Drag seekbar / time back to 0: overlay must STILL NOT show
act(() => {
listeners['timeupdate']?.()
video.dispatchEvent(new Event('timeupdate'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
})
it('initializes Hls.js with capLevelToPlayerSize false when video is HLS', () => {
const hlsVideo: AssetInfo = {
id: 'video-hls',
name: 'video-hls.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(mockHlsConstructor).toHaveBeenCalledWith(
expect.objectContaining({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
}),
)
expect(mockHlsInstance.attachMedia).toHaveBeenCalledWith(video)
expect(mockHlsInstance.loadSource).toHaveBeenCalledWith('https://cdn.example.com/master.m3u8')
})
it('switches HLS resolution immediately using currentLevel', () => {
const hlsVideo: AssetInfo = {
id: 'video-hls-switch',
name: 'video-hls-switch.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
// Open settings / quality dropdown
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton).toBeTruthy()
fireEvent.pointerDown(settingsButton!, { pointerType: 'mouse', button: 0 })
fireEvent.click(settingsButton!)
// Click 720p (index 1 in levels: 1080p is index 0, 720p is index 1)
const items = document.querySelectorAll('[role="menuitem"]')
const item720p = Array.from(items).find((el) => el.textContent?.includes('720p'))
expect(item720p).toBeTruthy()
act(() => {
fireEvent.click(item720p!)
})
// currentLevel should be set immediately to 1 (720p)
expect(mockHlsInstance.currentLevel).toBe(1)
// Click Auto
const itemAuto = Array.from(document.querySelectorAll('[role="menuitem"]')).find(
(el) => el.textContent?.includes('Auto') || el.textContent === '',
)
if (itemAuto) {
act(() => {
fireEvent.click(itemAuto)
})
// Auto should reset currentLevel to -1
expect(mockHlsInstance.currentLevel).toBe(-1)
}
})
it('does not hijack subsequent seeks if paused HLS switch did not emit seeked', () => {
vi.useFakeTimers()
try {
const hlsVideo: AssetInfo = {
id: 'video-hls-stale-seek',
name: 'video-hls-stale-seek.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Video is paused at 0s (frame 0)
Object.defineProperty(video, 'paused', { value: true, configurable: true })
video.currentTime = 0
// Open settings and click 720p
const settingsButton = container.querySelector('button:has(.lucide-settings)')
fireEvent.pointerDown(settingsButton!, { pointerType: 'mouse', button: 0 })
fireEvent.click(settingsButton!)
const item720p = Array.from(document.querySelectorAll('[role="menuitem"]')).find((el) =>
el.textContent?.includes('720p'),
)
act(() => {
fireEvent.click(item720p!)
})
// Simulate that no seeked event occurred within the timeout (e.g. 1500ms elapses)
act(() => {
vi.advanceTimersByTime(2000)
})
// Now user performs an unrelated seek to frame 150 (currentTime = 5.0)
video.currentTime = 5.0
act(() => {
fireEvent(video, new Event('seeked'))
})
// video.currentTime must remain at frame 150, NOT rewound to frame 0
expect(Math.floor(video.currentTime * 30 + 0.001)).toBe(150)
expect(video.currentTime).toBeCloseTo(5.0167, 3)
} finally {
vi.useRealTimers()
}
})
it('displays 2160p instead of 3840p for 4K vertical video when HLS switches level on Auto', () => {
mockHlsInstance.levels = [{ height: 3840, width: 2160, bitrate: 10000000 }]
const hlsVideo: AssetInfo = {
id: 'video-hls-vertical-4k',
name: 'vertical-4k.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [{ width: 2160, height: 3840, resolution: '2160p' }],
},
metadata: {
originalWidth: 2160,
originalHeight: 3840,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const levelSwitchedHandler = mockHlsInstance.on.mock.calls.find(
([event]) => event === 'hlsLevelSwitched',
)?.[1]
expect(levelSwitchedHandler).toBeDefined()
act(() => {
levelSwitchedHandler('hlsLevelSwitched', { level: 0 })
})
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton?.textContent).toContain('2160p')
expect(settingsButton?.textContent).not.toContain('3840p')
})
it('derives resolution from long side for legacy portrait MP4 transcode without explicit resolution', () => {
const legacyPortraitVideo: AssetInfo = {
id: 'legacy-portrait-mp4',
name: 'portrait.mp4',
proxyType: 'video',
media: {
videoTranscodes: [
{
key: 'portrait-1080p.mp4',
url: 'https://cdn.example.com/portrait-1080p.mp4',
width: 1080,
height: 1920,
size: 5000000,
},
],
metadata: {
originalWidth: 1080,
originalHeight: 1920,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={legacyPortraitVideo} />)
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton?.textContent).toContain('1080p')
expect(settingsButton?.textContent).not.toContain('1920p')
})
it('sets poster on video element from file.preview.thumbnailUrl', () => {
const videoWithThumb: AssetInfo = {
id: 'video-thumb',
name: 'thumb.mp4',
proxyType: 'video',
preview: {
thumbnailUrl: 'https://cdn.example.com/poster.jpg',
},
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={videoWithThumb} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(video.getAttribute('poster')).toBe('https://cdn.example.com/poster.jpg')
})
it('shows loading spinner initially and transitions to play overlay on loadeddata', () => {
const testVideo: AssetInfo = {
id: 'video-loading-test',
name: 'loading.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// 1. Initially loading spinner is visible, play overlay is not
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
expect(container.querySelector('[data-testid="video-play-overlay"]')).toBeNull()
// 2. Control bar inner wrapper is dimmed & non-interactive
const controlBarWrapper = container.querySelector('.pointer-events-none.opacity-60')
expect(controlBarWrapper).not.toBeNull()
// 3. Dispatch loadeddata
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
// Spinner is gone, play overlay is visible, controls are interactive
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
expect(container.querySelector('[data-testid="video-play-overlay"]')).not.toBeNull()
expect(container.querySelector('.pointer-events-none.opacity-60')).toBeNull()
})
it('debounces waiting stall with 200ms delay and cancels spinner on quick recovery', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'video-stall-test',
name: 'stall.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Become ready first
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Stalls briefly (<200ms)
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Immediately after waiting, spinner should NOT be shown yet (debounced)
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Recovers quickly at 100ms
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('playing'))
})
act(() => {
vi.advanceTimersByTime(150)
})
// Spinner still not shown
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Now a long stall (>200ms)
act(() => {
video.dispatchEvent(new Event('waiting'))
})
act(() => {
vi.advanceTimersByTime(250)
})
// Spinner should now appear!
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
// Resume playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
it('does not cancel waiting stall debounce when progress events fire during stall', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'video-stall-progress-test',
name: 'stall-progress.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// 1. Initial ready state
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 2. Play starts (paused === false)
Object.defineProperty(video, 'paused', { value: false, configurable: true, writable: true })
act(() => {
video.dispatchEvent(new Event('playing'))
})
// 3. Stalls: dispatches waiting
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Spinner not visible immediately (within 200ms debounce)
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 4. Progress event fires at 100ms while paused is false (data still trickling in)
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('progress'))
})
// Should NOT clear debounce or mark loading false
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 5. Advance past 200ms debounce (advance remaining 110ms)
act(() => {
vi.advanceTimersByTime(110)
})
// Spinner should now appear despite the progress event!
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
// 6. Playback resumes: dispatch playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
// Spinner disappears
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
it('renders failed state when file has no media and status is failed', () => {
const failedVideo = {
id: 'failed-video',
name: 'failed-video.mp4',
proxyType: 'video',
status: 'failed',
media: {
error: 'Transcoding failed due to server error',
},
} as unknown as AssetInfo
const { getByTestId, getByText } = render(<VideoViewer file={failedVideo} />)
expect(getByTestId('video-viewer-failed-state')).toBeDefined()
expect(getByText('Transcoding failed due to server error')).toBeDefined()
})
})
@@ -1,16 +1,17 @@
import { client } from '@/ui/api/client'
import { m } from '@/ui/paraglide/messages.js'
import { cn } from '@/ui/lib/utils'
import { Play, AudioLines } from 'lucide-react'
import { Play, AudioLines, Loader2, AlertCircle } from 'lucide-react'
import React, { useCallback, useEffect, useRef, useState, useImperativeHandle } from 'react'
import videojs from 'video.js'
import type Player from 'video.js/dist/types/player'
import Hls from 'hls.js'
import { useFramePlayer } from './use-frame-player'
import { resolveTotalFrames } from './utils'
import { calculateFrameCenterTime, resolveTotalFrames } from './utils'
import { VideoControlBar, type PlayerState, type DisplayTranscode } from './video-control-bar'
import { MobileVideoControlBar } from './mobile-video-control-bar'
import { useIsMobile } from '@/ui/hooks/use-mobile'
import DrawingCanvas from '@/ui/components/drawing-canvas'
import { useAnnotationStore } from '@/ui/stores/annotation-store'
import { getVideoResolutionLabel } from '@/ui/lib/media'
import { FileViewerProps, MediaController } from '../types'
import { centeredPan, fitScale, zoomAtPoint } from '../pan-zoom'
import { usePanZoomGestures } from '../use-pan-zoom'
@@ -31,28 +32,22 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
},
ref,
) => {
const localPlayerRef = useRef<Player | null>(null)
const playerRef = localPlayerRef
const resolutions: DisplayTranscode[] = (data.media?.videoTranscodes ?? []).map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const hlsRef = useRef<Hls | null>(null)
const videoRef = useRef<HTMLVideoElement | null>(null)
const [activeAutoResolution, setActiveAutoResolution] = useState<string | undefined>(undefined)
const [isHlsManualSupported, setIsHlsManualSupported] = useState<boolean>(true)
const isHls = Boolean(data.media?.isHls && data.media.hls?.url)
const baseTranscodes = isHls
? (data.media?.hls?.resolutions ?? [])
: (data.media?.videoTranscodes ?? [])
return {
...t,
resolution,
}
})
const resolutions: DisplayTranscode[] = baseTranscodes.map((t) => ({
...t,
resolution: getVideoResolutionLabel(t),
}))
// Only transcoded proxy versions are ever displayed; the raw original file
// is never used as a playback source.
const hasMedia = resolutions.length > 0 && !!data.media?.metadata
// We use a container ref to manually append the video element
const videoContainerRef = useRef<HTMLDivElement>(null)
const hasMedia = (isHls || resolutions.length > 0) && !!data.media?.metadata
// Logic to select best resolution based on screen size and dynamic range
const getInitialResolution = (): DisplayTranscode | null => {
@@ -90,7 +85,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
return bestFit || sortedResolutions[sortedResolutions.length - 1]
}
const initialRes = getInitialResolution()
const initialRes = isHls ? null : getInitialResolution()
// State
const [state, setState] = useState<PlayerState>({
@@ -104,66 +99,75 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
playbackRate: 1,
isFullScreen: false,
showFrames: false,
currentResolution: initialRes?.resolution ?? '',
currentSrc: initialRes?.url ?? '',
currentResolution: isHls ? 'Auto' : (initialRes?.resolution ?? ''),
currentSrc: isHls ? data.media!.hls!.url : (initialRes?.url ?? ''),
isCurrentHdr: initialRes?.hdr ?? false,
})
const [hasStartedPlaying, setHasStartedPlaying] = useState(false)
const containerRef = useRef<HTMLDivElement>(null)
const rootRef = useRef<HTMLDivElement>(null)
const controlsTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const pendingResolutionCorrectionRef = useRef<(() => void) | null>(null)
useImperativeHandle(ref, () => ({
play: () => {
if (localPlayerRef.current) {
localPlayerRef.current.play()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
videoRef.current?.play().catch(() => {})
},
pause: () => {
if (localPlayerRef.current) {
localPlayerRef.current.pause()
}
videoRef.current?.pause()
},
seekTo: (second: number) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
if (localPlayerRef.current) {
localPlayerRef.current.currentTime(second)
if (videoRef.current) {
videoRef.current.currentTime = second
}
},
getCurrentTime: () => {
return (localPlayerRef.current ? localPlayerRef.current.currentTime() : 0) ?? 0
return (videoRef.current ? videoRef.current.currentTime : 0) ?? 0
},
getDuration: () => {
return (localPlayerRef.current ? localPlayerRef.current.duration() : 0) ?? 0
return (videoRef.current ? videoRef.current.duration : 0) ?? 0
},
}))
const isOverControlsRef = useRef(false)
const [videoHtmlEl, setVideoHtmlEl] = useState<HTMLVideoElement | undefined>(undefined)
const videoRef = useRef<HTMLVideoElement | null>(null)
const [zoom, setZoom] = useState(1)
const [pan, setPan] = useState({ x: 0, y: 0 })
const [hasManuallyZoomed, setHasManuallyZoomed] = useState(false)
const [containerSize, setContainerSize] = useState({ width: 0, height: 0 })
useEffect(() => {
const res = getInitialResolution()
const res = isHls ? null : getInitialResolution()
setState((prev) => ({
...prev,
isPlaying: false,
progress: 0,
currentTime: 0,
duration: data.media?.metadata?.duration || 0,
currentResolution: res?.resolution ?? '',
currentSrc: res?.url ?? '',
currentResolution: isHls ? 'Auto' : (res?.resolution ?? ''),
currentSrc: isHls ? (data.media?.hls?.url ?? '') : (res?.url ?? ''),
isCurrentHdr: res?.hdr ?? false,
}))
setActiveAutoResolution(undefined)
setHasManuallyZoomed(false)
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
setBuffered(0)
lastProcessedStartTimeRef.current = null
setHasStartedPlaying(false)
}, [data.id, Boolean(initialRes?.url)])
}, [data.id, Boolean(isHls ? data.media?.hls?.url : initialRes?.url)])
const vidW = data.media?.metadata?.originalWidth || 1920
const vidH = data.media?.metadata?.originalHeight || 1080
@@ -252,6 +256,8 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const [buffered, setBuffered] = useState(0)
const [isControlsVisible, setIsControlsVisible] = useState(true)
const [isPlayerReady, setIsPlayerReady] = useState(false)
const [isLoading, setIsLoading] = useState(true)
const waitingTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const lastProcessedStartTimeRef = useRef<number | null>(null)
// Frame-accurate hook and derived state
@@ -266,6 +272,10 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const handleSeekToFrame = useCallback(
(frame: number) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(frame)
},
@@ -281,6 +291,9 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
currentTime,
duration,
progress,
activeAutoResolution,
isHlsManualSupported,
isLoading,
}
// Trigger time update event reactively
@@ -290,146 +303,258 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
}
}, [currentFrame, frameRate, onTimeUpdate])
// Initialize Video.js
// Initialize player (Hls.js or native HTML5 video)
const targetSrc = isHls ? (data.media?.hls?.url ?? '') : (getInitialResolution()?.url ?? '')
useEffect(() => {
if (!videoContainerRef.current) return
const video = videoRef.current
if (!video) return
const initialRes = getInitialResolution()
const targetSrc = initialRes?.url ?? ''
if (!targetSrc) return
// Clean up previous player if exists
if (playerRef.current && !playerRef.current.isDisposed()) {
playerRef.current.dispose()
playerRef.current = null
videoRef.current = null
// Clean up previous HLS instance if exists
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
// 1. Create the video element manually
// This avoids conflicts with React Strict Mode where the DOM might not match React's virtual DOM expectations after Video.js modifies it.
const videoElement = document.createElement('video-js')
videoElement.classList.add('vjs-big-play-centered', '!h-full', '!w-full')
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
setActiveAutoResolution(undefined)
// Keep it in DOM, enable visible rendering
videoElement.style.pointerEvents = 'none'
const startAutoPlay = () => {
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch(() => {
video.muted = true
setState((prev) => ({ ...prev, isMuted: true }))
video.play()?.catch(() => {})
})
}
}
// Append to our container
videoContainerRef.current.appendChild(videoElement)
let hls: Hls | null = null
// 2. Initialize the player
const player = (playerRef.current = videojs(videoElement, {
controls: false,
autoplay: false,
preload: 'auto',
playsinline: true,
sources: [
{
src: targetSrc,
type: 'video/mp4',
},
],
}))
if (isHls) {
if (Hls.isSupported()) {
setIsHlsManualSupported(true)
hls = new Hls({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
})
hlsRef.current = hls
hls.attachMedia(video)
hls.loadSource(targetSrc)
if (autoPlay) {
const startAutoPlay = () => {
if (player.isDisposed()) return
const playPromise = player.play()
if (playPromise !== undefined) {
playPromise.catch(() => {
if (player.isDisposed()) return
player.muted(true)
setState((prev) => ({ ...prev, isMuted: true }))
player.play()?.catch(() => {})
})
hls.on(Hls.Events.MANIFEST_PARSED, () => {
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
video.addEventListener('canplay', startAutoPlay, { once: true })
}
}
})
hls.on(Hls.Events.LEVEL_SWITCHED, (_event, eventData) => {
const level = hls?.levels[eventData.level]
if (level) {
const matched = resolutions.find(
(r) => r.width === level.width && r.height === level.height,
)
if (matched?.resolution) {
setActiveAutoResolution(matched.resolution)
} else {
const shortSide =
level.width && level.height
? Math.min(level.width, level.height)
: level.height || level.width
setActiveAutoResolution(shortSide ? `${shortSide}p` : undefined)
}
}
})
hls.on(Hls.Events.ERROR, (_event, eventData) => {
if (eventData.fatal) {
switch (eventData.type) {
case Hls.ErrorTypes.NETWORK_ERROR:
hls?.startLoad()
break
case Hls.ErrorTypes.MEDIA_ERROR:
hls?.recoverMediaError()
break
default:
hls?.destroy()
break
}
}
})
} else if (video.canPlayType('application/vnd.apple.mpegurl')) {
// Native Safari HLS fallback
setIsHlsManualSupported(false)
video.src = targetSrc
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
video.addEventListener('canplay', startAutoPlay, { once: true })
}
}
}
player.ready(() => {
if (player.isDisposed()) return
if (player.readyState() >= 2) {
} else {
// Native MP4
video.src = targetSrc
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
player.one('canplay', startAutoPlay)
video.addEventListener('canplay', startAutoPlay, { once: true })
}
})
}
}
// Extract video element for Konva
const htmlVid = videoElement.querySelector('video')
if (htmlVid) {
setVideoHtmlEl(htmlVid)
videoRef.current = htmlVid
} else {
// Fallback: try player.tech().el()
// Wait for ready?
player.ready(() => {
const techEl = player.tech({ iWillNotUseThisInPlugins: true })?.el() as HTMLVideoElement
if (techEl) {
setVideoHtmlEl(techEl)
videoRef.current = techEl
}
})
const clearWaitingTimeout = () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
}
player.on('play', () => {
const handlePlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setState((p) => ({ ...p, isPlaying: true }))
setHasStartedPlaying(true)
onPlay?.()
})
player.on('pause', () => {
}
const handlePause = () => {
setState((p) => ({ ...p, isPlaying: false }))
onPause?.()
})
player.on('ended', () => setState((p) => ({ ...p, isPlaying: false })))
player.on('loadedmetadata', () => {
}
const handleEnded = () => {
clearWaitingTimeout()
setIsLoading(false)
setState((p) => ({ ...p, isPlaying: false }))
}
const handleLoadedData = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
})
player.on('loadstart', () => {
}
const handleCanPlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
}
const handlePlaying = () => {
clearWaitingTimeout()
setIsLoading(false)
}
const handleWaiting = () => {
clearWaitingTimeout()
waitingTimeoutRef.current = setTimeout(() => {
setIsLoading(true)
}, 200)
}
const handleLoadStart = () => {
clearWaitingTimeout()
setIsLoading(true)
setIsPlayerReady(false)
})
player.on('timeupdate', () => {
const playerDuration = player.duration() || data.media?.metadata?.duration || 0
const bufferedEnd = player.bufferedEnd()
if (playerDuration > 0) {
setBuffered((bufferedEnd / playerDuration) * 100)
}
})
player.on('volumechange', () => {
const vol = player.volume()
const muted = player.muted()
setState((prev) => ({
...prev,
volume: vol || 0,
isMuted: muted || false,
}))
})
player.on('ratechange', () => {
setState((prev) => ({
...prev,
playbackRate: player.playbackRate() || 1,
}))
})
player.on('error', () => {
console.error('VideoJS Error:', player.error())
})
// Set initial state
player.volume(state.volume)
// 4. Cleanup
return () => {
if (player && !player.isDisposed()) {
player.dispose()
playerRef.current = null
videoRef.current = null
}
const handleProgress = () => {
const vidDuration = video.duration || data.media?.metadata?.duration || 0
if (vidDuration > 0 && video.buffered.length > 0) {
let bufferedEnd = 0
for (let i = 0; i < video.buffered.length; i++) {
if (
video.buffered.start(i) <= video.currentTime &&
video.currentTime <= video.buffered.end(i)
) {
bufferedEnd = video.buffered.end(i)
break
}
}
if (bufferedEnd === 0) {
bufferedEnd = video.buffered.end(video.buffered.length - 1)
}
setBuffered((bufferedEnd / vidDuration) * 100)
}
}
}, [data.id, Boolean(initialRes?.url), autoPlay])
const handleVolumeChange = () => {
setState((prev) => ({
...prev,
volume: video.volume,
isMuted: video.muted,
}))
}
const handleRateChange = () => {
setState((prev) => ({
...prev,
playbackRate: video.playbackRate,
}))
}
const handleError = () => {
clearWaitingTimeout()
setIsLoading(false)
console.error('Video Error:', video.error)
}
if (video.readyState >= 2) {
setIsLoading(false)
setIsPlayerReady(true)
}
video.addEventListener('play', handlePlay)
video.addEventListener('pause', handlePause)
video.addEventListener('ended', handleEnded)
video.addEventListener('loadeddata', handleLoadedData)
video.addEventListener('canplay', handleCanPlay)
video.addEventListener('playing', handlePlaying)
video.addEventListener('waiting', handleWaiting)
video.addEventListener('loadstart', handleLoadStart)
video.addEventListener('timeupdate', handleProgress)
video.addEventListener('progress', handleProgress)
video.addEventListener('volumechange', handleVolumeChange)
video.addEventListener('ratechange', handleRateChange)
video.addEventListener('error', handleError)
// Set initial state
video.volume = state.volume
// Cleanup
return () => {
clearWaitingTimeout()
video.removeEventListener('play', handlePlay)
video.removeEventListener('pause', handlePause)
video.removeEventListener('ended', handleEnded)
video.removeEventListener('loadeddata', handleLoadedData)
video.removeEventListener('canplay', handleCanPlay)
video.removeEventListener('playing', handlePlaying)
video.removeEventListener('waiting', handleWaiting)
video.removeEventListener('loadstart', handleLoadStart)
video.removeEventListener('timeupdate', handleProgress)
video.removeEventListener('progress', handleProgress)
video.removeEventListener('volumechange', handleVolumeChange)
video.removeEventListener('ratechange', handleRateChange)
video.removeEventListener('error', handleError)
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
}
}, [data.id, targetSrc, isHls, autoPlay])
// Handle changes to startTime (e.g., clicking different chunks in search results)
useEffect(() => {
@@ -503,14 +628,20 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
// -- Event Handlers --
const togglePlay = useCallback(() => {
if (isLoading) return
// Disable click-to-play if drawing
if (useAnnotationStore.getState().isDrawing) return
const player = playerRef.current
if (!player) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (player.paused() || player.ended()) {
const playPromise = player.play()
const video = videoRef.current
if (!video) return
if (video.paused || video.ended) {
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch((error) => {
// Ignore AbortError which happens when pausing rapidly after playing
@@ -519,9 +650,9 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
})
}
} else {
player.pause()
video.pause()
}
}, [])
}, [isLoading])
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
@@ -565,10 +696,10 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
}, [togglePlay])
const toggleLoop = () => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const newLoop = !state.isLooping
player.loop(newLoop)
video.loop = newLoop
setState((prev) => ({ ...prev, isLooping: newLoop }))
}
@@ -580,48 +711,121 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
isMuted: newVolume === 0, // If dragging to 0, consider it muted
}))
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
// Set volume first so if we unmute, it is at the correct level
player.volume(newVolume)
video.volume = newVolume
// Manage mute state based on volume
if (newVolume > 0 && player.muted()) {
player.muted(false)
} else if (newVolume === 0 && !player.muted()) {
player.muted(true)
if (newVolume > 0 && video.muted) {
video.muted = false
} else if (newVolume === 0 && !video.muted) {
video.muted = true
}
}
const toggleMute = () => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const isMuted = player.muted()
const isMuted = video.muted
if (isMuted) {
player.muted(false)
video.muted = false
// If volume was 0 (e.g. user dragged to 0), restore to default 0.5 so they hear something
if (player.volume() === 0) {
player.volume(0.5)
if (video.volume === 0) {
video.volume = 0.5
}
} else {
player.muted(true)
video.muted = true
}
}
const changePlaybackRate = (rate: number) => {
const player = playerRef.current
if (!player) return
player.playbackRate(rate)
const video = videoRef.current
if (!video) return
video.playbackRate = rate
setState((prev) => ({ ...prev, playbackRate: rate }))
}
const changeResolution = (res: DisplayTranscode) => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const wasPlaying = !player.paused()
const currentT = player.currentTime()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (isHls && hlsRef.current) {
const hls = hlsRef.current
const wasPaused = video.paused
const targetFrame = currentFrame
if (res.resolution === 'Auto') {
if (state.currentResolution === 'Auto' && hls.currentLevel === -1) {
return
}
hls.currentLevel = -1
setState((prev) => ({
...prev,
currentResolution: 'Auto',
}))
} else {
const targetIndex = hls.levels.findIndex(
(lvl) =>
lvl.height === res.height ||
Math.max(lvl.width, lvl.height) === Math.max(res.width, res.height),
)
if (targetIndex !== -1) {
if (targetIndex === hls.currentLevel && state.currentResolution === res.resolution) {
return
}
hls.currentLevel = targetIndex
setState((prev) => ({
...prev,
currentResolution: res.resolution,
isCurrentHdr: res.hdr ?? false,
}))
}
}
// Guarantee frame accuracy if paused: re-nudge to safe center time once fragment is decoded
if (wasPaused) {
let timer: ReturnType<typeof setTimeout> | null = null
const cleanup = () => {
if (timer) {
clearTimeout(timer)
timer = null
}
video.removeEventListener('seeked', onSeeked)
hls.off?.(Hls.Events.ERROR, cleanup)
if (pendingResolutionCorrectionRef.current === cleanup) {
pendingResolutionCorrectionRef.current = null
}
}
const onSeeked = () => {
cleanup()
const safeCenterTime = calculateFrameCenterTime(targetFrame, frameRate)
const frameDuration = 1 / frameRate
if (Math.abs(video.currentTime - safeCenterTime) > frameDuration / 4) {
video.currentTime = safeCenterTime
}
}
// Safety timeout (1.5s): if no seeked event fires from rendition switch, cleanly detach
timer = setTimeout(cleanup, 1500)
pendingResolutionCorrectionRef.current = cleanup
video.addEventListener('seeked', onSeeked)
hls.once?.(Hls.Events.ERROR, cleanup)
}
return
}
const wasPlaying = !video.paused
const currentT = video.currentTime
setState((prev) => ({
...prev,
@@ -630,26 +834,27 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
isCurrentHdr: res.hdr ?? false,
}))
player.src({ type: 'video/mp4', src: res.url })
video.src = res.url || ''
player.one('loadedmetadata', () => {
player.currentTime(currentT)
const onLoadedMetadata = () => {
video.removeEventListener('loadedmetadata', onLoadedMetadata)
video.currentTime = currentT
if (wasPlaying) {
const playPromise = player.play()
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch((error) => {
if (error.name === 'AbortError') return
console.error('Play after seek failed:', error)
if (error.name !== 'AbortError') console.error('Play after seek failed:', error)
})
}
}
player.playbackRate(state.playbackRate)
})
video.playbackRate = state.playbackRate
}
video.addEventListener('loadedmetadata', onLoadedMetadata)
}
const toggleFullScreen = () => {
const rootEl = rootRef.current
const videoEl = videoHtmlEl || videoRef.current
const videoEl = videoRef.current
// 1. Standard W3C Fullscreen API (Desktop Chrome/Firefox/Safari, Android, iPadOS)
if (rootEl && typeof rootEl.requestFullscreen === 'function') {
@@ -727,7 +932,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
document.addEventListener('fullscreenchange', onFsChange)
// iOS WebKit fullscreen events on <video>
const videoEl = videoHtmlEl || videoRef.current
const videoEl = videoRef.current
const onWebkitBeginFs = () => {
setState((prev) => ({ ...prev, isFullScreen: true }))
}
@@ -746,7 +951,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
videoEl.removeEventListener('webkitendfullscreen', onWebkitEndFs)
}
}
}, [videoHtmlEl])
}, [])
// Center pan is owned by `pan` state (set by auto-fit / gestures).
const scale = zoom
@@ -754,9 +959,34 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const isMobile = useIsMobile()
if (!hasMedia) {
if (data.status === 'failed') {
const errorMessage = data.media?.error
return (
<div
data-testid="video-viewer-failed-state"
className="flex h-full w-full flex-col items-center justify-center gap-3 p-6 text-center"
>
<AlertCircle className="h-12 w-12 text-destructive/80" />
<div className="space-y-1 max-w-md">
<h3 className="text-base font-semibold text-foreground">
{m.media_processing_failed()}
</h3>
<p className="text-sm text-muted-foreground">{m.media_processing_failed_desc()}</p>
{errorMessage && (
<div
title={errorMessage}
className="mt-3 rounded-lg border border-destructive/20 bg-destructive/10 p-3 text-xs text-destructive text-left line-clamp-3 break-words"
>
{errorMessage}
</div>
)}
</div>
</div>
)
}
return (
<div className="flex h-full w-full items-center justify-center">
<p className="text-muted-foreground">Media is not available.</p>
<p className="text-muted-foreground">{m.media_not_available()}</p>
</div>
)
}
@@ -783,7 +1013,6 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
'flex-1 bg-black cursor-pointer relative flex items-center justify-center overflow-hidden min-h-0 touch-none',
)}
onClick={togglePlay}
data-vjs-player
data-testid="video-area"
>
{/* Native Video Layer (Hardware-accelerated, full HDR EDR) */}
@@ -799,16 +1028,21 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
transformOrigin: '0 0',
}}
>
<div
ref={videoContainerRef}
className="w-full h-full [&_.video-js]:!w-full [&_.video-js]:!h-full [&_video]:!w-full [&_video]:!h-full [&_video]:!block [&_video]:!object-contain"
<video
ref={videoRef}
className="w-full h-full block object-contain pointer-events-none"
playsInline
preload="auto"
poster={data.preview?.thumbnailUrl || undefined}
/>
</div>
)}
{isAudio && (
<div
ref={videoContainerRef}
<video
ref={videoRef}
className="absolute inset-0 pointer-events-none opacity-0"
playsInline
preload="auto"
/>
)}
@@ -847,9 +1081,24 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
)
)}
{/* Loading Spinner Overlay */}
{isLoading && (
<div
className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200"
data-testid="video-loading-spinner"
>
<div className="w-16 h-16 bg-white/10 backdrop-blur-sm rounded-full flex items-center justify-center border-2 border-white/30">
<Loader2 className="w-8 h-8 text-white animate-spin" />
</div>
</div>
)}
{/* Initial Big Play Button Overlay (shown only before playback starts) */}
{!hasStartedPlaying && !isDrawing && (
<div className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200">
{!isLoading && !hasStartedPlaying && !isDrawing && (
<div
className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200"
data-testid="video-play-overlay"
>
<div className="w-20 h-20 bg-white/10 backdrop-blur-sm rounded-full flex items-center justify-center border-2 border-white/30">
<Play className="w-10 h-10 text-white ml-1 fill-white" />
</div>
@@ -883,6 +1132,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
onMouseEnter={handleControlsMouseEnter}
onMouseLeave={handleControlsMouseLeave}
allowDownload={allowDownload}
isLoading={isLoading}
/>
) : (
<VideoControlBar
@@ -909,6 +1159,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
onMouseEnter={handleControlsMouseEnter}
onMouseLeave={handleControlsMouseLeave}
allowDownload={allowDownload}
isLoading={isLoading}
/>
)}
</div>
+13 -6
View File
@@ -19,7 +19,7 @@ Shumai's video player is a decoupled, frame-locked system that coordinates three
▼ │
┌────────────────────────────┴──┐
│ Native HTML5 Video Player │
│ (VideoJS Engine Wrapper) │
│ (Hls.js / Native MSE) │
└───────────────────────────────┘
▲ ▲
│ │
@@ -32,7 +32,7 @@ Shumai's video player is a decoupled, frame-locked system that coordinates three
└──────────────┘ └──────────────┘
```
1. **Native HTML5 Video Player (VideoJS Engine Wrapper):** Manages the underlying browser media stream, buffering, audio clock, and presentation.
1. **Native HTML5 Video Player (Hls.js / Native MSE):** Manages the underlying browser media stream, buffering, audio clock, and presentation. Direct `<video>` element with zero library wrappers for MP4, and lightweight `hls.js` for adaptive HLS streams.
2. **Seekbar (Scrubber & Progress Bar):** Visualizes playback progress and maps horizontal screen clicks/drags to playhead percentages.
3. **Comment Area (Sidebar):** Renders comment list, displays timestamp labels, captures drawings, and triggers seeks to target frames.
@@ -204,14 +204,21 @@ When playing, the playhead loop coordinates two mechanisms in parallel to preven
---
### G. Switch Resolution
#### 1. HLS Streams (`hls.js`)
* **Immediate Rendition Switch (`currentLevel`)**: When the user selects a specific rendition (e.g. `1080p`), the player assigns `hls.currentLevel = targetIndex` rather than `nextLevel`. This causes `hls.js` to immediately flush the MSE buffer around `currentTime` and fetch fragments for the target rendition (matching Frame.io's responsive behavior).
* **Auto Mode**: When selecting `Auto`, `hls.currentLevel = -1` is set to re-enable adaptive bitrate (ABR). The player listens to `Hls.Events.LEVEL_SWITCHED` to update the control bar label dynamically (e.g., `Auto (1080p)`).
* **Paused Frame-Center Re-Nudge**: If the user switches quality while paused on a frame (e.g. Frame 120), the player listens for the subsequent `seeked` event once the new fragment renders and re-nudges `video.currentTime` to `calculateFrameCenterTime(currentFrame, frameRate)`. This guarantees zero frame drift across quality switches.
#### 2. Normal MP4 Streams (Native Video)
1. User selects a different resolution (e.g. `720p` or `Original`).
2. The player intercepts the click:
* Captures `wasPlaying = !player.paused()` and the current playhead time: `currentT = player.currentTime()`.
* Updates the source URL: `player.src({ type: 'video/mp4', src: res.url })`.
* Captures `wasPlaying = !video.paused` and the current playhead time: `currentT = video.currentTime`.
* Updates the source URL: `video.src = res.url`.
3. The player registers a one-shot `loadedmetadata` event listener on the new source:
* Restores playhead time: `player.currentTime(currentT)`.
* Restores playhead time: `video.currentTime = currentT`.
* Restores playback state (`play()` if `wasPlaying` was true) and playback rate.
4. The seek triggered by `player.currentTime(currentT)` fires a `seeked` event, which automatically calls `handleExternalSeeked`, recalculating the frame index and nudging the playhead to the safe frame center on the new media stream.
4. The seek triggered by `video.currentTime = currentT` fires a `seeked` event, which automatically calls `handleExternalSeeked`, recalculating the frame index and nudging the playhead to the safe frame center on the new media stream.
---
-2
View File
@@ -5,8 +5,6 @@
@source "./node_modules/@ferrucc-io/emoji-picker";
@source "../../node_modules/@ferrucc-io/emoji-picker";
@import 'video.js/dist/video-js.css' layer(base);
@custom-variant dark (&:is(.dark *));
:root {
+46 -1
View File
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest'
import { isImageFileName, getBestTranscode } from './media'
import { isImageFileName, getBestTranscode, getVideoResolutionLabel } from './media'
describe('isImageFileName', () => {
it('identifies web-supported image formats as images', () => {
@@ -49,3 +49,48 @@ describe('getBestTranscode', () => {
expect(getBestTranscode(transcodes, 1500)?.width).toBe(1200)
})
})
describe('getVideoResolutionLabel', () => {
it('preserves explicit resolution when present', () => {
expect(getVideoResolutionLabel({ width: 854, height: 480, resolution: '480p' })).toBe('480p')
expect(getVideoResolutionLabel({ width: 480, height: 854, resolution: '480p' })).toBe('480p')
expect(getVideoResolutionLabel({ width: 1920, height: 1080, resolution: '1080p' })).toBe(
'1080p',
)
})
it('derives resolution from long side for portrait legacy transcodes without explicit resolution', () => {
// 4K portrait
expect(getVideoResolutionLabel({ width: 2160, height: 3840 })).toBe('2160p')
// 2K / 1440p portrait
expect(getVideoResolutionLabel({ width: 1440, height: 2560 })).toBe('1440p')
// 1080p portrait
expect(getVideoResolutionLabel({ width: 1080, height: 1920 })).toBe('1080p')
// 720p portrait
expect(getVideoResolutionLabel({ width: 720, height: 1280 })).toBe('720p')
// 480p portrait
expect(getVideoResolutionLabel({ width: 480, height: 854 })).toBe('480p')
// 360p portrait
expect(getVideoResolutionLabel({ width: 360, height: 640 })).toBe('360p')
// 180p portrait
expect(getVideoResolutionLabel({ width: 180, height: 320 })).toBe('180p')
})
it('derives resolution from long side for landscape legacy transcodes without explicit resolution', () => {
expect(getVideoResolutionLabel({ width: 3840, height: 2160 })).toBe('2160p')
expect(getVideoResolutionLabel({ width: 2560, height: 1440 })).toBe('1440p')
expect(getVideoResolutionLabel({ width: 1920, height: 1080 })).toBe('1080p')
expect(getVideoResolutionLabel({ width: 1280, height: 720 })).toBe('720p')
expect(getVideoResolutionLabel({ width: 854, height: 480 })).toBe('480p')
expect(getVideoResolutionLabel({ width: 640, height: 360 })).toBe('360p')
expect(getVideoResolutionLabel({ width: 320, height: 180 })).toBe('180p')
})
it('falls back to short side or single dimension for non-standard sizes', () => {
expect(getVideoResolutionLabel({ width: 200, height: 100 })).toBe('100p')
expect(getVideoResolutionLabel({ height: 720 })).toBe('720p')
expect(getVideoResolutionLabel({ width: 500 })).toBe('500p')
expect(getVideoResolutionLabel(null)).toBe('')
expect(getVideoResolutionLabel(undefined)).toBe('')
})
})
+40
View File
@@ -40,3 +40,43 @@ export function getBestTranscode(
// If no candidate >= screenWidth, simply return the largest available (first in original sorted)
return sorted[0]
}
export function getVideoResolutionLabel(
transcode?: {
width?: number
height?: number
resolution?: string
} | null,
): string {
if (!transcode) return ''
if (transcode.resolution) {
return transcode.resolution
}
const width = transcode.width ?? 0
const height = transcode.height ?? 0
if (width > 0 && height > 0) {
const longSide = Math.max(width, height)
if (longSide >= 3840) return '2160p'
if (longSide >= 2560) return '1440p'
if (longSide >= 1920) return '1080p'
if (longSide >= 1280) return '720p'
if (longSide >= 854) return '480p'
if (longSide >= 640) return '360p'
if (longSide >= 320) return '180p'
const shortSide = Math.min(width, height)
return `${shortSide}p`
}
if (height > 0) {
return `${height}p`
}
if (width > 0) {
return `${width}p`
}
return ''
}
+28 -4
View File
@@ -13,6 +13,7 @@
"personal_settings": "Personal Settings",
"team_settings": "Team Settings",
"ai_settings": "AI Settings",
"team_ai_settings": "Team AI Settings",
"welcome_back": "Welcome Back",
"create_account": "Create Account",
"select_a_team": "Select a Team",
@@ -39,7 +40,8 @@
"no_account_prompt": "Don't have an account? ",
"sign_up": "Sign up",
"demo_access": "Demo Access:",
"demo_access_instructions": "Use \"foo@bar.com\" as the email and \"foo\" as the password to login (this demo account is read-only).",
"demo_access_instructions": "This demo account is a read-only \"Reviewer\" role user.",
"login_as_demo_user": "Login as demo user",
"signup_subtitle": "Start building and organizing your space.",
"signup_failed": "Signup failed",
@@ -246,14 +248,25 @@
"failed_update_profile": "Failed to update profile",
"failed_load_settings": "Failed to load settings.",
"media_processing": "Media Processing",
"media_processing_description": "Manage your team's media transcoding configurations.",
"media_processing": "Video Transcoding",
"media_processing_description": "Manage your team's video transcoding configurations.",
"video_transcoding": "Video Transcoding",
"video_transcoding_description": "Manage your team's video transcoding configurations.",
"transcode_policy": "Transcode Policy",
"transcode_policy_description": "When HLS is disabled, both playback and download use MP4. When HLS is enabled, playback uses HLS streaming while download uses MP4. Note that enabling HLS may significantly increase storage usage.",
"transcode_settings": "Transcode Settings",
"transcode_description": "Manage your team's media transcoding configurations.",
"transcode_description": "Manage your team's video transcoding configurations.",
"transcode_strategy_note": "Strategy: We select the best resolution from your list that supports the input quality. Content is never upscaled.",
"hardware_acceleration_section_description": "Configure hardware-accelerated video encoding to offload workloads to GPUs, or use CPU software encoding.",
"mp4_transcoding": "MP4 Transcoding",
"mp4_transcoding_description": "Generate web-compatible MP4 files for file downloads and standard video playback.",
"video_strategy": "Video Strategy",
"best_match": "Best match",
"best_match_description": "Generates a single optimal resolution matching the source quality.",
"multi_resolutions": "Multi resolutions",
"multi_resolutions_description": "Generates selected resolutions up to the source quality.",
"video_resolution_ladders": "Resolution Ladders",
"video_resolution_ladders_description": "Select the target resolutions to generate for MP4 transcoding. Resolutions are only generated up to the source video resolution.",
"all_resolutions": "All resolutions",
"all_resolutions_description": "Generates all supported resolutions up to the source quality.",
"hardware_acceleration": "Hardware Acceleration",
@@ -263,6 +276,11 @@
"hardware_acceleration_auto_description": "Automatically probes and uses GPU/Hardware acceleration (NVENC, QSV, VA-API, AMF, VideoToolbox). If hardware acceleration is unsupported on the host, video transcoding may fail.",
"ffmpeg_threads": "FFmpeg Threads",
"ffmpeg_threads_description": "Maximum number of CPU threads used by FFmpeg during transcoding. Set to 0 for Auto.",
"hls_streaming": "HLS Streaming",
"hls_streaming_description": "Enable HTTP Live Streaming (HLS) with adaptive bitrate ladders for smooth video playback.",
"hls_resolution_ladders": "Resolution Ladders",
"hls_resolution_ladders_description": "Select the target resolutions to generate for HLS streaming. Ladders are only generated up to the source video resolution.",
"quality_auto": "Auto",
"threads_auto": "Auto",
"threads_count": "{count} threads",
"hdr": "HDR",
@@ -765,6 +783,12 @@
"failed_to_upload": "Failed to upload",
"by": "by",
"failed_to_upload_parenthesized": "(Failed to upload)",
"processing_failed": "Processing failed",
"processing_failed_parenthesized": "(Processing failed)",
"transcoding_failed": "Transcoding Failed",
"status_failed": "Failed",
"media_processing_failed": "Media processing failed",
"media_processing_failed_desc": "An error occurred while processing this media file. You can still download the original file.",
"n_items_count": "{count} Item(s)",
"name_column": "Name",
"size_column": "Size",
+28 -4
View File
@@ -13,6 +13,7 @@
"personal_settings": "个人设置",
"team_settings": "团队设置",
"ai_settings": "AI 设置",
"team_ai_settings": "团队 AI 设置",
"welcome_back": "欢迎回来",
"create_account": "创建账户",
"select_a_team": "选择团队",
@@ -39,7 +40,8 @@
"no_account_prompt": "还没有账户?",
"sign_up": "注册",
"demo_access": "演示访问:",
"demo_access_instructions": "使用 \"foo@bar.com\" 作为邮箱,\"foo\" 作为密码登录(该演示账户为只读)。",
"demo_access_instructions": "该演示账户为只读 \"Reviewer\" 角色用户。",
"login_as_demo_user": "以演示用户登录",
"signup_subtitle": "开始构建和整理您的空间。",
"signup_failed": "注册失败",
@@ -246,14 +248,25 @@
"failed_update_profile": "更新个人资料失败",
"failed_load_settings": "加载设置失败。",
"media_processing": "媒体处理",
"media_processing_description": "管理团队的媒体转码配置。",
"media_processing": "视频转码",
"media_processing_description": "管理团队的视频转码配置。",
"video_transcoding": "视频转码",
"video_transcoding_description": "管理团队的视频转码配置。",
"transcode_policy": "转码策略",
"transcode_policy_description": "未启用 HLS 时,播放和下载均使用 MP4。启用 HLS 后,播放将使用 HLS 自适应流,而下载使用 MP4。请注意,启用 HLS 可能会显著增加存储空间占用。",
"transcode_settings": "转码设置",
"transcode_description": "管理团队的媒体转码配置。",
"transcode_description": "管理团队的视频转码配置。",
"transcode_strategy_note": "策略:我们会从您的列表中选择支持输入质量的最佳分辨率。内容不会被放大。",
"hardware_acceleration_section_description": "配置硬件加速视频编码以将工作负载分流至 GPU,或使用 CPU 软件编码。",
"mp4_transcoding": "MP4 转码",
"mp4_transcoding_description": "生成适用于文件下载和标准视频播放的 Web 兼容 MP4 文件。",
"video_strategy": "视频策略",
"best_match": "最佳匹配",
"best_match_description": "生成与源质量匹配的单一最优分辨率。",
"multi_resolutions": "多分辨率",
"multi_resolutions_description": "生成选中的不超过源质量的分辨率。",
"video_resolution_ladders": "分辨率阶梯",
"video_resolution_ladders_description": "选择 MP4 转码的目标分辨率。仅生成不超过原始视频分辨率的分辨率阶梯。",
"all_resolutions": "所有分辨率",
"all_resolutions_description": "生成不超过源质量的所有支持分辨率。",
"hardware_acceleration": "硬件加速",
@@ -263,6 +276,11 @@
"hardware_acceleration_auto_description": "自动检测并使用 GPU/硬件加速(NVENC、QSV、VA-API、AMF、VideoToolbox)。若宿主硬件不支持加速,视频转码可能会失败。",
"ffmpeg_threads": "FFmpeg 线程数",
"ffmpeg_threads_description": "FFmpeg 转码时使用的最大 CPU 线程数。设置为 0 表示自动。",
"hls_streaming": "HLS 流媒体",
"hls_streaming_description": "启用自适应码率阶梯的 HTTP Live Streaming (HLS),带来流畅的视频流播放体验。",
"hls_resolution_ladders": "分辨率阶梯",
"hls_resolution_ladders_description": "选择 HLS 流媒体生成的目标分辨率。仅会生成不超过原始视频分辨率的分辨率阶梯。",
"quality_auto": "自动",
"threads_auto": "自动",
"threads_count": "{count} 线程",
"hdr": "HDR",
@@ -765,6 +783,12 @@
"failed_to_upload": "上传失败",
"by": "由",
"failed_to_upload_parenthesized": "(上传失败)",
"processing_failed": "处理失败",
"processing_failed_parenthesized": "(处理失败)",
"transcoding_failed": "转码失败",
"status_failed": "失败",
"media_processing_failed": "媒体处理失败",
"media_processing_failed_desc": "处理此媒体文件时发生错误。您仍可以下载原始文件。",
"n_items_count": "{count} 个项目",
"name_column": "名称",
"size_column": "大小",
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/webui",
"version": "0.4.9",
"version": "0.5.0",
"private": true,
"type": "module",
"main": "./index.html",
@@ -63,6 +63,7 @@
"cmdk": "1.1.1",
"date-fns": "4.4.0",
"embla-carousel-react": "8.6.0",
"hls.js": "1.7.3",
"hono": "4.13.5",
"input-otp": "1.5.0",
"konva": "10.3.2",
@@ -86,7 +87,6 @@
"ulid": "3.0.2",
"use-image": "1.1.4",
"vaul": "1.1.2",
"video.js": "8.24.0",
"zod": "4.5.4",
"zustand": "5.0.15"
},
+68 -10
View File
@@ -9,7 +9,7 @@ import { useAuthStore } from '@/ui/stores/auth'
import { zodResolver } from '@hookform/resolvers/zod'
import { useQuery } from '@tanstack/react-query'
import { createFileRoute, redirect, useNavigate } from '@tanstack/react-router'
import { ArrowRight, Loader2, Lock, LogIn, Mail } from 'lucide-react'
import { ArrowRight, Loader2, Lock, LogIn, Mail, UserCheck } from 'lucide-react'
import { useEffect, useState } from 'react'
import { useForm } from 'react-hook-form'
import { z } from 'zod'
@@ -27,6 +27,7 @@ function LoginPage() {
const setUser = useAuthStore((state) => state.setUser)
const [error, setError] = useState<string | null>(null)
const [loading, setLoading] = useState(false)
const [demoLoading, setDemoLoading] = useState(false)
const { data: signupInfo, isLoading: isSignupInfoLoading } = useQuery({
queryKey: ['/signup-info'],
@@ -69,6 +70,30 @@ function LoginPage() {
}
}
const onDemoLogin = async () => {
setDemoLoading(true)
setError(null)
form.clearErrors()
form.setValue('email', 'foo@bar.com')
form.setValue('password', 'foo')
const { data: session, error: signInError } = await signIn.email({
email: 'foo@bar.com',
password: 'foo',
})
setDemoLoading(false)
if (signInError) {
setError(signInError.message || m.invalid_credentials())
return
}
if (session) {
setUser(session.user)
navigate({ to: '/' })
}
}
if (isSignupInfoLoading) {
return (
<div className="flex items-center justify-center min-h-screen bg-zinc-100/80 dark:bg-black">
@@ -89,14 +114,6 @@ function LoginPage() {
<p className="text-zinc-500 dark:text-zinc-400 text-sm mt-1">{m.login_subtitle()}</p>
</div>
{isDemoEnv && (
<div className="mb-6 p-4 bg-blue-500/10 border border-blue-500/20 text-blue-700 dark:text-blue-400 rounded-xl text-sm leading-relaxed">
<p>
<strong>{m.demo_access()}</strong> {m.demo_access_instructions()}
</p>
</div>
)}
{error && (
<div className="mb-6 p-3 bg-red-500/10 border border-red-500/20 text-red-600 dark:text-red-400 rounded-xl text-sm text-center font-medium">
{error}
@@ -155,7 +172,7 @@ function LoginPage() {
<Button
type="submit"
className="w-full h-11 bg-gradient-to-r from-rose-500 to-orange-500 hover:from-rose-600 hover:to-orange-600 text-white font-semibold rounded-xl shadow-md shadow-rose-500/10 hover:shadow-rose-500/20 active:scale-[0.98] transition-all flex items-center justify-center gap-2 border-0"
disabled={loading}
disabled={loading || demoLoading}
>
{loading ? (
<>
@@ -171,6 +188,47 @@ function LoginPage() {
</Button>
</form>
{isDemoEnv && (
<div className="mt-4 space-y-4">
<div className="relative">
<div className="absolute inset-0 flex items-center">
<span className="w-full border-t border-zinc-200/80 dark:border-zinc-800" />
</div>
<div className="relative flex justify-center text-xs uppercase">
<span className="bg-white dark:bg-zinc-900 px-2 text-zinc-400 dark:text-zinc-500 font-medium">
{m.or()}
</span>
</div>
</div>
<Button
type="button"
variant="outline"
onClick={onDemoLogin}
disabled={loading || demoLoading}
className="w-full h-11 border-zinc-200 dark:border-zinc-800 hover:bg-zinc-100 dark:hover:bg-zinc-800 font-semibold rounded-xl transition-all flex items-center justify-center gap-2 shadow-xs cursor-pointer"
>
{demoLoading ? (
<>
<Loader2 className="h-4 w-4 animate-spin" />
{m.logging_in()}
</>
) : (
<>
<UserCheck className="w-4 h-4 text-zinc-500 dark:text-zinc-400" />
{m.login_as_demo_user()}
</>
)}
</Button>
<div className="p-3 bg-blue-500/10 border border-blue-500/20 text-blue-700 dark:text-blue-400 rounded-xl text-xs leading-relaxed text-center">
<p>
<strong>{m.demo_access()}</strong> {m.demo_access_instructions()}
</p>
</div>
</div>
)}
<div className="mt-8 pt-6 border-t border-zinc-200/30 dark:border-zinc-800/30 text-center text-sm">
<span className="text-zinc-500 dark:text-zinc-400">{m.no_account_prompt()}</span>
<a
@@ -300,6 +300,7 @@ function FileViewPage() {
key: t.key,
width: t.width,
height: t.height,
resolution: t.resolution,
hdr: t.hdr,
})),
},
@@ -1,10 +1,16 @@
import { createLazyFileRoute } from '@tanstack/react-router'
import {
VideoTranscodeStrategy,
VideoResolutionLadder,
DEFAULT_VIDEO_RESOLUTIONS,
HardwareAcceleration,
UpdateTeamSettingsRequest,
TeamSettingsResponse,
HlsResolutionLadder,
DEFAULT_HLS_RESOLUTIONS,
} from '@shumai/dtos'
import { Switch } from '@/ui/components/ui/switch'
import { Checkbox } from '@/ui/components/ui/checkbox'
import { client } from '@/ui/api/client'
import { AgentsSettings } from '@/ui/components/settings/AgentsSettings'
import { ProvidersSettings } from '@/ui/components/settings/ProvidersSettings'
@@ -85,6 +91,118 @@ const VALID_SETTINGS_TABS: readonly SettingsTab[] = [
'developer',
]
const ALL_VIDEO_LADDERS: VideoResolutionLadder[] = ['480p', '720p', '1080p', '1440p', '2160p']
const ALL_HLS_LADDERS: HlsResolutionLadder[] = ['480p', '720p', '1080p', '1440p', '2160p']
function useCoalescedLadderSettings<TLadder extends VideoResolutionLadder | HlsResolutionLadder>({
teamId,
settingKey,
field,
serverResolutions,
defaultResolutions,
}: {
teamId: string
settingKey: 'transcode.videoResolutions' | 'transcode.hlsResolutions'
field: 'videoResolutions' | 'hlsResolutions'
serverResolutions: TLadder[] | undefined
defaultResolutions: TLadder[]
}) {
const queryClient = useQueryClient()
const pendingResolutionsRef = useRef<TLadder[] | null>(null)
const [optimisticResolutions, setOptimisticResolutions] = useState<TLadder[] | null>(null)
const queueRef = useRef<Promise<void>>(Promise.resolve())
const isUpdatingRef = useRef<boolean>(false)
useEffect(() => {
if (!isUpdatingRef.current) {
pendingResolutionsRef.current = null
setOptimisticResolutions(null)
}
}, [serverResolutions])
const currentResolutions: TLadder[] =
optimisticResolutions ?? serverResolutions ?? defaultResolutions
const handleToggle = (ladder: TLadder) => {
const current =
pendingResolutionsRef.current ??
optimisticResolutions ??
serverResolutions ??
defaultResolutions
// If currently checked and it's the last remaining option, do not allow unselecting
if (current.includes(ladder) && current.length === 1) {
return
}
const updated = current.includes(ladder)
? current.filter((l) => l !== ladder)
: [...current, ladder]
if (updated.length === 0) return
pendingResolutionsRef.current = updated
setOptimisticResolutions(updated)
queryClient.setQueryData(
['teams', teamId, 'settings'],
(old: TeamSettingsResponse | undefined) => {
if (!old) return old
return {
...old,
transcode: {
...old.transcode,
[field]: updated,
},
}
},
)
isUpdatingRef.current = true
queueRef.current = queueRef.current
.then(async () => {
const target = pendingResolutionsRef.current
if (!target) return
const res = await client.api.teams[':teamId'].settings.$patch({
param: { teamId },
json: {
key: settingKey,
value: target,
} as UpdateTeamSettingsRequest,
})
if (!res.ok) throw new Error('Failed to update settings')
await res.json()
if (pendingResolutionsRef.current === target) {
pendingResolutionsRef.current = null
isUpdatingRef.current = false
setOptimisticResolutions(null)
await queryClient.invalidateQueries({
queryKey: ['teams', teamId, 'settings'],
})
toast.success(m.settings_updated())
}
})
.catch((err) => {
console.error(err)
pendingResolutionsRef.current = null
isUpdatingRef.current = false
setOptimisticResolutions(null)
queryClient.invalidateQueries({
queryKey: ['teams', teamId, 'settings'],
})
toast.error(m.failed_update_settings())
})
}
return {
currentResolutions,
handleToggle,
}
}
function getTabFromHash(): SettingsTab {
if (typeof window === 'undefined') return 'general'
const hash = window.location.hash.replace(/^#/, '')
@@ -93,7 +211,7 @@ function getTabFromHash(): SettingsTab {
: 'general'
}
function TeamSettingsPage() {
export function TeamSettingsPage() {
const { teamId } = Route.useParams()
const queryClient = useQueryClient()
const [activeTab, setActiveTabState] = useState<SettingsTab>(getTabFromHash)
@@ -293,6 +411,10 @@ function TeamSettingsPage() {
queryClient.invalidateQueries({
queryKey: ['teams', teamId, 'settings'],
})
toast.success(m.settings_updated())
},
onError: () => {
toast.error(m.failed_update_settings())
},
})
@@ -306,6 +428,25 @@ function TeamSettingsPage() {
})
}
const { currentResolutions: currentVideoResolutions, handleToggle: handleVideoLadderToggle } =
useCoalescedLadderSettings({
teamId,
settingKey: 'transcode.videoResolutions',
field: 'videoResolutions',
serverResolutions: (settings as TeamSettingsResponse | undefined)?.transcode
?.videoResolutions,
defaultResolutions: DEFAULT_VIDEO_RESOLUTIONS,
})
const { currentResolutions: currentHlsResolutions, handleToggle: handleHlsLadderToggle } =
useCoalescedLadderSettings({
teamId,
settingKey: 'transcode.hlsResolutions',
field: 'hlsResolutions',
serverResolutions: (settings as TeamSettingsResponse | undefined)?.transcode?.hlsResolutions,
defaultResolutions: DEFAULT_HLS_RESOLUTIONS,
})
const handleHardwareAccelerationChange = (value: HardwareAcceleration) => {
updateSettings({
teamId,
@@ -333,6 +474,16 @@ function TeamSettingsPage() {
})
}
const handleHlsEnabledChange = (enabled: boolean) => {
updateSettings({
teamId,
data: {
key: 'transcode.hlsEnabled',
value: enabled,
},
})
}
if (isSettingsLoading || isMeLoading) {
return (
<div className="flex h-screen items-center justify-center">
@@ -350,8 +501,12 @@ function TeamSettingsPage() {
(settings as any)?.transcode?.videoStrategy || VideoTranscodeStrategy.best_match
if (currentVideoStrategy === 'single' || currentVideoStrategy === 'disable') {
currentVideoStrategy = VideoTranscodeStrategy.best_match
} else if (currentVideoStrategy === 'full') {
currentVideoStrategy = VideoTranscodeStrategy.all
} else if (
currentVideoStrategy === 'full' ||
currentVideoStrategy === 'all' ||
currentVideoStrategy === 'multi'
) {
currentVideoStrategy = VideoTranscodeStrategy.multi
}
const currentHardwareAcceleration =
@@ -359,6 +514,8 @@ function TeamSettingsPage() {
(settings as any)?.transcode?.hardwareAcceleration || HardwareAcceleration.off
const currentThreads = localThreads ?? serverThreads
const currentHlsEnabled =
(settings as TeamSettingsResponse | undefined)?.transcode?.hlsEnabled ?? false
return (
<div className="h-full bg-background font-sans selection:bg-primary/20 transition-colors duration-300">
@@ -436,7 +593,7 @@ function TeamSettingsPage() {
)}
>
<Film className="w-5 h-5" />
{m.media_processing()}
{m.video_transcoding()}
{activeTab === 'transcode' && (
<div className="ml-auto w-1.5 h-1.5 rounded-full bg-sidebar-primary" />
)}
@@ -496,7 +653,7 @@ function TeamSettingsPage() {
{me?.role === 'owner' && (
<>
<div className="mt-6 mb-2 px-4 text-xs font-semibold text-sidebar-foreground/60 uppercase tracking-wider pt-4 border-t border-sidebar-border">
{m.ai_settings()}
{m.team_ai_settings()}
</div>
<button
@@ -606,7 +763,7 @@ function TeamSettingsPage() {
<div>
<h2 className="text-2xl font-bold text-foreground">
{activeTab === 'general' && m.general_settings()}
{activeTab === 'transcode' && m.media_processing()}
{activeTab === 'transcode' && m.video_transcoding()}
{activeTab === 'quotas' && m.resource_quotas()}
{activeTab === 'appearance' && m.appearance_settings()}
{activeTab === 'email' && m.email_notifications()}
@@ -621,7 +778,7 @@ function TeamSettingsPage() {
</h2>
<p className="text-sm text-muted-foreground mt-1">
{activeTab === 'general' && m.general_settings_description()}
{activeTab === 'transcode' && m.transcode_description()}
{activeTab === 'transcode' && m.video_transcoding_description()}
{activeTab === 'quotas' && m.resource_quotas_description()}
{activeTab === 'appearance' && m.appearance_settings_description()}
{activeTab === 'email' && m.email_notifications_description()}
@@ -769,19 +926,22 @@ function TeamSettingsPage() {
)}
{activeTab === 'transcode' && (
<div className="h-full overflow-y-auto pr-1 pb-8">
<div className="h-full overflow-y-auto space-y-6 pr-1 pb-8">
{/* Transcode Policy Card */}
<Card>
<CardHeader>
<CardTitle>{m.transcode_settings()}</CardTitle>
<CardDescription>
<span className="block mb-2">{m.transcode_description()}</span>
<span className="block">{m.transcode_strategy_note()}</span>
</CardDescription>
<CardTitle>{m.transcode_policy()}</CardTitle>
<CardDescription>{m.transcode_policy_description()}</CardDescription>
</CardHeader>
<CardContent className="space-y-6">
{/* Video Strategy */}
<div className="space-y-3">
<h3 className="text-lg font-medium">{m.video_strategy()}</h3>
{/* MP4 Transcoding */}
<div className="space-y-4">
<div className="space-y-0.5">
<h3 className="text-lg font-medium">{m.mp4_transcoding()}</h3>
<p className="text-sm text-muted-foreground">
{m.mp4_transcoding_description()}
</p>
</div>
<div className="space-y-3">
<div
className={cn(
@@ -803,92 +963,186 @@ function TeamSettingsPage() {
<div
className={cn(
'cursor-pointer rounded-lg border p-4 transition-all hover:border-primary',
currentVideoStrategy === VideoTranscodeStrategy.all
currentVideoStrategy === VideoTranscodeStrategy.multi
? 'border-primary bg-primary/5'
: 'border-border',
)}
onClick={() => handleVideoStrategyChange(VideoTranscodeStrategy.all)}
onClick={() => handleVideoStrategyChange(VideoTranscodeStrategy.multi)}
>
<div className="font-semibold">{m.all_resolutions()}</div>
<div className="font-semibold">{m.multi_resolutions()}</div>
<div className="text-sm text-muted-foreground">
{m.all_resolutions_description()}
{m.multi_resolutions_description()}
</div>
</div>
{currentVideoStrategy === VideoTranscodeStrategy.multi && (
<div className="space-y-3 pt-2 pl-1">
<div className="space-y-1">
<h4 className="text-sm font-medium">
{m.video_resolution_ladders()}
</h4>
<p className="text-xs text-muted-foreground">
{m.video_resolution_ladders_description()}
</p>
</div>
<div className="flex flex-wrap gap-4 pt-1">
{ALL_VIDEO_LADDERS.map((ladder) => {
const checked = currentVideoResolutions.includes(ladder)
const isLastSelected =
checked && currentVideoResolutions.length === 1
return (
<label
key={ladder}
className={cn(
'flex items-center space-x-2 text-sm',
isLastSelected
? 'cursor-not-allowed opacity-50'
: 'cursor-pointer',
)}
>
<Checkbox
checked={checked}
disabled={isLastSelected}
onCheckedChange={() => handleVideoLadderToggle(ladder)}
/>
<span>{ladder}</span>
</label>
)
})}
</div>
</div>
)}
</div>
</div>
{/* Hardware Acceleration */}
<div className="space-y-3 pt-6 border-t border-border">
<h3 className="text-lg font-medium">{m.hardware_acceleration()}</h3>
<div className="space-y-3">
<div
className={cn(
'cursor-pointer rounded-lg border p-4 transition-all hover:border-primary',
currentHardwareAcceleration === HardwareAcceleration.off
? 'border-primary bg-primary/5'
: 'border-border',
)}
onClick={() =>
handleHardwareAccelerationChange(HardwareAcceleration.off)
}
>
<div className="font-semibold">{m.hardware_acceleration_off()}</div>
<div className="text-sm text-muted-foreground">
{m.hardware_acceleration_off_description()}
</div>
</div>
<div
className={cn(
'cursor-pointer rounded-lg border p-4 transition-all hover:border-primary',
currentHardwareAcceleration === HardwareAcceleration.auto
? 'border-primary bg-primary/5'
: 'border-border',
)}
onClick={() =>
handleHardwareAccelerationChange(HardwareAcceleration.auto)
}
>
<div className="font-semibold">{m.hardware_acceleration_auto()}</div>
<div className="text-sm text-muted-foreground">
{m.hardware_acceleration_auto_description()}
</div>
</div>
</div>
</div>
{/* FFmpeg Threads */}
{/* HLS Streaming */}
<div className="space-y-4 pt-6 border-t border-border">
<div className="flex items-center justify-between">
<div>
<h3 className="text-lg font-medium">{m.ffmpeg_threads()}</h3>
<div className="space-y-0.5">
<h3 className="text-lg font-medium">{m.hls_streaming()}</h3>
<p className="text-sm text-muted-foreground">
{m.ffmpeg_threads_description()}
{m.hls_streaming_description()}
</p>
</div>
<span className="font-mono text-sm font-semibold px-2.5 py-1 rounded bg-muted text-foreground border border-border">
{currentThreads === 0
? m.threads_auto()
: m.threads_count({ count: currentThreads })}
</span>
</div>
<div className="pt-2 px-1">
<Slider
value={[currentThreads]}
min={0}
max={32}
step={1}
onValueChange={([val]) => setLocalThreads(val)}
onValueCommit={([val]) => {
setLocalThreads(val)
handleThreadsChange(val)
}}
<Switch
checked={currentHlsEnabled}
onCheckedChange={handleHlsEnabledChange}
/>
<div className="flex justify-between text-xs text-muted-foreground mt-2">
<span>{m.threads_auto()} (0)</span>
<span>16</span>
<span>32</span>
</div>
{currentHlsEnabled && (
<div className="space-y-3 pt-2 pl-1">
<div className="space-y-1">
<h4 className="text-sm font-medium">{m.hls_resolution_ladders()}</h4>
<p className="text-xs text-muted-foreground">
{m.hls_resolution_ladders_description()}
</p>
</div>
<div className="flex flex-wrap gap-4 pt-1">
{ALL_HLS_LADDERS.map((ladder) => {
const checked = currentHlsResolutions.includes(ladder)
const isLastSelected = checked && currentHlsResolutions.length === 1
return (
<label
key={ladder}
className={cn(
'flex items-center space-x-2 text-sm',
isLastSelected
? 'cursor-not-allowed opacity-50'
: 'cursor-pointer',
)}
>
<Checkbox
checked={checked}
disabled={isLastSelected}
onCheckedChange={() => handleHlsLadderToggle(ladder)}
/>
<span>{ladder}</span>
</label>
)
})}
</div>
</div>
)}
</div>
</CardContent>
</Card>
{/* Hardware Acceleration Card */}
<Card>
<CardHeader>
<CardTitle>{m.hardware_acceleration()}</CardTitle>
<CardDescription>
{m.hardware_acceleration_section_description()}
</CardDescription>
</CardHeader>
<CardContent className="space-y-3">
<div
className={cn(
'cursor-pointer rounded-lg border p-4 transition-all hover:border-primary',
currentHardwareAcceleration === HardwareAcceleration.off
? 'border-primary bg-primary/5'
: 'border-border',
)}
onClick={() => handleHardwareAccelerationChange(HardwareAcceleration.off)}
>
<div className="font-semibold">{m.hardware_acceleration_off()}</div>
<div className="text-sm text-muted-foreground">
{m.hardware_acceleration_off_description()}
</div>
</div>
<div
className={cn(
'cursor-pointer rounded-lg border p-4 transition-all hover:border-primary',
currentHardwareAcceleration === HardwareAcceleration.auto
? 'border-primary bg-primary/5'
: 'border-border',
)}
onClick={() => handleHardwareAccelerationChange(HardwareAcceleration.auto)}
>
<div className="font-semibold">{m.hardware_acceleration_auto()}</div>
<div className="text-sm text-muted-foreground">
{m.hardware_acceleration_auto_description()}
</div>
</div>
</CardContent>
</Card>
{/* FFmpeg Threads Card */}
<Card>
<CardHeader>
<div className="flex items-center justify-between">
<div>
<CardTitle>{m.ffmpeg_threads()}</CardTitle>
<CardDescription className="mt-1.5">
{m.ffmpeg_threads_description()}
</CardDescription>
</div>
<span className="font-mono text-sm font-semibold px-2.5 py-1 rounded bg-muted text-foreground border border-border">
{currentThreads === 0
? m.threads_auto()
: m.threads_count({ count: currentThreads })}
</span>
</div>
</CardHeader>
<CardContent>
<div className="pt-2 px-1">
<Slider
value={[currentThreads]}
min={0}
max={32}
step={1}
onValueChange={([val]) => setLocalThreads(val)}
onValueCommit={([val]) => {
setLocalThreads(val)
handleThreadsChange(val)
}}
/>
<div className="flex justify-between text-xs text-muted-foreground mt-2">
<span>{m.threads_auto()} (0)</span>
<span>16</span>
<span>32</span>
</div>
</div>
</CardContent>
@@ -0,0 +1,559 @@
// @vitest-environment happy-dom
import { cleanup, render, screen, within, fireEvent, waitFor, act } from '@testing-library/react'
import React from 'react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { TeamSettingsPage } from './settings.lazy'
import { toast } from 'sonner'
import { VideoTranscodeStrategy } from '@shumai/dtos'
interface PatchSettingsArgs {
param: { teamId: string }
json: { key: string; value: string[] }
}
const mockGetSettings = vi.fn()
const mockPatchSettings = vi.fn()
const mockGetMe = vi.fn()
vi.mock('@tanstack/react-router', () => ({
createLazyFileRoute: () => (opts: Record<string, unknown>) => ({
...opts,
useParams: () => ({ teamId: 'team-1' }),
}),
useNavigate: () => vi.fn(),
Link: ({ children, ...props }: React.AnchorHTMLAttributes<HTMLAnchorElement>) => (
<a {...props}>{children}</a>
),
}))
vi.mock('@/ui/api/client', () => ({
client: {
api: {
teams: {
':teamId': {
settings: {
$get: () => mockGetSettings(),
$patch: (args: unknown) => mockPatchSettings(args),
},
me: {
$get: () => mockGetMe(),
$patch: vi.fn().mockResolvedValue({ ok: true, json: async () => ({}) }),
},
},
},
},
},
}))
vi.mock('sonner', () => ({
toast: {
success: vi.fn(),
error: vi.fn(),
},
}))
describe('TeamSettingsPage Transcode Settings', () => {
let queryClient: QueryClient
beforeEach(() => {
vi.clearAllMocks()
window.location.hash = '#transcode'
queryClient = new QueryClient({
defaultOptions: {
queries: { retry: false },
},
})
mockGetMe.mockResolvedValue({
ok: true,
json: async () => ({
id: 'user-1',
name: 'Owner',
role: 'owner',
}),
})
mockGetSettings.mockResolvedValue({
ok: true,
json: async () => ({
transcode: {
videoStrategy: VideoTranscodeStrategy.multi,
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: true,
hlsResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
},
}),
})
})
afterEach(() => {
cleanup()
})
it('keeps pending selections optimistically and serializes consecutive toggles without submitting stale arrays', async () => {
let resolveFirstPatch!: (value: unknown) => void
const firstPatchPromise = new Promise((resolve) => {
resolveFirstPatch = resolve
})
const patchCalls: PatchSettingsArgs[] = []
mockPatchSettings.mockImplementation(async (args: PatchSettingsArgs) => {
patchCalls.push(args)
if (patchCalls.length === 1) {
await firstPatchPromise
}
return {
ok: true,
json: async () => ({
transcode: {
videoResolutions: args.json.value,
},
}),
}
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
// Wait for settings to load
await waitFor(() => {
expect(screen.getAllByText('480p').length).toBeGreaterThan(0)
expect(screen.getAllByText('720p').length).toBeGreaterThan(0)
})
const label480p = screen.getAllByText('480p')[0].closest('label')
const checkbox480p = label480p?.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox480p).toBeDefined()
expect(checkbox480p.getAttribute('aria-checked')).toBe('true')
const label720p = screen.getAllByText('720p')[0].closest('label')
const checkbox720p = label720p?.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox720p).toBeDefined()
expect(checkbox720p.getAttribute('aria-checked')).toBe('true')
// Click 480p to uncheck it
act(() => {
fireEvent.click(checkbox480p)
})
// Optimistically 480p should be unchecked immediately
expect(checkbox480p.getAttribute('aria-checked')).toBe('false')
// Wait for the first PATCH request to be in-flight
await waitFor(() => {
expect(patchCalls.length).toBe(1)
})
expect(patchCalls[0].json.value).toEqual(['720p', '1080p', '1440p', '2160p'])
// Rapid consecutive click: Click 720p to uncheck it WHILE first patch is in-flight
act(() => {
fireEvent.click(checkbox720p)
})
// Optimistically 720p should also be unchecked immediately
expect(checkbox720p.getAttribute('aria-checked')).toBe('false')
// Resolve first patch so the queue advances to the second queued update
await act(async () => {
resolveFirstPatch({
ok: true,
json: async () => ({
transcode: {
videoResolutions: ['720p', '1080p', '1440p', '2160p'],
},
}),
})
})
// Wait for the second queued patch to complete
await waitFor(() => {
expect(patchCalls.length).toBe(2)
})
// The second patch MUST NOT submit stale array with 480p present!
// It must contain neither 480p nor 720p
expect(patchCalls[1].json.value).toEqual(['1080p', '1440p', '2160p'])
expect(patchCalls[1].json.value.includes('480p')).toBe(false)
expect(patchCalls[1].json.value.includes('720p')).toBe(false)
// Verify toast notification on success
expect(toast.success).toHaveBeenCalled()
})
it('coalesces rapid consecutive toggles before network dispatch into a single PATCH', async () => {
const patchCalls: PatchSettingsArgs[] = []
mockPatchSettings.mockImplementation(async (args: PatchSettingsArgs) => {
patchCalls.push(args)
return {
ok: true,
json: async () => ({
transcode: {
videoResolutions: args.json.value,
},
}),
}
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getAllByText('480p').length).toBeGreaterThan(0)
})
const label480p = screen.getAllByText('480p')[0].closest('label')
const checkbox480p = label480p?.querySelector('button[role="checkbox"]') as HTMLElement
const label720p = screen.getAllByText('720p')[0].closest('label')
const checkbox720p = label720p?.querySelector('button[role="checkbox"]') as HTMLElement
// Rapid synchronous clicks in the same event tick
act(() => {
fireEvent.click(checkbox480p)
fireEvent.click(checkbox720p)
})
// Both reflect optimistically
expect(checkbox480p.getAttribute('aria-checked')).toBe('false')
expect(checkbox720p.getAttribute('aria-checked')).toBe('false')
// Wait for the coalesced PATCH to complete
await waitFor(() => {
expect(patchCalls.length).toBeGreaterThanOrEqual(1)
})
// The submitted payload must reflect both toggles
const lastPatch = patchCalls[patchCalls.length - 1]
expect(lastPatch.json.value).toEqual(['1080p', '1440p', '2160p'])
})
it('disables unselecting the last remaining video resolution ladder in MP4 selector', async () => {
mockGetSettings.mockResolvedValue({
ok: true,
json: async () => ({
transcode: {
videoStrategy: VideoTranscodeStrategy.multi,
videoResolutions: ['1080p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: true,
hlsResolutions: ['480p', '720p', '1080p'],
},
}),
})
mockPatchSettings.mockResolvedValue({
ok: true,
json: async () => ({
transcode: {
videoResolutions: ['1080p', '720p'],
},
}),
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getAllByText('1080p').length).toBeGreaterThan(0)
})
const mp4Desc = screen.getByText(
'Select the target resolutions to generate for MP4 transcoding. Resolutions are only generated up to the source video resolution.',
)
const mp4Container = mp4Desc.closest('.space-y-3') as HTMLElement
expect(mp4Container).toBeDefined()
const label1080p = within(mp4Container).getByText('1080p').closest('label')!
const checkbox1080p = label1080p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox1080p).toBeDefined()
expect(checkbox1080p.getAttribute('aria-checked')).toBe('true')
expect(checkbox1080p.hasAttribute('disabled')).toBe(true)
expect(label1080p.className).toContain('cursor-not-allowed')
const label720p = within(mp4Container).getByText('720p').closest('label')!
const checkbox720p = label720p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox720p.getAttribute('aria-checked')).toBe('false')
expect(checkbox720p.hasAttribute('disabled')).toBe(false)
expect(label720p.className).toContain('cursor-pointer')
// Clicking the sole selected resolution must not unselect it or call patch
act(() => {
fireEvent.click(checkbox1080p)
})
expect(checkbox1080p.getAttribute('aria-checked')).toBe('true')
expect(mockPatchSettings).not.toHaveBeenCalled()
// Clicking 720p to add it works
act(() => {
fireEvent.click(checkbox720p)
})
expect(checkbox720p.getAttribute('aria-checked')).toBe('true')
await waitFor(() => {
expect(mockPatchSettings).toHaveBeenCalledWith(
expect.objectContaining({
json: {
key: 'transcode.videoResolutions',
value: ['1080p', '720p'],
},
}),
)
})
})
it('disables unselecting the last remaining HLS resolution ladder in HLS selector', async () => {
mockGetSettings.mockResolvedValue({
ok: true,
json: async () => ({
transcode: {
videoStrategy: VideoTranscodeStrategy.multi,
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: true,
hlsResolutions: ['720p'],
},
}),
})
mockPatchSettings.mockResolvedValue({
ok: true,
json: async () => ({
transcode: {
hlsResolutions: ['720p', '1080p'],
},
}),
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getAllByText('720p').length).toBeGreaterThan(0)
})
const hlsDesc = screen.getByText(
'Select the target resolutions to generate for HLS streaming. Ladders are only generated up to the source video resolution.',
)
const hlsContainer = hlsDesc.closest('.space-y-3') as HTMLElement
expect(hlsContainer).toBeDefined()
const label720p = within(hlsContainer).getByText('720p').closest('label')!
const checkbox720p = label720p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox720p).toBeDefined()
expect(checkbox720p.getAttribute('aria-checked')).toBe('true')
expect(checkbox720p.hasAttribute('disabled')).toBe(true)
expect(label720p.className).toContain('cursor-not-allowed')
const label1080p = within(hlsContainer).getByText('1080p').closest('label')!
const checkbox1080p = label1080p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox1080p.getAttribute('aria-checked')).toBe('false')
expect(checkbox1080p.hasAttribute('disabled')).toBe(false)
expect(label1080p.className).toContain('cursor-pointer')
// Clicking the sole selected HLS resolution must not unselect it or call patch
act(() => {
fireEvent.click(checkbox720p)
})
expect(checkbox720p.getAttribute('aria-checked')).toBe('true')
expect(mockPatchSettings).not.toHaveBeenCalled()
})
it('keeps pending selections optimistically and serializes consecutive toggles for HLS resolutions without submitting stale arrays', async () => {
let resolveFirstPatch!: (value: unknown) => void
const firstPatchPromise = new Promise((resolve) => {
resolveFirstPatch = resolve
})
const patchCalls: PatchSettingsArgs[] = []
mockPatchSettings.mockImplementation(async (args: PatchSettingsArgs) => {
patchCalls.push(args)
if (patchCalls.length === 1) {
await firstPatchPromise
}
return {
ok: true,
json: async () => ({
transcode: {
hlsResolutions: args.json.value,
},
}),
}
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getAllByText('480p').length).toBeGreaterThan(1)
})
const hlsDesc = screen.getByText(
'Select the target resolutions to generate for HLS streaming. Ladders are only generated up to the source video resolution.',
)
const hlsContainer = hlsDesc.closest('.space-y-3') as HTMLElement
const label480p = within(hlsContainer).getByText('480p').closest('label')!
const checkbox480p = label480p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox480p.getAttribute('aria-checked')).toBe('true')
const label720p = within(hlsContainer).getByText('720p').closest('label')!
const checkbox720p = label720p.querySelector('button[role="checkbox"]') as HTMLElement
expect(checkbox720p.getAttribute('aria-checked')).toBe('true')
// Click 480p to uncheck it
act(() => {
fireEvent.click(checkbox480p)
})
expect(checkbox480p.getAttribute('aria-checked')).toBe('false')
// Wait for the first PATCH request to be in-flight
await waitFor(() => {
expect(patchCalls.length).toBe(1)
})
expect(patchCalls[0].json.value).toEqual(['720p', '1080p', '1440p', '2160p'])
// Rapid consecutive click: Click 720p to uncheck it WHILE first patch is in-flight
act(() => {
fireEvent.click(checkbox720p)
})
expect(checkbox720p.getAttribute('aria-checked')).toBe('false')
// Resolve first patch so the queue advances
await act(async () => {
resolveFirstPatch({
ok: true,
json: async () => ({
transcode: {
hlsResolutions: ['720p', '1080p', '1440p', '2160p'],
},
}),
})
})
// Wait for the second queued patch to complete
await waitFor(() => {
expect(patchCalls.length).toBe(2)
})
// The second patch must not submit stale array with 480p present
expect(patchCalls[1].json.value).toEqual(['1080p', '1440p', '2160p'])
expect(patchCalls[1].json.value.includes('480p')).toBe(false)
expect(patchCalls[1].json.value.includes('720p')).toBe(false)
expect(toast.success).toHaveBeenCalled()
})
it('coalesces rapid consecutive toggles for HLS resolutions into a single PATCH', async () => {
const patchCalls: PatchSettingsArgs[] = []
mockPatchSettings.mockImplementation(async (args: PatchSettingsArgs) => {
patchCalls.push(args)
return {
ok: true,
json: async () => ({
transcode: {
hlsResolutions: args.json.value,
},
}),
}
})
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getAllByText('480p').length).toBeGreaterThan(1)
})
const hlsDesc = screen.getByText(
'Select the target resolutions to generate for HLS streaming. Ladders are only generated up to the source video resolution.',
)
const hlsContainer = hlsDesc.closest('.space-y-3') as HTMLElement
const label480p = within(hlsContainer).getByText('480p').closest('label')!
const checkbox480p = label480p.querySelector('button[role="checkbox"]') as HTMLElement
const label720p = within(hlsContainer).getByText('720p').closest('label')!
const checkbox720p = label720p.querySelector('button[role="checkbox"]') as HTMLElement
// Rapid synchronous clicks in the same event tick
act(() => {
fireEvent.click(checkbox480p)
fireEvent.click(checkbox720p)
})
// Both reflect optimistically
expect(checkbox480p.getAttribute('aria-checked')).toBe('false')
expect(checkbox720p.getAttribute('aria-checked')).toBe('false')
// Wait for the coalesced PATCH to complete
await waitFor(() => {
expect(patchCalls.length).toBeGreaterThanOrEqual(1)
})
const lastPatch = patchCalls[patchCalls.length - 1]
expect(lastPatch.json.value).toEqual(['1080p', '1440p', '2160p'])
})
it('renders individual cards for Transcode Policy, Hardware Acceleration, and FFmpeg Threads with updated labels', async () => {
render(
<QueryClientProvider client={queryClient}>
<TeamSettingsPage />
</QueryClientProvider>,
)
await waitFor(() => {
expect(screen.getByRole('heading', { level: 2, name: /Video Transcoding/i })).toBeDefined()
})
// Sidebar navigation
expect(screen.getByText('Team AI Settings')).toBeDefined()
expect(screen.getByRole('button', { name: /Video Transcoding/i })).toBeDefined()
// Cards
expect(screen.getByText('Transcode Policy')).toBeDefined()
expect(
screen.getByText(
'When HLS is disabled, both playback and download use MP4. When HLS is enabled, playback uses HLS streaming while download uses MP4. Note that enabling HLS may significantly increase storage usage.',
),
).toBeDefined()
expect(screen.getByText('Hardware Acceleration')).toBeDefined()
expect(screen.getByText('FFmpeg Threads')).toBeDefined()
// MP4 Transcoding sub-section
expect(screen.getByText('MP4 Transcoding')).toBeDefined()
expect(
screen.getByText(
'Generate web-compatible MP4 files for file downloads and standard video playback.',
),
).toBeDefined()
// Redundant "Transcode Settings" card title should not be rendered
expect(screen.queryByRole('heading', { level: 3, name: 'Transcode Settings' })).toBeNull()
})
})

Some files were not shown because too many files have changed in this diff Show More