Compare commits

...
23 Commits
Author SHA1 Message Date
yiling f8beb195dd Release v0.5.1 2026-10-06 22:29:50 +08:00
yiling a97996cb59 docs(changelog): note clickable notification cards 2026-10-06 22:18:18 +08:00
jorgeandYiling-J f870ca9b49 feat(asset): link to assets on notification cards. (#451)
* feat(asset): link to assets on notification cards.

* Update packages/webui/components/notification-card.tsx

Co-authored-by: Yiling-J <njjyl723@gmail.com>

---------

Co-authored-by: Yiling-J <njjyl723@gmail.com>
2026-10-06 22:14:11 +08:00
Yiling-J 2e5ff281a8 feat(transcode): add hardware decoding support for nvenc, qsv, and rkmpp (#452)
* feat(transcode): add hardware decoding support for nvenc, qsv, and rkmpp

* refactor(transcode): eliminate redundant hardware decode gating and dead condition
2026-10-06 17:35:14 +08:00
yiling de8decf113 fix(ci): prevent tagging main on PR workflow_dispatch docker builds 2026-10-06 11:49:49 +08:00
Cameron HarrisandClaude Opus 5.5 dd57fb9ed2 feat(transcode): decode and scale on the GPU for hardware transcodes (VA-API) (#449)
* feat(transcode): decode and scale on the GPU for VA-API transcodes

When the selected encoder is h264_vaapi, decode with -hwaccel vaapi and
scale with scale_vaapi so frames stay in GPU memory, instead of decoding
and scaling on the CPU and hwupload-ing before the encoder. On a Haswell
iGPU this makes 4K proxy/HLS renditions 3-5x faster.

Fallback order: GPU decode + VA-API encode -> software decode + VA-API
encode (previous behavior) -> libx264. Watermark overlays, HDR sources
and rotated sources skip GPU decode: ffmpeg cannot auto-insert its
rotation filter on hardware frames, so rotated input would otherwise come
out sideways instead of failing. SHUMAI_VAAPI_HW_DECODE=false disables it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(transcode): document VA-API GPU decoding and SHUMAI_VAAPI_HW_DECODE

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(transcode): generic hardware decode config, stall watchdog, reuse probed rotation

Follow-ups from comparing with Immich and Jellyfin:

- Replace the VA-API-specific helpers with a per-encoder HW_DECODE_CONFIGS
  table (input args, output args, GPU scale filter) and build the MP4 and
  HLS input/filter graph in one shared method so the paths cannot drift.
  Only h264_vaapi has an entry; other backends can be added once tested on
  real hardware. The opt-out is now SHUMAI_HW_DECODE=false.
- Kill a GPU-decode attempt with SIGKILL when its output stops growing for
  SHUMAI_HW_DECODE_STALL_TIMEOUT (default 120s) and fall back, since some
  VA-API driver failures hang ffmpeg instead of exiting (immich#31933).
- Add -noautoscale on the GPU-decode path so a mid-stream decoder reinit
  (e.g. 1080 -> 1088) cannot auto-insert a software scaler into the GPU
  graph (jellyfin#17133).
- Pass the rotation already probed into MediaMetadata through the workflow
  and activities as sourceRotation; ffprobe only runs when it is missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:00:01 +08:00
Yiling-J 9a640632aa ci(release): allow manual Docker image build from external PR (#450) 2026-10-05 09:48:48 +08:00
yiling e61641d7ec fix(workflow): increase activity startToCloseTimeout to 120 minutes 2026-10-04 10:22:31 +08:00
Yiling-J 7ca8ba8b2a fix(transcode): resolve RAW transcoding hang in Docker deployments (#447) 2026-10-03 12:45:12 +08:00
Yiling-J 8805d5799a feat(transcode): add dcraw_emu fallback for RAW image preview generation (#446) 2026-10-03 11:03:31 +08:00
yiling 013c4a8156 Release v0.5.0 2026-10-02 20:28:37 +08:00
Yiling-J 20b5ebeefd fix(asset): mark asset as failed and display failure state in webui when transcoding fails (#445)
* fix(asset): mark asset as failed and display failure state in webui when transcoding fails

* fix(transcode): do not mark asset as failed if media processing already succeeded

* test(workflow): add e2e test for video transcode failure status transition

* fix(webui): update failed file card badge to top-right Transcoding Failed label with error tooltip
2026-10-02 20:09:32 +08:00
Yiling-J daa20aed20 fix(webui): add video loading state and poster placeholder (#444)
* fix(webui): add video loading state and poster placeholder

* fix(webui): prevent buffer progress events from canceling stall indicator
2026-10-02 16:19:49 +08:00
Yiling-J 3564d4d164 fix(transcode): avoid selecting attached picture streams for video transcoding (#443)
* fix(transcode): avoid selecting attached picture streams for video transcoding

Filter out attached picture streams (disposition.attached_pic) during video probing and stream selection, matching Immich's default and bitrate selection strategy. Forward explicit primary video and audio stream indices to FFmpeg pipelines to prevent picking embedded cover art or thumbnail streams.

* fix(transcode): skip poster and sprite generation for audio-only assets and add multi-stream e2e tests

* test(workflow): scope pixel format to h264 stream and use yuvj420p for mjpeg cover art
2026-10-02 13:51:17 +08:00
Yiling-J 206763e966 feat(auth): add demo user login button and update demo description (#442)
* feat(auth): add demo user login button and update description

* feat(auth): move demo hint below button and add discord link
2026-10-01 22:04:15 +08:00
Yiling-J 4e4da595a6 feat(settings): redesign video transcoding settings into dedicated cards and rename team AI settings (#441)
* feat(settings): redesign video transcoding settings into dedicated cards and rename team AI settings

* feat(settings): rename Video Strategy to MP4 Transcoding with sub-description
2026-10-01 21:19:25 +08:00
Yiling-J a21a0c9655 fix(settings): prevent deselecting last ladder and coalesce rapid toggles (#440) 2026-10-01 16:35:47 +08:00
Yiling-J fc7282581f fix(share): prevent unwatermarked HLS playback on public shares before transcode completion (#439) 2026-10-01 15:41:45 +08:00
Yiling-J e007054f47 feat(transcode): add multi-resolution strategy with configurable ladders and fallback (#438)
* feat(transcode): add multi-resolution strategy with configurable ladders and fallback

* feat(settings): show toast on settings update and add manual verification plan to PR template

* fix(webui): preserve video resolution and fix 480p label in download dropdowns

* fix(webui): display standard short-side resolution for vertical videos in seekbar

* fix(webui): fix portrait transcode resolution labeling and serialize settings toggles
2026-10-01 13:27:50 +08:00
Yiling-J a84da8678e feat(webui): switch from video.js to hls.js and native video player (#437)
* feat(webui): switch from video.js to hls.js and native video player

* fix(webui): prevent stale seek hijacking and guard native HLS compare mode
2026-09-30 20:23:44 +08:00
Yiling-J 123a8cb5f4 feat(transcode): add HLS live streaming support (#436)
* feat(transcode): add HLS live streaming support

* fix(hls): resolve version stacks for public HLS and add workflow E2E test
2026-09-30 16:31:40 +08:00
Yiling-J 5fe22ac52c feat(transcode): add initial camera RAW image support (#435)
- Add centralized RAW file extension set and detection helper for 23 common camera RAW formats
- Update getProxyType and EXTENSION_MIME_TYPES to route RAW formats into the image pipeline
- Add exiftool-vendored runtime dependency and mark as external in build-npm script
- Implement embedded JPEG extraction to temporary file with priority JpgFromRaw2 -> JpgFromRaw -> PreviewImage
- Read and apply EXIF orientation from RAW container to prevent rotation/mirroring issues
- Integrate into TranscodeService.getImageInfo and transcodeImage with proper temporary file cleanup
- Add hasUsablePreview guard in transcodeImageWorkflow to gracefully handle RAW files without embedded previews
- Add unit, service, workflow, and E2E tests for RAW processing
2026-09-29 21:09:27 +08:00
Yiling-J 11d21444cf fix(trash): prevent empty trash dialog from closing before API call completes (#434)
AlertDialogAction from Radix UI auto-closes the dialog on click.
Added e.preventDefault() so the dialog stays open and shows the
'Deleting...' loading state until the operation finishes.

Fixes both desktop and mobile file browsers.
2026-09-29 10:40:13 +08:00
115 changed files with 11082 additions and 1384 deletions
+11 -1
View File
@@ -10,6 +10,10 @@ on:
description: 'Docker image tag (e.g. feature-vaapi)'
required: false
type: string
pr:
description: 'Pull request number (e.g. 449)'
required: false
type: string
jobs:
build-and-push:
@@ -20,7 +24,10 @@ jobs:
steps:
- name: Checkout repository
id: checkout
uses: actions/checkout@v4
with:
ref: ${{ inputs.pr && format('refs/pull/{0}/head', inputs.pr) || github.ref }}
# Set up Docker Buildx
- name: Set up Docker Buildx
@@ -51,8 +58,11 @@ jobs:
crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai
tags: |
type=semver,pattern={{version}}
type=ref,event=branch
type=ref,event=branch,enable=${{ github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && !inputs.pr && !inputs.tag) }}
type=raw,value=pr-${{ inputs.pr }},enable=${{ github.event_name == 'workflow_dispatch' && inputs.pr != '' && inputs.tag == '' }}
type=raw,value=${{ inputs.tag }},enable=${{ github.event_name == 'workflow_dispatch' && inputs.tag != '' }}
labels: |
org.opencontainers.image.revision=${{ steps.checkout.outputs.commit }}
# Build and push Docker image
- name: Build and push Docker image
+7
View File
@@ -92,6 +92,13 @@ Briefly describe what this PR changes and why.
- Describe the checks or tests you ran.
- Include relevant outputs if applicable.
## Manual Verification Plan
- Provide concise instructions for manually verifying this change.
- Assume familiarity with the project and existing development workflow.
- Skip boilerplate steps (e.g. starting the app, logging in, opening common pages) unless they are part of the change.
- Focus on what needs to be verified, including scenarios, edge cases, and expected results.
## Notes
Add any additional context, caveats, or follow-up work.
+36
View File
@@ -12,6 +12,42 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Changed
## [0.5.1] - 2026-10-06
### Added
* **notification**: Notification cards are now clickable, taking you directly to the relevant file
* **transcode**: Add hardware decoding and GPU scaling support for NVIDIA NVENC, Intel QuickSync Video (QSV), Rockchip (RKMPP), and VA-API (Intel and AMD GPUs), significantly speeding up video proxy and HLS rendition generation across supported platforms. Set `SHUMAI_HW_DECODE=false` to turn hardware decoding off
* **transcode**: Add `dcraw_emu` fallback to generate WebP previews for camera RAW files without embedded JPEG previews
### Fixed
### Changed
## [0.5.0] - 2026-10-02
### Added
* **transcode**: Add configurable target resolution ladders (480p, 720p, 1080p, 1440p, 2160p) for MP4 Multi Resolutions transcoding in team settings, with unified resolution matching and lowest-resolution fallback for both MP4 and HLS
* **transcode**: Add HLS live streaming with fMP4/CMAF adaptive bitrate ladders (480p, 720p, 1080p, 1440p, 2160p), configurable in Transcode Settings, with automatic quality switching, manual resolution selection in the video viewer, seamless rendition switching, and full watermarking support for shared links
* **transcode**: Add basic camera RAW image support for popular camera formats (including Sony ARW, Canon CR2/CR3, Nikon NEF, Adobe DNG, Fujifilm RAF, and Panasonic RW2), extracting high-quality embedded JPEG previews and automatically applying camera orientation for fast thumbnail and proxy generation
### Fixed
- **asset**: Fix an issue where media files remained stuck in a perpetual "Preparing..." state when transcoding failed, by displaying a clear failed status and error details in the file list and file viewer
* **transcode**: Fix an issue where videos or audio-only files containing attached cover art or picture streams could cause transcoding to fail during poster generation or select a single static image instead of the video
* **webui**: Fix inconsistent video resolution labels across download menus, mobile file headers, and the video player by correctly deriving resolutions from the video's short side, including portrait videos and 480p videos
* **trash**: Fix an issue where the "Empty Trash" confirmation dialog would close immediately instead of showing a loading state while files were being permanently deleted
### Changed
* **settings**: Redesign team Video Transcoding settings with dedicated cards for Transcode Policy, Hardware Acceleration, and FFmpeg Threads, clearly explaining MP4 and HLS playback vs download behavior and storage implications
* **settings**: Rename AI Settings group to Team AI Settings in settings navigation
## [0.4.9] - 2026-09-28
### Added
+2
View File
@@ -52,6 +52,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
poppler-utils \
imagemagick \
fonts-noto-cjk \
libraw-bin \
libimage-exiftool-perl \
&& . /etc/os-release \
&& TARGETARCH=$(dpkg --print-architecture) \
&& FFMPEG_VERSION="8.1.2-5" \
+6 -4
View File
@@ -162,23 +162,24 @@ Shumai requires the following system packages on Linux:
- **`ffmpeg`** – Used for media transcoding and metadata extraction.
- **`poppler`** (`poppler-utils`) – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`).
- **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB.
- **`dcraw_emu`** (`libraw` / `libraw-bin`) – Used for camera RAW format image decoding and preview generation.
- **`bubblewrap`**, **`socat`**, and **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`) for process isolation, networking, and workspace search.
Install all required packages with one command:
> Ubuntu/Debian
```bash
sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep
sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep libraw-bin
```
> Fedora
```bash
sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep
sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep LibRaw
```
> Arch Linux
```bash
sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep
sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep libraw
```
> [!NOTE]
@@ -201,12 +202,13 @@ Shumai requires:
- **`ffmpeg`** – Used for media transcoding and metadata extraction.
- **`poppler`** – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`).
- **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB.
- **`dcraw_emu`** (`libraw`) – Used for camera RAW format image decoding and preview generation.
- **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`).
Install the required packages with Homebrew:
```bash
brew install ffmpeg poppler imagemagick ripgrep
brew install ffmpeg poppler imagemagick ripgrep libraw
```
---
+7 -5
View File
@@ -187,23 +187,24 @@ mkdir shumai && cd shumai
- **`ffmpeg`** —— 用于媒体转码和元数据提取。
- **`poppler`** (`poppler-utils`) —— 用于 PDF 页面图像提取和 PDF 雪碧图预览生成 (`pdftoppm`)。
- **`imagemagick`** —— 用于 PSD 格式图像转码和 sRGB 色彩空间转换。
- **`dcraw_emu`** (`libraw` / `libraw-bin`) —— 用于相机 RAW 格式图像解码和预览图生成。
- **`bubblewrap`**、**`socat`** 和 **`ripgrep`** —— AI Agent 沙箱(`anthropic-experimental/sandbox-runtime`)所需,用于进程隔离、网络通信和工作区搜索。
可以使用以下命令一次性安装所有依赖:
> Ubuntu/Debian
```bash
sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep
sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep libraw-bin
```
> Fedora
```bash
sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep
sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep LibRaw
```
> Arch Linux
```bash
sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep
sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep libraw
```
> [!NOTE]
@@ -215,7 +216,7 @@ sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep
> sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
> ```
>
> 或者,配置 AppArmor Profile,为相关可执行文件授予所需的 `userns` 权限。
> 或者,配置 AppArmor Profile,为相关可执行文件授予所需的 `userns`权限。
---
@@ -226,12 +227,13 @@ sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep
- **`ffmpeg`** —— 用于媒体转码和元数据提取。
- **`poppler`** —— 用于 PDF 页面图像提取和 PDF 雪碧图预览生成 (`pdftoppm`)。
- **`imagemagick`** —— 用于 PSD 格式图像转码和 sRGB 色彩空间转换。
- **`dcraw_emu`** (`libraw`) —— 用于相机 RAW 格式图像解码和预览图生成。
- **`ripgrep`** —— AI Agent 沙箱(`anthropic-experimental/sandbox-runtime`)所需。
使用 Homebrew 安装所需依赖:
```bash
brew install ffmpeg poppler imagemagick ripgrep
brew install ffmpeg poppler imagemagick ripgrep libraw
```
---
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/premiere-uxp",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "commonjs",
"scripts": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"dependencies": {
+35 -50
View File
@@ -33,7 +33,7 @@
},
"apps/agent": {
"name": "@shumai/agent-app",
"version": "0.4.9",
"version": "0.5.1",
"bin": {
"shumai-agent": "./src/index.ts",
},
@@ -45,7 +45,7 @@
},
"apps/cli": {
"name": "@shumai/cli",
"version": "0.4.9",
"version": "0.5.1",
"bin": {
"shumai-cli": "./src/index.ts",
},
@@ -60,7 +60,7 @@
},
"apps/premiere-uxp": {
"name": "@shumai/premiere-uxp",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@shumai/api": "workspace:*",
"@shumai/dtos": "workspace:*",
@@ -114,7 +114,7 @@
},
"apps/transcode": {
"name": "@shumai/transcode-app",
"version": "0.4.9",
"version": "0.5.1",
"bin": {
"shumai-transcode": "./src/index.ts",
},
@@ -126,7 +126,7 @@
},
"apps/web": {
"name": "@shumai/web-app",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@shumai/agent": "workspace:*",
"@shumai/api": "workspace:*",
@@ -146,7 +146,7 @@
},
"packages/agent": {
"name": "@shumai/agent",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.74",
"@earendil-works/pi-agent-core": "0.82.1",
@@ -171,7 +171,7 @@
},
"packages/api": {
"name": "@shumai/api",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@hono/zod-validator": "0.7.6",
"@shumai/core": "workspace:*",
@@ -185,7 +185,7 @@
},
"packages/core": {
"name": "@shumai/core",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@ai-sdk/alibaba": "2.0.38",
"@ai-sdk/amazon-bedrock": "5.0.67",
@@ -223,6 +223,7 @@
"date-fns": "4.4.0",
"dotenv": "17.4.2",
"dotenv-expand": "13.0.0",
"exiftool-vendored": "38.1.0",
"extract-zip": "2.0.1",
"hono": "4.13.5",
"jittered-fractional-indexing": "1.0.1",
@@ -246,7 +247,7 @@
},
"packages/db": {
"name": "@shumai/db",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@prisma/adapter-pg": "7.10.0",
@@ -262,7 +263,7 @@
},
"packages/dtos": {
"name": "@shumai/dtos",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"zod": "4.5.4",
@@ -270,7 +271,7 @@
},
"packages/e2e": {
"name": "@shumai/e2e",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@google/genai": "2.19.0",
@@ -285,11 +286,11 @@
},
"packages/tableprinter": {
"name": "@shumai/tableprinter",
"version": "0.4.9",
"version": "0.5.1",
},
"packages/transcode": {
"name": "@shumai/transcode",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -303,7 +304,7 @@
},
"packages/webui": {
"name": "@shumai/webui",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@base-ui/react": "1.7.0",
"@dnd-kit/abstract": "0.5.0",
@@ -363,6 +364,7 @@
"cmdk": "1.1.1",
"date-fns": "4.4.0",
"embla-carousel-react": "8.6.0",
"hls.js": "1.7.3",
"hono": "4.13.5",
"input-otp": "1.5.0",
"konva": "10.3.2",
@@ -386,7 +388,6 @@
"ulid": "3.0.2",
"use-image": "1.1.4",
"vaul": "1.1.2",
"video.js": "8.24.0",
"zod": "4.5.4",
"zustand": "5.0.15",
},
@@ -405,7 +406,7 @@
},
"packages/workflow-core": {
"name": "@shumai/workflow-core",
"version": "0.4.9",
"version": "0.5.1",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -911,6 +912,8 @@
"@oxc-project/types": ["@oxc-project/types@0.147.0", "", {}, "sha512-IJ3s6ltHLp45S0bh7phkX+gJO7A1Wuz2EaqpAhb8WjqDwbzMiWKHhyyT42tskaWjEYXtHtVCPpnBJVT9+dcRLg=="],
"@photostructure/tz-lookup": ["@photostructure/tz-lookup@11.7.0", "", {}, "sha512-QbKHXvleluMIlAHf2JiQTJNU5rLQBeQhLobyRqgXZ6NNGKqNikaRkjYviJ4nMU7Y75QE7bBYM8xjiFGjAukyhw=="],
"@pinojs/redact": ["@pinojs/redact@0.4.0", "", {}, "sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg=="],
"@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="],
@@ -1561,6 +1564,8 @@
"@types/lodash": ["@types/lodash@4.17.25", "", {}, "sha512-+K1NIO8I+F9/wNulfVvu23QYd0Pe9/OCqRrim4NoYIf1VoEDL90Ve4ClzpyqBLc7NpGGWRvYNCKZ1BE/Jpf8dQ=="],
"@types/luxon": ["@types/luxon@3.7.5", "", {}, "sha512-jJ41Q4z6ZVO260MNDdHfW7+7a5iMiX8Mr6ZJHcmgrvhZha6dz5704o/lF2kKl6URjH6ivEL97w9xS/MgpJEphg=="],
"@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="],
"@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="],
@@ -1627,12 +1632,6 @@
"@vercel/oidc": ["@vercel/oidc@3.2.0", "", {}, "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug=="],
"@videojs/http-streaming": ["@videojs/http-streaming@3.17.5", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.2", "aes-decrypter": "^4.0.2", "global": "^4.4.0", "m3u8-parser": "^7.2.0", "mpd-parser": "^1.4.0", "mux.js": "7.1.0", "video.js": "^7 || ^8" } }, "sha512-PRRQVu0hYW5h0XvRWpIyIpEatpMZzZtf7L1avV89Hnu+69f+0c7oLsSL9jwpg2nbvzGkdX5rQxQkz5Vc7tFR9w=="],
"@videojs/vhs-utils": ["@videojs/vhs-utils@4.1.2", "", { "dependencies": { "@babel/runtime": "^7.12.5", "global": "^4.4.0" } }, "sha512-1Dsv5dtWOq0LP1G37O46FQFA2tKAZON5UMe6dFPxT7JvPhEx8QIXcFQxan8+hHI3fX6aomFRFrUBJNgRYQDSXA=="],
"@videojs/xhr": ["@videojs/xhr@2.7.0", "", { "dependencies": { "@babel/runtime": "^7.5.5", "global": "~4.4.0", "is-function": "^1.0.1" } }, "sha512-giab+EVRanChIupZK7gXjHy90y3nncA2phIOyG3Ne5fvpiMJzvqYwiTOnEVW2S4CoYcuKJkomat7bMXA/UoUZQ=="],
"@visx/curve": ["@visx/curve@4.0.1-alpha.0", "", { "dependencies": { "@visx/vendor": "4.0.0-alpha.0" } }, "sha512-jRu61Uz274pV1zyioXmboyrLutYbnKsgjj4njSGCnhdXj5GkZvZbg+ThDb6oOzoAnJOBRLz4rzPlWvNJOzuVMg=="],
"@visx/event": ["@visx/event@4.0.1-alpha.0", "", { "dependencies": { "@types/react": "*", "@visx/point": "4.0.1-alpha.0" } }, "sha512-EQqCMSv/s8NbFjo+hz3FKsvvYfP+2QslsFJ/24/O5l/W+7UC6J6aAvO0ujVwrTwdYbuQ+vhxKi1xdPdKR/qj1g=="],
@@ -1697,8 +1696,6 @@
"@workflow/serde": ["@workflow/serde@4.1.0", "", {}, "sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ=="],
"@xmldom/xmldom": ["@xmldom/xmldom@0.8.15", "", {}, "sha512-/5NV/vDALVFDXgLmfsy9TRCBlKwO2LNBFzpzvb9iIj+jR+eSc6DLYYvVOdivT/jm7MtU6TebYuRmzEOI7w40UA=="],
"@xtuc/ieee754": ["@xtuc/ieee754@1.2.0", "", {}, "sha512-DX8nKgqcGwsc0eJSqYt5lwP4DH5FlHnmuWWBRy7X0NcaGR0ZtuyeESgMwTYVEtxmsNGY+qit4QYT/MIYTOTPeA=="],
"@xtuc/long": ["@xtuc/long@4.2.2", "", {}, "sha512-NuHqBY1PB/D8xU6s/thBgOAiAP7HOYDQ32+BFZILJ8ivkUkAHQnWfn6WhL79Owj1qmUnoN/YPhktdIoucipkAQ=="],
@@ -1711,8 +1708,6 @@
"adm-zip": ["adm-zip@0.6.0", "", {}, "sha512-XleryMhbuksdKtofnWZ9Sk+4CUTbms4Mb/EU32SZwToAyZ5RgVos/ki8n+yr0LWHOGKuakbXTuuYNHLQjhddgg=="],
"aes-decrypter": ["aes-decrypter@4.0.2", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.1", "global": "^4.4.0", "pkcs7": "^1.0.4" } }, "sha512-lc+/9s6iJvuaRe5qDlMTpCFjnwpkeOXp8qP3oiZ5jsj1MRg+SBVUmmICrhxHvc8OELSmc+fEyyxAuppY6hrWzw=="],
"agent-base": ["agent-base@7.1.4", "", {}, "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ=="],
"ai": ["ai@7.0.84", "", { "dependencies": { "@ai-sdk/gateway": "4.0.68", "@ai-sdk/provider": "4.0.8", "@ai-sdk/provider-utils": "5.0.33" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-znHNBWADOy79k8EK7636kDm5LmnjF4tMzegC2TfmlLDobDZbs0z0OXD2sxtH3eDTp5Re+pJQESvjinWJ6+kadQ=="],
@@ -1795,6 +1790,8 @@
"baseline-browser-mapping": ["baseline-browser-mapping@2.11.20", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-H0ulySigv6icDJ1F7SjtdCD6PrhTpdYCmP0CactWy1+ekh0AFd0o1Wn5T8b+hnTmdBx19u9yhL6wvCylXMY7zw=="],
"batch-cluster": ["batch-cluster@19.2.0", "", {}, "sha512-mvozaN/7TYxl+lmXDYH+HocCcRMvW25QmghrJux2y93o+NhHPQ4wir0nBS1p8oTlinjc/l5LaNX+VFxNISrqgQ=="],
"bcrypt-pbkdf": ["bcrypt-pbkdf@1.0.2", "", { "dependencies": { "tweetnacl": "^0.14.3" } }, "sha512-qeFIXtP4MSoi6NLqO12WfqARWWuCKi2Rn/9hJLEmtB5yTNr9DqFWkJRCf2qShWzPeAMRnOgCrq0sg/KLv5ES9w=="],
"better-auth": ["better-auth@1.6.11", "", { "dependencies": { "@better-auth/core": "1.6.11", "@better-auth/drizzle-adapter": "1.6.11", "@better-auth/kysely-adapter": "1.6.11", "@better-auth/memory-adapter": "1.6.11", "@better-auth/mongo-adapter": "1.6.11", "@better-auth/prisma-adapter": "1.6.11", "@better-auth/telemetry": "1.6.11", "@better-auth/utils": "0.4.0", "@better-fetch/fetch": "1.1.21", "@noble/ciphers": "^2.1.1", "@noble/hashes": "^2.0.1", "better-call": "1.3.5", "defu": "^6.1.4", "jose": "^6.1.3", "kysely": "^0.28.17", "nanostores": "^1.1.1", "zod": "^4.3.6" }, "peerDependencies": { "@lynx-js/react": "*", "@prisma/client": "^5.0.0 || ^6.0.0 || ^7.0.0", "@sveltejs/kit": "^2.0.0", "@tanstack/react-start": "^1.0.0", "@tanstack/solid-start": "^1.0.0", "better-sqlite3": "^12.0.0", "drizzle-kit": ">=0.31.4", "drizzle-orm": "^0.45.2", "mongodb": "^6.0.0 || ^7.0.0", "mysql2": "^3.0.0", "next": "^14.0.0 || ^15.0.0 || ^16.0.0", "pg": "^8.0.0", "prisma": "^5.0.0 || ^6.0.0 || ^7.0.0", "react": "^18.0.0 || ^19.0.0", "react-dom": "^18.0.0 || ^19.0.0", "solid-js": "^1.0.0", "svelte": "^4.0.0 || ^5.0.0", "vitest": "^2.0.0 || ^3.0.0 || ^4.0.0", "vue": "^3.0.0" }, "optionalPeers": ["@lynx-js/react", "@prisma/client", "@sveltejs/kit", "@tanstack/react-start", "@tanstack/solid-start", "better-sqlite3", "drizzle-kit", "drizzle-orm", "mongodb", "mysql2", "next", "pg", "prisma", "react", "react-dom", "solid-js", "svelte", "vitest", "vue"] }, "sha512-Wwt6+q07dwIhsp6XiM7L1qSXVUWBEtNl+eZvwM778CguFqDZFBN9Pt6LtFaHl55t8Z+Zc//5kxcbgDY8/79vFQ=="],
@@ -2001,8 +1998,6 @@
"dom-accessibility-api": ["dom-accessibility-api@0.5.16", "", {}, "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg=="],
"dom-walk": ["dom-walk@0.1.2", "", {}, "sha512-6QvTW9mrGeIegrFXdtQi9pk7O/nSK6lSdXW2eqUspN5LWD7UTji2Fqw5V2YLjBpHEoU9Xl/eUWNpDeZvoyOv2w=="],
"dotenv": ["dotenv@17.4.2", "", {}, "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw=="],
"dotenv-expand": ["dotenv-expand@13.0.0", "", { "dependencies": { "dotenv": "^17.4.2" } }, "sha512-aBfBS8eYIeXmpHI9ThIlA7/WLq+SLt18iXUZhb52rW89QLKQFoIpPG1bPeewoPZsTyjSSO3T7234FBVUM1V2rA=="],
@@ -2105,6 +2100,12 @@
"eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="],
"exiftool-vendored": ["exiftool-vendored@38.1.0", "", { "dependencies": { "@photostructure/tz-lookup": "^11.7.0", "@types/luxon": "^3.7.5", "batch-cluster": "^19.2.0", "he": "^1.2.0", "luxon": "^3.7.2" }, "optionalDependencies": { "exiftool-vendored.exe": "13.59.2", "exiftool-vendored.pl": "13.59.2" } }, "sha512-LIERqM/Qb0oqJlSswLFKWsxGVH81skp7omsX2BPKA/dEzfLP5syNO5KL7k5UlMCf+m0jqEXrKTQr/xV5vScPow=="],
"exiftool-vendored.exe": ["exiftool-vendored.exe@13.59.2", "", { "os": "win32" }, "sha512-U0l0CApIGO+AjpFYCiNCApQsMdlf6sqCyEl4wlKKqvwVAEVcvMRw91k7zj32OPFZE+/+XlLFsc6876MpOqBLiw=="],
"exiftool-vendored.pl": ["exiftool-vendored.pl@13.59.2", "", { "os": "!win32", "bin": { "exiftool": "bin/exiftool" } }, "sha512-5B80JT9phZSUsc7zz77rQ6DfW0YssTpJ5qW/1bOH1N8XQq/jcn+9eRW9HFwhVT9uoIliw9Fg007ST3pUzQIzyw=="],
"expect-type": ["expect-type@1.4.0", "", {}, "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA=="],
"exsolve": ["exsolve@1.1.1", "", {}, "sha512-9U/jZUgjnSGyntRr6y5Muu1MJcwFl6kPu7k8qLF0IMNfLqvw0NZ4nnVDq0RVoZ0RvCyumib4Ez3KYrVfilrw+g=="],
@@ -2213,8 +2214,6 @@
"glob-to-regex.js": ["glob-to-regex.js@1.2.0", "", { "peerDependencies": { "tslib": "2" } }, "sha512-QMwlOQKU/IzqMUOAZWubUOT8Qft+Y0KQWnX9nK3ch0CJg0tTp4TvGZsTfudYKv2NzoQSyPcnA6TYeIQ3jGichQ=="],
"global": ["global@4.4.0", "", { "dependencies": { "min-document": "^2.19.0", "process": "^0.11.10" } }, "sha512-wv/LAoHdRE3BeTGz53FAamhGlPLhlssK45usmGFThIi4XqnBmjKQ16u+RNbP7WvigRZDxUsM0J3gcQ5yicaL0w=="],
"globals": ["globals@17.11.0", "", {}, "sha512-Z2I8hM+PbJDXQDq3Icgpzv+mPdwr68iZUU9d5WW4FuXfDUQfkZaZuvjMv42/5crNyw154+9+VWXbYrUgDXbxNw=="],
"globalthis": ["globalthis@1.0.4", "", { "dependencies": { "define-properties": "^1.2.1", "gopd": "^1.0.1" } }, "sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ=="],
@@ -2253,10 +2252,14 @@
"hast-util-whitespace": ["hast-util-whitespace@3.0.0", "", { "dependencies": { "@types/hast": "^3.0.0" } }, "sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw=="],
"he": ["he@1.2.0", "", { "bin": { "he": "bin/he" } }, "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw=="],
"heap-js": ["heap-js@2.7.1", "", {}, "sha512-EQfezRg0NCZGNlhlDR3Evrw1FVL2G3LhU7EgPoxufQKruNBSYA8MiRPHeWbU+36o+Fhel0wMwM+sLEiBAlNLJA=="],
"help-me": ["help-me@5.0.0", "", {}, "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg=="],
"hls.js": ["hls.js@1.7.3", "", {}, "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A=="],
"hono": ["hono@4.13.5", "", {}, "sha512-O6+/eCYRkzzzy0rPWwKLiGBR1nFuUPZynnwjxN1MBA62NNqbT0wQEzQyK2gSO5yDIDB336sXQleAhOHrzlYyKw=="],
"html-url-attributes": ["html-url-attributes@3.0.1", "", {}, "sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ=="],
@@ -2317,8 +2320,6 @@
"is-fullwidth-code-point": ["is-fullwidth-code-point@3.0.0", "", {}, "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg=="],
"is-function": ["is-function@1.0.2", "", {}, "sha512-lw7DUp0aWXYg+CBCN+JKkcE0Q2RayZnSvnZBlwgxHBQhqt5pZNVy4Ri7H9GmmXkdu7LUthszM+Tor1u/2iBcpQ=="],
"is-generator-function": ["is-generator-function@1.1.2", "", { "dependencies": { "call-bound": "^1.0.4", "generator-function": "^2.0.0", "get-proto": "^1.0.1", "has-tostringtag": "^1.0.2", "safe-regex-test": "^1.1.0" } }, "sha512-upqt1SkGkODW9tsGNG5mtXTXtECizwtS2kA161M+gJPc1xdb/Ax629af6YrTwcOeQHbewrPNlE5Dx7kzvXTizA=="],
"is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="],
@@ -2471,9 +2472,9 @@
"lucide-react": ["lucide-react@1.37.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-LPsB4rD1TD6wZu1djKOf9vUnS1jTNaHbolXebXDgiTdb6jeA1agIJhJsIybCmjKmQClcOaal1o1OaiYahEftyQ=="],
"lz-string": ["lz-string@1.5.0", "", { "bin": { "lz-string": "bin/bin.js" } }, "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ=="],
"luxon": ["luxon@3.7.2", "", {}, "sha512-vtEhXh/gNjI9Yg1u4jX/0YVPMvxzHuGgCm6tC5kZyb08yjGWGnqAjGJvcXbqQR2P3MyMEFnRbpcdFS6PBcLqew=="],
"m3u8-parser": ["m3u8-parser@7.2.0", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.1", "global": "^4.4.0" } }, "sha512-CRatFqpjVtMiMaKXxNvuI3I++vUumIXVVT/JpCpdU/FynV/ceVw1qpPyyBNindL+JlPMSesx+WX1QJaZEJSaMQ=="],
"lz-string": ["lz-string@1.5.0", "", { "bin": { "lz-string": "bin/bin.js" } }, "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ=="],
"magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="],
@@ -2545,8 +2546,6 @@
"mime-match": ["mime-match@1.0.2", "", { "dependencies": { "wildcard": "^1.1.0" } }, "sha512-VXp/ugGDVh3eCLOBCiHZMYWQaTNUHv2IJrut+yXA6+JbLPXHglHwfS/5A5L0ll+jkCY7fIzRJcH6OIunF+c6Cg=="],
"min-document": ["min-document@2.19.2", "", { "dependencies": { "dom-walk": "^0.1.0" } }, "sha512-8S5I8db/uZN8r9HSLFVWPdJCvYOejMcEC82VIzNUc6Zkklf/d1gg2psfE79/vyhWOj4+J8MtwmoOz3TmvaGu5A=="],
"minimatch": ["minimatch@10.2.6", "", { "dependencies": { "brace-expansion": "^5.0.8" } }, "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A=="],
"minimist": ["minimist@1.2.8", "", {}, "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA=="],
@@ -2559,12 +2558,8 @@
"mkdirp-classic": ["mkdirp-classic@0.5.3", "", {}, "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A=="],
"mpd-parser": ["mpd-parser@1.4.0", "", { "dependencies": { "@babel/runtime": "^7.12.5", "@videojs/vhs-utils": "^4.1.2", "@xmldom/xmldom": "^0.8.3", "global": "^4.4.0" }, "bin": { "mpd-to-m3u8-json": "bin/parse.js" } }, "sha512-blbA7XpAaOdC/PR0Tu8GiUxlx6CpBuRowQPYV7lhi9Yr9G9+dkPXIUjqjzc/NCn/uZZPopluaJlIYntIEI/VLw=="],
"ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
"mux.js": ["mux.js@7.1.0", "", { "dependencies": { "@babel/runtime": "^7.11.2", "global": "^4.4.0" }, "bin": { "muxjs-transmux": "bin/transmux.js" } }, "sha512-NTxawK/BBELJrYsZThEulyUMDVlLizKdxyAsMuzoCD1eFj97BVaA8D/CvKsKu6FOLYkFojN5CbM9h++ZTZtknA=="],
"mysql2": ["mysql2@3.15.3", "", { "dependencies": { "aws-ssl-profiles": "^1.1.1", "denque": "^2.1.0", "generate-function": "^2.3.1", "iconv-lite": "^0.7.0", "long": "^5.2.1", "lru.min": "^1.0.0", "named-placeholders": "^1.1.3", "seq-queue": "^0.0.5", "sqlstring": "^2.3.2" } }, "sha512-FBrGau0IXmuqg4haEZRBfHNWB5mUARw6hNwPDXXGg0XzVJ50mr/9hb267lvpVMnhZ1FON3qNd4Xfcez1rbFwSg=="],
"named-placeholders": ["named-placeholders@1.1.6", "", { "dependencies": { "lru.min": "^1.1.0" } }, "sha512-Tz09sEL2EEuv5fFowm419c1+a/jSMiBjI9gHxVLrVdbUkkNUUfjsVYs9pVZu5oCon/kmRh9TfLEObFtkVxmY0w=="],
@@ -2695,8 +2690,6 @@
"pkce-challenge": ["pkce-challenge@5.0.1", "", {}, "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ=="],
"pkcs7": ["pkcs7@1.0.4", "", { "dependencies": { "@babel/runtime": "^7.5.5" }, "bin": { "pkcs7": "bin/cli.js" } }, "sha512-afRERtHn54AlwaF2/+LFszyAANTCggGilmcmILUzEjvs3XgFZT+xE6+QWQcAGmu4xajy+Xtj7acLOPdx5/eXWQ=="],
"pkg-types": ["pkg-types@2.3.1", "", { "dependencies": { "confbox": "^0.2.4", "exsolve": "^1.0.8", "pathe": "^2.0.3" } }, "sha512-y+ichcgc2LrADuhLNAx8DFjVfgz91pRxfZdI3UDhxHvcVEZsenLO+7XaU5vOp0u/7V/wZ+plyuQxtrDlZJ+yeg=="],
"playwright": ["playwright@1.61.1", "", { "dependencies": { "playwright-core": "1.61.1" }, "optionalDependencies": { "fsevents": "2.3.2" }, "bin": { "playwright": "cli.js" } }, "sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ=="],
@@ -3101,14 +3094,6 @@
"victory-vendor": ["victory-vendor@37.3.6", "", { "dependencies": { "@types/d3-array": "^3.0.3", "@types/d3-ease": "^3.0.0", "@types/d3-interpolate": "^3.0.1", "@types/d3-scale": "^4.0.2", "@types/d3-shape": "^3.1.0", "@types/d3-time": "^3.0.0", "@types/d3-timer": "^3.0.0", "d3-array": "^3.1.6", "d3-ease": "^3.0.1", "d3-interpolate": "^3.0.1", "d3-scale": "^4.0.2", "d3-shape": "^3.1.0", "d3-time": "^3.0.0", "d3-timer": "^3.0.1" } }, "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ=="],
"video.js": ["video.js@8.24.0", "", { "dependencies": { "@babel/runtime": "^7.28.4", "@videojs/http-streaming": "^3.17.5", "@videojs/vhs-utils": "^4.1.2", "@videojs/xhr": "2.7.0", "aes-decrypter": "^4.0.2", "global": "4.4.0", "m3u8-parser": "^7.2.0", "mpd-parser": "^1.4.0", "mux.js": "^7.0.1", "videojs-contrib-quality-levels": "4.1.0", "videojs-font": "4.2.0", "videojs-vtt.js": "0.15.5" } }, "sha512-ZUuxTlDRkEWhpp2mvGCSFzI0Tc3zcC6IUZbIhBcZ3QW0Ms+IRp/tzYkhqexcMYwrcukRFiWWJmRANsLKvv8Vsg=="],
"videojs-contrib-quality-levels": ["videojs-contrib-quality-levels@4.1.0", "", { "dependencies": { "global": "^4.4.0" }, "peerDependencies": { "video.js": "^8" } }, "sha512-TfrXJJg1Bv4t6TOCMEVMwF/CoS8iENYsWNKip8zfhB5kTcegiFYezEA0eHAJPU64ZC8NQbxQgOwAsYU8VXbOWA=="],
"videojs-font": ["videojs-font@4.2.0", "", {}, "sha512-YPq+wiKoGy2/M7ccjmlvwi58z2xsykkkfNMyIg4xb7EZQQNwB71hcSsB3o75CqQV7/y5lXkXhI/rsGAS7jfEmQ=="],
"videojs-vtt.js": ["videojs-vtt.js@0.15.5", "", { "dependencies": { "global": "^4.3.1" } }, "sha512-yZbBxvA7QMYn15Lr/ZfhhLPrNpI/RmCSCqgIff57GC2gIrV5YfyzLfLyZMj0NnZSAz8syB4N0nHXpZg9MyrMOQ=="],
"vite": ["vite@8.2.2", "", { "dependencies": { "lightningcss": "^1.33.0", "picomatch": "^4.0.5", "postcss": "^8.5.26", "rolldown": "~1.2.4", "tinyglobby": "^0.2.17" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.4.0 || ^0.5.0", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q=="],
"vite-tsconfig-paths": ["vite-tsconfig-paths@6.1.1", "", { "dependencies": { "debug": "^4.1.1", "globrex": "^0.1.2", "tsconfck": "^3.0.3" }, "peerDependencies": { "vite": "*" } }, "sha512-2cihq7zliibCCZ8P9cKJrQBkfgdvcFkOOc3Y02o3GWUDLgqjWsZudaoiuOwO/gzTzy17cS5F7ZPo4bsnS4DGkg=="],
+39
View File
@@ -3,6 +3,45 @@ title: "Changelog"
description: "Recent updates, new features, and improvements in Shumai"
---
<Update label="v0.5.1" description="October 6, 2026">
### Added
* **notification**: Notification cards are now clickable, taking you directly to the relevant file
* **transcode**: Add hardware decoding and GPU scaling support for NVIDIA NVENC, Intel QuickSync Video (QSV), Rockchip (RKMPP), and VA-API (Intel and AMD GPUs), significantly speeding up video proxy and HLS rendition generation across supported platforms. Set `SHUMAI_HW_DECODE=false` to turn hardware decoding off
* **transcode**: Add `dcraw_emu` fallback to generate WebP previews for camera RAW files without embedded JPEG previews
### Fixed
### Changed
</Update>
<Update label="v0.5.0" description="October 2, 2026">
### Added
* **transcode**: Add configurable target resolution ladders (480p, 720p, 1080p, 1440p, 2160p) for MP4 Multi Resolutions transcoding in team settings, with unified resolution matching and lowest-resolution fallback for both MP4 and HLS
* **transcode**: Add HLS live streaming with fMP4/CMAF adaptive bitrate ladders (480p, 720p, 1080p, 1440p, 2160p), configurable in Transcode Settings, with automatic quality switching, manual resolution selection in the video viewer, seamless rendition switching, and full watermarking support for shared links
* **transcode**: Add basic camera RAW image support for popular camera formats (including Sony ARW, Canon CR2/CR3, Nikon NEF, Adobe DNG, Fujifilm RAF, and Panasonic RW2), extracting high-quality embedded JPEG previews and automatically applying camera orientation for fast thumbnail and proxy generation
### Fixed
- **asset**: Fix an issue where media files remained stuck in a perpetual "Preparing..." state when transcoding failed, by displaying a clear failed status and error details in the file list and file viewer
* **transcode**: Fix an issue where videos or audio-only files containing attached cover art or picture streams could cause transcoding to fail during poster generation or select a single static image instead of the video
* **webui**: Fix inconsistent video resolution labels across download menus, mobile file headers, and the video player by correctly deriving resolutions from the video's short side, including portrait videos and 480p videos
* **trash**: Fix an issue where the "Empty Trash" confirmation dialog would close immediately instead of showing a loading state while files were being permanently deleted
### Changed
* **settings**: Redesign team Video Transcoding settings with dedicated cards for Transcode Policy, Hardware Acceleration, and FFmpeg Threads, clearly explaining MP4 and HLS playback vs download behavior and storage implications
* **settings**: Rename AI Settings group to Team AI Settings in settings navigation
</Update>
<Update label="v0.4.9" description="September 28, 2026">
### Added
@@ -55,6 +55,20 @@ Shumai includes a built-in usability probe that tests hardware encoders at start
You should see `card0` and `renderD128` (or similar).
2. Shumai's Docker image automatically adjusts group permissions on container startup so the non-root application user can access host `/dev/dri` devices without permission errors.
#### GPU decoding
With VA-API, Shumai decodes and resizes video on the GPU as well as encoding it, so frames never leave GPU memory. This is much faster than decoding on the CPU, especially for 4K sources on integrated graphics. (Other backends currently decode on the CPU and encode on the GPU.)
Some videos automatically use CPU decoding with GPU encoding instead:
- Codecs your GPU cannot decode, such as HEVC or 10-bit video on older Intel chips. Shumai tries the GPU first and retries on the CPU within a second.
- Rotated videos, such as portrait phone recordings, so they keep the correct orientation.
- Watermarked share links and HDR sources.
If a GPU-decode attempt stops making progress for 2 minutes (some driver bugs make ffmpeg hang instead of failing), Shumai stops it and retries on the CPU. Change the limit with `SHUMAI_HW_DECODE_STALL_TIMEOUT` (in seconds).
To always decode on the CPU, set `SHUMAI_HW_DECODE=false` on the `shumai` service.
### WSL2 (Windows)
- Ensure you are running Windows 11 with WSL2 and have installed the appropriate GPU drivers on the Windows host.
+3
View File
@@ -44,6 +44,9 @@ To handle media processing and security isolation, the host machine must have th
- **FFmpeg & FFprobe**: Required for video transcoding and metadata extraction.
- _macOS_: `brew install ffmpeg`
- _Linux (Ubuntu/Debian)_: `sudo apt-get install ffmpeg`
- **dcraw_emu (LibRaw)**: Required for camera RAW image decoding and preview generation.
- _macOS_: `brew install libraw`
- _Linux (Ubuntu/Debian)_: `sudo apt-get install libraw-bin`
- **Agent Sandbox Utilities**: Used to run custom automation scripts securely.
- _macOS_: `brew install ripgrep`
- _Linux (Ubuntu/Debian)_: `sudo apt-get install bubblewrap socat ripgrep`
+61 -21
View File
@@ -1,31 +1,77 @@
---
title: Media Transcoding Settings
description: Configure video transcode strategies and preview proxy generation.
title: Video Transcoding
description: Configure video transcode policies, MP4 and HLS streaming resolutions, hardware acceleration, and FFmpeg threads.
---
High-resolution camera formats and raw media files cannot be rendered natively in web browsers. Shumai transcodes these assets into web-friendly streaming formats in the background.
High-resolution camera formats and raw media files cannot be rendered natively in web browsers. Shumai transcodes video assets into web-friendly streaming and preview formats in the background.
Team Owners can customize video transcoding behavior from the **Team Settings > Video Transcoding** page.
Regardless of the selected transcoding strategy, Shumai's pipeline will **always** generate at least one lightweight `180p` proxy version of the raw file to ensure fast scrubbing, thumbnail generation, and instant previews.
---
## Video Transcode Strategies
## Transcode Policy
Team Owners can select how the transcoding engine handles uploaded files. Shumai evaluates the original file's dimensions against its standard target resolutions (2160p, 1080p, 720p, 540p, and 360p) to determine the output.
The **Transcode Policy** card configures both MP4 video transcoding and HLS adaptive streaming.
### 1. Best Match
### Playback and Download Behavior
- **Behavior**: Shumai checks the longest dimension (width or height) of the uploaded file. It then finds the highest standard resolution that is equal to or smaller than the original file, preventing unnecessary upscaling. If the original video is exceptionally small, it defaults to a minimum target of `360p`.
- **Output**: Generates exactly two web-optimized files: the universal `180p` proxy and the single calculated "best match" resolution.
- **Use Case**: This is the recommended option. It ensures web compatibility without wasting processing time and storage space on redundant lower-resolution versions.
Shumai uses a dual-format strategy for video delivery:
### 2. Transcode All
- **When HLS is disabled**: Both in-browser video playback and file downloads will use the generated MP4 file(s).
- **When HLS is enabled**: In-browser video playback uses HLS (HTTP Live Streaming) for smooth, adaptive bitrate playback, while file downloads will use the generated MP4 file(s).
- **Behavior**: Shumai calculates the "best match" resolution exactly as it does in the previous strategy. However, instead of stopping at one version, it processes the video into the best match _and_ every standard resolution smaller than it.
- **Output**: Generates a full cascade of files. For example, if the best match for an upload is `1080p`, Shumai will generate the `180p` proxy, `1080p`, `720p`, `540p`, and `360p` versions.
- **Use Case**: Ideal for environments requiring adaptive bitrate streaming, ensuring consistent playback performance across all client devices regardless of varying network conditions.
<Warning>
**Storage Usage Notice**: Enabling HLS generates multi-bitrate segment files (`.ts` / `.m4s`) and
playlist manifests in addition to standard MP4 proxies. This can significantly increase storage
consumption on your storage bucket.
</Warning>
![Transcode Settings](../transcode.webp)
### MP4 Transcoding
Generate web-compatible MP4 files for file downloads and standard video playback. Team Owners can select how MP4 transcodes are generated for uploaded files:
#### 1. Best Match
- **Behavior**: Shumai checks the longest dimension of the uploaded video and generates a single optimal resolution matching or smaller than the source quality, preventing unnecessary upscaling.
- **Output**: Generates the lightweight `180p` proxy and the calculated best-match MP4 file.
- **Use Case**: Recommended default. Minimizes processing time and storage space while ensuring universal playback and download compatibility.
#### 2. Multi Resolutions
- **Behavior**: Select specific target resolution ladders to generate: `480p`, `720p`, `1080p`, `1440p`, and `2160p`. Resolutions are only generated up to the source video resolution (content is never upscaled).
- **Output**: Generates the `180p` proxy plus all selected resolution ladders that do not exceed the source resolution.
- **Use Case**: Ideal when users need to download multiple specific resolutions (e.g. 720p for fast sharing and 1080p for review).
### HLS Streaming
When **HLS Streaming** is toggled on, Shumai produces an adaptive bitrate HLS package for uploaded videos:
- **Resolution Ladders**: Configure which resolutions to generate for HLS streaming (`480p`, `720p`, `1080p`, `1440p`, `2160p`). Ladders are only generated up to the original video resolution.
- **Player Integration**: The web player will automatically stream via HLS when available, dynamically switching bitrates based on client network bandwidth.
![Video Transcoding](../transcode.webp)
---
## Hardware Acceleration
To offload encoding workloads from CPUs to GPUs, Shumai supports hardware-accelerated video encoding:
- **Disabled (Software)**: Uses the standard CPU `libx264` software encoder with Constant Rate Factor (CRF).
- **Auto Detect**: Automatically probes and leverages available GPU/hardware acceleration (NVIDIA NVENC, Intel QuickSync, AMD/Intel VA-API, Apple VideoToolbox, Rockchip RKMPP).
See the [Hardware Transcoding Guide](/configuration/hardware-transcoding) for Docker Compose configuration, GPU passthrough, and driver prerequisites.
---
## FFmpeg Threads
The **FFmpeg Threads** slider controls the maximum number of CPU threads used by FFmpeg during transcoding tasks:
- **Auto (0)**: FFmpeg automatically selects the optimal thread count based on host CPU core availability.
- **Manual (1 - 32)**: Restricts FFmpeg to a fixed number of threads, useful for throttling CPU consumption on shared worker nodes.
---
@@ -33,10 +79,4 @@ Team Owners can select how the transcoding engine handles uploaded files. Shumai
Transcoding is resource-intensive. Depending on your scaling needs, your administrator can run the transcode engine on the main server (Single-Server mode) or delegate tasks to specialized transcoding worker machines (Temporal Worker mode).
Note: In all modes, the processing machine must have the standard system media utilities (`ffmpeg` and `ffprobe`) installed.
---
## Hardware Acceleration
To offload encoding workloads to GPUs (NVIDIA NVENC, Intel QuickSync, AMD/Intel VA-API, or Rockchip RKMPP), Shumai supports hardware-accelerated transcoding. See the [Hardware Transcoding Guide](/configuration/hardware-transcoding) for Docker Compose configuration, GPU passthrough, and driver prerequisites.
In all modes, the processing machine must have system media utilities (`ffmpeg` and `ffprobe`) installed. See the [Temporal Deployment Guide](/configuration/temporal) for setting up distributed transcoding workers.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"repository": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+4
View File
@@ -154,6 +154,10 @@ export async function generateEmbeddingActivity(params: GenerateEmbeddingParams)
start.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-map',
'0:a:0?',
'-t',
(end - start).toString(),
'-c',
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/api",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+43
View File
@@ -774,4 +774,47 @@ describe('file api', () => {
mockGetProjectIds.mockRestore()
})
describe('GET /files/:fileId/m3u8 routes', () => {
it('returns master playlist with correct content-type', async () => {
const mockMaster = vi
.spyOn(assetService, 'getHlsMasterPlaylist')
.mockResolvedValue('#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n')
const app = new Hono().use('*', authMiddleware).route('/', fileRoute)
const res = await app.request('/files/asset-1/m3u8/master.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('#EXTM3U')
expect(text).toContain('1080p/index.m3u8')
expect(mockMaster).toHaveBeenCalledWith({ assetId: 'asset-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user1', name: 'Test User' },
permission: Permission.Read,
type: 'asset',
id: 'asset-1',
})
mockMaster.mockRestore()
})
it('returns variant playlist with rewritten URLs', async () => {
const mockVariant = vi
.spyOn(assetService, 'getHlsVariantPlaylist')
.mockResolvedValue(
'#EXTM3U\n#EXT-X-MAP:URI="http://presigned/init.mp4"\nhttp://presigned/seg0.m4s\n',
)
const app = new Hono().use('*', authMiddleware).route('/', fileRoute)
const res = await app.request('/files/asset-1/m3u8/1080p/index.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('http://presigned/init.mp4')
expect(mockVariant).toHaveBeenCalledWith({ assetId: 'asset-1', resolution: '1080p' })
mockVariant.mockRestore()
})
})
})
+65
View File
@@ -480,5 +480,70 @@ const route = new Hono<{ Variables: { user: User } }>()
return c.json({ url })
},
)
.get('/files/:fileId/m3u8', async (c) => {
const fileId = c.req.param('fileId')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsMasterPlaylist({ assetId: fileId })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/master.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsMasterPlaylist({ assetId: fileId })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/:resolution', async (c) => {
const fileId = c.req.param('fileId')
let resolution = c.req.param('resolution')
if (resolution.endsWith('.m3u8')) {
resolution = resolution.replace(/\.m3u8$/, '')
}
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsVariantPlaylist({ assetId: fileId, resolution })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
.get('/files/:fileId/m3u8/:resolution/index.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const resolution = c.req.param('resolution')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: fileId,
})
const playlist = await assetService.getHlsVariantPlaylist({ assetId: fileId, resolution })
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
})
export default route
+129 -6
View File
@@ -38,6 +38,8 @@ import type { AssetInfo } from '@shumai/dtos'
async function applyWatermarkToAssetMedia(
asset: AssetInfo,
watermarkMedia: PrismaJson.MediaInfo | null,
shareId?: string,
password?: string,
): Promise<AssetInfo> {
if (!asset.media) return asset
if (asset.proxyType !== 'video' && asset.proxyType !== 'image') return asset
@@ -74,6 +76,19 @@ async function applyWatermarkToAssetMedia(
)
: []
if (watermarkMedia?.isHls && watermarkMedia.hls?.key && shareId) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
updatedMedia.isHls = true
updatedMedia.hls = {
key: watermarkMedia.hls.key,
url: `/api/shares/${shareId}/files/${asset.id}/m3u8/master.m3u8${pwQuery}`,
resolutions: watermarkMedia.hls.resolutions,
}
} else {
updatedMedia.isHls = false
delete updatedMedia.hls
}
return {
...asset,
media: updatedMedia,
@@ -170,7 +185,8 @@ const route = app
zValidator('query', sharedChildrenRequestSchema),
async (c) => {
const folderId = c.req.param('folderId')
const password = c.req.header('x-share-password')
const password =
c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
const req = c.req.valid('query')
try {
@@ -197,7 +213,7 @@ const route = app
// Resolve symlink targets once and reuse for the media lookup, so each
// item costs a single query instead of two.
const targetIds = await Promise.all(
res.data.map((item) => assetService.resolveTargetAssetId(item.id)),
res.data.map((item) => assetService.resolveLatestVersionId(item.id)),
)
const wfMap = await watermarkService.getCompletedWatermarkMediaMap(
targetIds,
@@ -206,9 +222,21 @@ const route = app
res.data = await Promise.all(
res.data.map(async (item, index) =>
applyWatermarkToAssetMedia(item, wfMap.get(targetIds[index]) ?? null),
applyWatermarkToAssetMedia(
item,
wfMap.get(targetIds[index]) ?? null,
shareLink.id,
password,
),
),
)
} else if (res.data.length > 0) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
for (const item of res.data) {
if (item.media?.isHls && item.media.hls?.key) {
item.media.hls.url = `/api/shares/${shareLink.id}/files/${item.id}/m3u8/master.m3u8${pwQuery}`
}
}
}
return c.json(res)
@@ -219,7 +247,7 @@ const route = app
)
.get('/shares/:shareId/files/:fileId', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
@@ -232,12 +260,20 @@ const route = app
shareLink.watermarkConfigId &&
(asset.proxyType === 'video' || asset.proxyType === 'image')
) {
const targetAssetId = await assetService.resolveTargetAssetId(fileId)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const wfMap = await watermarkService.getCompletedWatermarkMediaMap(
[targetAssetId],
shareLink.watermarkConfigId,
)
asset = await applyWatermarkToAssetMedia(asset, wfMap.get(targetAssetId) ?? null)
asset = await applyWatermarkToAssetMedia(
asset,
wfMap.get(targetAssetId) ?? null,
shareLink.id,
password,
)
} else if (asset.media?.isHls && asset.media.hls?.key) {
const pwQuery = password ? `?p=${encodeURIComponent(password)}` : ''
asset.media.hls.url = `/api/shares/${shareLink.id}/files/${asset.id}/m3u8/master.m3u8${pwQuery}`
}
return c.json(asset)
@@ -360,5 +396,92 @@ const route = app
}
},
)
.get('/shares/:shareId/files/:fileId/m3u8', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
let playlist = await assetService.getHlsMasterPlaylist({
assetId: targetAssetId,
watermarkConfigId: shareLink.watermarkConfigId,
})
if (password) {
playlist = playlist.replace(
/([0-9]+p\/index\.m3u8)/g,
`$1?p=${encodeURIComponent(password)}`,
)
}
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/master.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
let playlist = await assetService.getHlsMasterPlaylist({
assetId: targetAssetId,
watermarkConfigId: shareLink.watermarkConfigId,
})
if (password) {
playlist = playlist.replace(
/([0-9]+p\/index\.m3u8)/g,
`$1?p=${encodeURIComponent(password)}`,
)
}
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/:resolution', async (c) => {
const fileId = c.req.param('fileId')
let resolution = c.req.param('resolution')
if (resolution.endsWith('.m3u8')) {
resolution = resolution.replace(/\.m3u8$/, '')
}
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const playlist = await assetService.getHlsVariantPlaylist({
assetId: targetAssetId,
resolution,
watermarkConfigId: shareLink.watermarkConfigId,
})
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
.get('/shares/:shareId/files/:fileId/m3u8/:resolution/index.m3u8', async (c) => {
const fileId = c.req.param('fileId')
const resolution = c.req.param('resolution')
const password = c.req.header('x-share-password') || c.req.query('password') || c.req.query('p')
try {
const shareLink = await shareService.verifyPublicAccess(fileId, password)
const targetAssetId = await assetService.resolveLatestVersionId(fileId)
const playlist = await assetService.getHlsVariantPlaylist({
assetId: targetAssetId,
resolution,
watermarkConfigId: shareLink.watermarkConfigId,
})
c.header('Content-Type', 'application/vnd.apple.mpegurl')
return c.body(playlist)
} catch (err) {
return handlePublicShareError(c, err)
}
})
export default route
+113 -1
View File
@@ -277,6 +277,12 @@ describe('Share API', () => {
},
],
imageTranscodes: [],
isHls: true,
hls: {
key: 'files/orig-hls/master.m3u8',
url: '/api/files/video-pending/m3u8/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
original: { key: 'files/orig.mp4' },
},
},
@@ -331,9 +337,11 @@ describe('Share API', () => {
size: 0,
},
])
// Pending item → original transcodes emptied
// Pending item → original transcodes emptied and HLS cleared
expect(body.data[1].media.videoTranscodes).toEqual([])
expect(body.data[1].media.imageTranscodes).toEqual([])
expect(body.data[1].media.isHls).toBe(false)
expect(body.data[1].media.hls).toBeUndefined()
// Non-media item → untouched
expect(body.data[2].media.pdfTranscode).toEqual({ url: 'u', key: 'files/doc.pdf' })
})
@@ -478,6 +486,33 @@ describe('Share API', () => {
expect(body.media.imageTranscodes).toEqual([])
})
test('clears HLS fields while watermark transcoding is in flight', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue(shareLinkWithWatermark)
vi.spyOn(watermarkService, 'getCompletedWatermarkMediaMap').mockResolvedValue(new Map())
vi.spyOn(assetService, 'getAsset').mockResolvedValue({
...videoAsset,
media: {
...videoAsset.media,
isHls: true,
hls: {
key: 'files/hls/master.m3u8',
url: '/api/files/file1/m3u8/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
},
})
const res = await app.request('/shares/share1/files/file1', {
method: 'GET',
headers: { 'x-share-password': 'pass' },
})
expect(res.status).toBe(200)
const body = await res.json()
expect(body.media.isHls).toBe(false)
expect(body.media.hls).toBeUndefined()
})
test('leaves non-video/image media untouched when watermark enabled', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue(shareLinkWithWatermark)
vi.spyOn(assetService, 'getAsset').mockResolvedValue({
@@ -866,4 +901,81 @@ describe('Share API', () => {
expect(getDownloadUrlSpy).not.toHaveBeenCalled()
})
})
describe('GET /shares/:shareId/files/:fileId/m3u8 routes', () => {
test('returns master playlist with query password preserved in variant URLs', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: null,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'getHlsMasterPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n',
)
const res = await app.request('/shares/share1/files/file1/m3u8/master.m3u8?p=secret123')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('1080p/index.m3u8?p=secret123')
expect(shareService.verifyPublicAccess).toHaveBeenCalledWith('file1', 'secret123')
expect(assetService.getHlsMasterPlaylist).toHaveBeenCalledWith({
assetId: 'file1',
watermarkConfigId: null,
})
})
test('returns variant playlist with rewritten segment URLs', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: 'wm1',
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'getHlsVariantPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-MAP:URI="http://presigned/init.mp4"\nhttp://presigned/seg0.m4s\n',
)
const res = await app.request('/shares/share1/files/file1/m3u8/1080p/index.m3u8')
expect(res.status).toBe(200)
expect(res.headers.get('content-type')).toContain('application/vnd.apple.mpegurl')
const text = await res.text()
expect(text).toContain('http://presigned/init.mp4')
expect(assetService.getHlsVariantPlaylist).toHaveBeenCalledWith({
assetId: 'file1',
resolution: '1080p',
watermarkConfigId: 'wm1',
})
})
test('returns 401 when password is invalid', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockRejectedValue(
new ShareLinkPasswordInvalidError('Invalid password'),
)
const res = await app.request('/shares/share1/files/file1/m3u8/master.m3u8?p=wrong')
expect(res.status).toBe(401)
const body = await res.json()
expect(body.error).toBe('Unauthorized')
})
test('resolves latest version id when fileId is a symlink to a version stack', async () => {
vi.spyOn(shareService, 'verifyPublicAccess').mockResolvedValue({
id: 'share1',
watermarkConfigId: null,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.spyOn(assetService, 'resolveLatestVersionId').mockResolvedValue('child-version-file-id')
vi.spyOn(assetService, 'getHlsMasterPlaylist').mockResolvedValue(
'#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n',
)
const res = await app.request('/shares/share1/files/symlink1/m3u8/master.m3u8')
expect(res.status).toBe(200)
expect(assetService.resolveLatestVersionId).toHaveBeenCalledWith('symlink1')
expect(assetService.getHlsMasterPlaylist).toHaveBeenCalledWith({
assetId: 'child-version-file-id',
watermarkConfigId: null,
})
})
})
})
+27 -4
View File
@@ -245,17 +245,17 @@ describe('team api', () => {
})
it('PATCH /teams/:teamId/settings updates settings', async () => {
mockUpdateSettings.mockResolvedValue({ transcode: { videoStrategy: 'all' } })
mockUpdateSettings.mockResolvedValue({ transcode: { videoStrategy: 'multi' } })
const res = await app.request('/teams/t1/settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ key: 'transcode.videoStrategy', value: 'all' }),
body: JSON.stringify({ key: 'transcode.videoStrategy', value: 'multi' }),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.transcode.videoStrategy).toBe('all')
expect(data.transcode.videoStrategy).toBe('multi')
expect(authzService.hasPermission).toHaveBeenCalledWith(
expect.objectContaining({
type: ResourceType.Team,
@@ -263,7 +263,30 @@ describe('team api', () => {
permission: Permission.Admin,
}),
)
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoStrategy', 'all')
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoStrategy', 'multi')
})
it('PATCH /teams/:teamId/settings updates transcode.videoResolutions', async () => {
mockUpdateSettings.mockResolvedValue({
transcode: { videoResolutions: ['720p', '1080p'] },
})
const res = await app.request('/teams/t1/settings', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
key: 'transcode.videoResolutions',
value: ['720p', '1080p'],
}),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.transcode.videoResolutions).toEqual(['720p', '1080p'])
expect(mockUpdateSettings).toHaveBeenCalledWith('t1', 'transcode.videoResolutions', [
'720p',
'1080p',
])
})
it('PATCH /teams/:teamId/settings updates transcode.hardwareAcceleration', async () => {
+2 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/core",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -42,6 +42,7 @@
"date-fns": "4.4.0",
"dotenv": "17.4.2",
"dotenv-expand": "13.0.0",
"exiftool-vendored": "38.1.0",
"extract-zip": "2.0.1",
"hono": "4.13.5",
"jittered-fractional-indexing": "1.0.1",
+108
View File
@@ -16,6 +16,7 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
presign: vi.fn().mockResolvedValue('http://mock-s3-url'),
putObject: vi.fn().mockResolvedValue(undefined),
getObject: vi.fn().mockResolvedValue({ buffer: Buffer.from('') }),
deleteObject: vi.fn().mockResolvedValue(1),
deletePrefix: vi.fn().mockResolvedValue(1),
copyObject: vi.fn().mockResolvedValue(undefined),
@@ -5251,4 +5252,111 @@ describe('AssetService — natural sort by name', () => {
expect(restored?.status).toBe(AssetStatus.processed)
})
})
describe('HLS playlists', () => {
const setupHlsAssets = async () => {
const team = await prisma.team.create({ data: { name: 'hls-team-' + Date.now() } })
const project = await prisma.project.create({
data: { name: 'hls-proj-' + Date.now(), teamId: team.id },
})
const rootFolder = await prisma.asset.create({
data: {
name: 'root',
type: AssetType.folder,
projectId: project.id,
status: AssetStatus.processed,
},
})
return { team, project, rootFolder }
}
it('getHlsMasterPlaylist throws 404 if asset is not HLS', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'regular.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: false,
} as PrismaJson.MediaInfo,
},
})
await expect(assetService.getHlsMasterPlaylist({ assetId: file.id })).rejects.toThrow(
'HLS playlist not found',
)
})
it('getHlsMasterPlaylist returns playlist string when asset is HLS', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'hls-video.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [
{ resolution: '1080p', width: 1920, height: 1080 },
{ resolution: '720p', width: 1280, height: 720 },
],
},
} as PrismaJson.MediaInfo,
},
})
const masterContent = '#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4628000\n1080p/index.m3u8\n'
vi.mocked(s3Service.getObject).mockResolvedValueOnce({
buffer: Buffer.from(masterContent),
} as unknown as Awaited<ReturnType<typeof s3Service.getObject>>)
const res = await assetService.getHlsMasterPlaylist({ assetId: file.id })
expect(res).toBe(masterContent)
expect(s3Service.getObject).toHaveBeenCalledWith(
expect.any(String),
'files/asset/hls/master.m3u8',
)
})
it('getHlsVariantPlaylist fetches and rewrites segment URLs', async () => {
const { project, rootFolder } = await setupHlsAssets()
const file = await prisma.asset.create({
data: {
name: 'hls-video.mp4',
type: AssetType.file,
projectId: project.id,
parentId: rootFolder.id,
status: AssetStatus.processed,
media: {
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
} as PrismaJson.MediaInfo,
},
})
const sampleVariant = '#EXTM3U\n#EXT-X-MAP:URI="init.mp4"\nsegment_000.m4s\n'
vi.mocked(s3Service.getObject).mockResolvedValueOnce({
buffer: Buffer.from(sampleVariant),
} as unknown as Awaited<ReturnType<typeof s3Service.getObject>>)
vi.mocked(s3Service.presign).mockResolvedValue('http://mock-s3-url')
const variant = await assetService.getHlsVariantPlaylist({
assetId: file.id,
resolution: '1080p',
})
expect(variant).toBeDefined()
expect(variant).toContain('#EXT-X-MAP:URI="http://mock-s3-url"')
expect(variant).toContain('http://mock-s3-url')
})
})
})
+93
View File
@@ -57,6 +57,7 @@ import { metadataService } from '@shumai/core/src/metadata/metadata'
import { uploadService } from '@shumai/core/src/upload/upload'
import path from 'path'
import { stemFromKey } from '@shumai/core/src/utils/filename'
import { rewriteM3u8WithPresignedUrls } from '@shumai/core/src/transcode/transcode'
export const assetInclude = {
creator: true,
@@ -2386,6 +2387,9 @@ export class AssetService {
'GET',
)
}
if (media && media.isHls && media.hls?.key) {
media.hls.url = `/api/files/${latestVersion.id}/m3u8/master.m3u8`
}
const key = latestVersion.storageKey?.key
if (key) {
@@ -3196,6 +3200,95 @@ export class AssetService {
req,
)
}
async getHlsMasterPlaylist(params: {
assetId: string
watermarkConfigId?: string | null
}): Promise<string> {
const bucket = process.env.S3_BUCKET || 'shumai'
const targetAssetId = await this.resolveLatestVersionId(params.assetId)
if (params.watermarkConfigId) {
const wf = await this.prismaClient.watermarkFile.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_watermarkConfigId: {
assetId: targetAssetId,
watermarkConfigId: params.watermarkConfigId,
},
},
})
const media = wf?.media as PrismaJson.MediaInfo | null
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const obj = await s3Service.getObject(bucket, media.hls.key)
return obj.buffer.toString('utf-8')
}
const asset = await this.prismaClient.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new HTTPException(404, { message: 'Asset not found' })
}
const media = asset.media as PrismaJson.MediaInfo | null
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const obj = await s3Service.getObject(bucket, media.hls.key)
return obj.buffer.toString('utf-8')
}
async getHlsVariantPlaylist(params: {
assetId: string
resolution: string
watermarkConfigId?: string | null
}): Promise<string> {
const bucket = process.env.S3_BUCKET || 'shumai'
if (!/^\d+p$/.test(params.resolution)) {
throw new HTTPException(400, { message: 'Invalid resolution format' })
}
const targetAssetId = await this.resolveLatestVersionId(params.assetId)
let media: PrismaJson.MediaInfo | null
if (params.watermarkConfigId) {
const wf = await this.prismaClient.watermarkFile.findUnique({
where: {
// eslint-disable-next-line @typescript-eslint/naming-convention
assetId_watermarkConfigId: {
assetId: targetAssetId,
watermarkConfigId: params.watermarkConfigId,
},
},
})
media = (wf?.media as PrismaJson.MediaInfo | null) ?? null
} else {
const asset = await this.prismaClient.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new HTTPException(404, { message: 'Asset not found' })
}
media = (asset.media as PrismaJson.MediaInfo | null) ?? null
}
if (!media?.isHls || !media.hls?.key) {
throw new HTTPException(404, { message: 'HLS playlist not found' })
}
const matchedRes = media.hls.resolutions?.find((r) => r.resolution === params.resolution)
const variantKey =
matchedRes?.key ||
path.posix.join(path.posix.dirname(media.hls.key), params.resolution, 'index.m3u8')
const obj = await s3Service.getObject(bucket, variantKey)
const rawM3u8 = obj.buffer.toString('utf-8')
const s3Prefix = path.posix.dirname(variantKey)
return await rewriteM3u8WithPresignedUrls(rawM3u8, s3Prefix, bucket)
}
}
export const assetService = new AssetService()
+18 -2
View File
@@ -369,8 +369,11 @@ describe('TeamService', () => {
semanticSearchEnabled: false,
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
},
appearance: { hideAgent: false },
})
@@ -383,24 +386,37 @@ describe('TeamService', () => {
semanticSearchEnabled: false,
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
},
appearance: { hideAgent: false },
})
await teamService.updateSettings(team.id, 'transcode.videoStrategy', 'all')
await teamService.updateSettings(team.id, 'transcode.videoStrategy', 'multi')
await teamService.updateSettings(team.id, 'transcode.videoResolutions', ['720p', '1080p'])
await teamService.updateSettings(team.id, 'transcode.hardwareAcceleration', 'auto')
await teamService.updateSettings(team.id, 'transcode.threads', 4)
await teamService.updateSettings(team.id, 'transcode.hlsEnabled', true)
await teamService.updateSettings(team.id, 'transcode.hlsResolutions', [
'720p',
'1080p',
'2160p',
])
await teamService.updateSettings(team.id, 'appearance.hideAgent', true)
const finalSettings = await teamService.getSettings(team.id)
expect(finalSettings).toEqual({
theme: 'dark',
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 4,
hlsEnabled: true,
hlsResolutions: ['720p', '1080p', '2160p'],
},
appearance: { hideAgent: true },
semanticSearchEnabled: false,
+50 -1
View File
@@ -35,6 +35,7 @@ export class TeamService {
settings: {
transcode: {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
},
@@ -312,8 +313,27 @@ export class TeamService {
if (!settings.transcode) {
settings.transcode = {
videoStrategy: 'best_match',
videoResolutions: ['480p', '720p', '1080p', '1440p', '2160p'],
hardwareAcceleration: 'off',
threads: 0,
hlsEnabled: false,
hlsResolutions: ['480p', '720p', '1080p'],
}
} else {
const strategyStr = settings.transcode.videoStrategy as string | undefined
if (strategyStr === 'single' || strategyStr === 'disable') {
settings.transcode.videoStrategy = 'best_match'
} else if (strategyStr === 'full' || strategyStr === 'all') {
settings.transcode.videoStrategy = 'multi'
}
if (!settings.transcode.videoResolutions) {
settings.transcode.videoResolutions = ['480p', '720p', '1080p', '1440p', '2160p']
}
if (settings.transcode.hlsEnabled === undefined) {
settings.transcode.hlsEnabled = false
}
if (!settings.transcode.hlsResolutions) {
settings.transcode.hlsResolutions = ['480p', '720p', '1080p']
}
}
@@ -336,10 +356,25 @@ export class TeamService {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.videoStrategy = value as NonNullable<
const rawStrategy = value as string
let strategy = rawStrategy
if (rawStrategy === 'single' || rawStrategy === 'disable') {
strategy = 'best_match'
} else if (rawStrategy === 'full' || rawStrategy === 'all') {
strategy = 'multi'
}
settings.transcode.videoStrategy = strategy as NonNullable<
TeamSettingsResponse['transcode']
>['videoStrategy']
delete settings['transcode.videoStrategy']
} else if (key === 'transcode.videoResolutions') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.videoResolutions = value as NonNullable<
TeamSettingsResponse['transcode']
>['videoResolutions']
delete settings['transcode.videoResolutions']
} else if (key === 'transcode.hardwareAcceleration') {
if (!settings.transcode) {
settings.transcode = {}
@@ -354,6 +389,20 @@ export class TeamService {
}
settings.transcode.threads = Number(value)
delete settings['transcode.threads']
} else if (key === 'transcode.hlsEnabled') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.hlsEnabled = Boolean(value)
delete settings['transcode.hlsEnabled']
} else if (key === 'transcode.hlsResolutions') {
if (!settings.transcode) {
settings.transcode = {}
}
settings.transcode.hlsResolutions = value as NonNullable<
TeamSettingsResponse['transcode']
>['hlsResolutions']
delete settings['transcode.hlsResolutions']
} else if (key === 'appearance.hideAgent') {
if (!settings.appearance) {
settings.appearance = {}
@@ -0,0 +1,737 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import * as fs from 'fs'
import {
extractEmbeddedJpeg,
validateExtractedJpeg,
validateExtractedImage,
decodeRawWithDcraw,
extractAndValidateRawPreview,
EXIF_ORIENTATION_TO_ROTATION,
withTimeout,
} from './raw-extract'
// Mock exiftool-vendored
vi.mock('exiftool-vendored', () => ({
exiftool: {
extractBinaryTag: vi.fn(),
read: vi.fn(),
},
}))
// Mock sharp
vi.mock('sharp', () => {
const mockMetadata = vi.fn()
const mockSharp = vi.fn(() => ({
metadata: mockMetadata,
}))
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(mockSharp as any)._mockMetadata = mockMetadata
return { default: mockSharp }
})
// Mock child_process
vi.mock('child_process', () => ({
execFile: vi.fn(),
}))
import { exiftool } from 'exiftool-vendored'
import sharp from 'sharp'
import { execFile } from 'child_process'
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mockMetadata = (sharp as any)._mockMetadata as ReturnType<typeof vi.fn>
const mockExtractBinaryTag = vi.mocked(exiftool.extractBinaryTag)
const mockRead = vi.mocked(exiftool.read)
const mockExecFile = vi.mocked(execFile)
describe('extractEmbeddedJpeg', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('extracts from JpgFromRaw2 first to a temporary file', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 1 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(1)
expect(mockExtractBinaryTag).toHaveBeenCalledWith(
'JpgFromRaw2',
'/path/to/photo.cr2',
expect.stringContaining('preview-JpgFromRaw2'),
)
// Test cleanup
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to JpgFromRaw when JpgFromRaw2 fails', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 6 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.nef')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(6)
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(2)
expect(mockExtractBinaryTag).toHaveBeenNthCalledWith(
2,
'JpgFromRaw',
'/path/to/photo.nef',
expect.stringContaining('preview-JpgFromRaw'),
)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to PreviewImage when JpgFromRaw2 and JpgFromRaw fail', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'preview-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 3 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.arw')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBe(3)
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(3)
expect(mockExtractBinaryTag).toHaveBeenNthCalledWith(
3,
'PreviewImage',
'/path/to/photo.arw',
expect.stringContaining('preview-PreviewImage'),
)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('returns null and cleans up when all tags fail', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
.mockRejectedValueOnce(new Error('0 output files created'))
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).toBeNull()
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(3)
})
it('returns undefined orientation when EXIF read fails', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockRejectedValueOnce(new Error('EXIF read failed'))
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
it('returns undefined orientation when orientation value is out of range', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 9 } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
it('returns undefined orientation when orientation is not a number', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 'Horizontal' } as never)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(result!.orientation).toBeUndefined()
result!.cleanup()
})
it('skips remaining tags and aborts early when extraction times out', async () => {
const origEnv = process.env.EXIFTOOL_TIMEOUT_MS
try {
process.env.EXIFTOOL_TIMEOUT_MS = '20'
// Hang on first tag
mockExtractBinaryTag.mockImplementationOnce(
() => new Promise((resolve) => setTimeout(resolve, 300)),
)
const result = await extractEmbeddedJpeg('/path/to/photo.cr2')
expect(result).toBeNull()
// Crucial: should break immediately after the first timeout instead of trying remaining 2 tags
expect(mockExtractBinaryTag).toHaveBeenCalledTimes(1)
} finally {
if (origEnv !== undefined) {
process.env.EXIFTOOL_TIMEOUT_MS = origEnv
} else {
delete process.env.EXIFTOOL_TIMEOUT_MS
}
}
})
})
describe('validateExtractedJpeg', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('returns dimensions for a valid JPEG file', async () => {
mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 })
const result = await validateExtractedJpeg('/tmp/valid-jpeg.jpg')
expect(result).toEqual({ width: 6000, height: 4000 })
})
it('returns null when metadata has zero width', async () => {
mockMetadata.mockResolvedValueOnce({ width: 0, height: 4000 })
const result = await validateExtractedJpeg('/tmp/bad-jpeg.jpg')
expect(result).toBeNull()
})
it('returns null when metadata has no dimensions', async () => {
mockMetadata.mockResolvedValueOnce({})
const result = await validateExtractedJpeg('/tmp/bad-jpeg.jpg')
expect(result).toBeNull()
})
it('returns null when sharp throws', async () => {
mockMetadata.mockRejectedValueOnce(new Error('Invalid image'))
const result = await validateExtractedJpeg('/tmp/corrupt-jpeg.jpg')
expect(result).toBeNull()
})
})
describe('validateExtractedImage', () => {
it('is an alias for validateExtractedJpeg', () => {
expect(validateExtractedImage).toBe(validateExtractedJpeg)
})
})
describe('decodeRawWithDcraw', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('decodes RAW using camera white balance, sRGB, and TIFF output without -h when longest dimension <= 8192', async () => {
mockRead.mockResolvedValueOnce({
ImageWidth: 6000,
ImageHeight: 4000,
Orientation: 1,
} as never)
let executedArgs: string[] = []
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
executedArgs = args as string[]
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.orientation).toBeUndefined()
expect(result!.rawWidth).toBe(6000)
expect(result!.rawHeight).toBe(4000)
expect(executedArgs).toContain('-w')
expect(executedArgs).toContain('-o')
expect(executedArgs).toContain('1')
expect(executedArgs).toContain('-T')
expect(executedArgs).toContain('-Z')
expect(executedArgs).not.toContain('-h')
expect(executedArgs[executedArgs.length - 1]).toBe('/path/to/photo.cr2')
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('enables half-size (-h) when longest dimension > 8192px and computes upright raw dimensions', async () => {
mockRead.mockResolvedValueOnce({
ImageWidth: 9504,
ImageHeight: 6336,
Orientation: 6,
} as never)
let executedArgs: string[] = []
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
executedArgs = args as string[]
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.arw')
expect(result).not.toBeNull()
expect(executedArgs).toContain('-h')
expect(result!.orientation).toBeUndefined()
// For orientation 6 (90 CW), upright raw dimensions are swapped
expect(result!.rawWidth).toBe(6336)
expect(result!.rawHeight).toBe(9504)
result!.cleanup()
})
it('omits -h when longest dimension is exactly 8192px', async () => {
mockRead.mockResolvedValueOnce({
ImageWidth: 8192,
ImageHeight: 5464,
Orientation: 1,
} as never)
let executedArgs: string[] = []
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
executedArgs = args as string[]
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.nef')
expect(result).not.toBeNull()
expect(executedArgs).not.toContain('-h')
result!.cleanup()
})
it('defaults to full size without -h when EXIF metadata cannot be read', async () => {
mockRead.mockRejectedValueOnce(new Error('Exif read failure'))
let executedArgs: string[] = []
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
executedArgs = args as string[]
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.dng')
expect(result).not.toBeNull()
expect(executedArgs).not.toContain('-h')
result!.cleanup()
})
it('defaults to full size without -h when EXIF metadata read times out', async () => {
const origEnv = process.env.EXIFTOOL_TIMEOUT_MS
try {
process.env.EXIFTOOL_TIMEOUT_MS = '20'
mockRead.mockImplementationOnce(() => new Promise((resolve) => setTimeout(resolve, 300)))
let executedArgs: string[] = []
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
executedArgs = args as string[]
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.dng')
expect(result).not.toBeNull()
expect(executedArgs).not.toContain('-h')
result!.cleanup()
} finally {
if (origEnv !== undefined) {
process.env.EXIFTOOL_TIMEOUT_MS = origEnv
} else {
delete process.env.EXIFTOOL_TIMEOUT_MS
}
}
})
it('returns null and cleans up when dcraw_emu exits with non-zero code', async () => {
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => {
const callback = typeof _opts === 'function' ? _opts : cb
const err = new Error('Command failed: dcraw_emu')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(err as any).code = 1
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(err)
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.cr2')
expect(result).toBeNull()
})
it('returns null and cleans up when dcraw_emu times out', async () => {
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => {
const callback = typeof _opts === 'function' ? _opts : cb
const err = new Error('Process timed out')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(err as any).code = 'ETIMEDOUT'
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(err)
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.cr2')
expect(result).toBeNull()
})
it('returns null and cleans up when dcraw_emu is not found (ENOENT)', async () => {
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => {
const callback = typeof _opts === 'function' ? _opts : cb
const err = new Error('spawn dcraw_emu ENOENT')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(err as any).code = 'ENOENT'
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(err)
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.cr2')
expect(result).toBeNull()
})
it('returns null and cleans up when output TIFF is missing or empty', async () => {
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => {
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await decodeRawWithDcraw('/path/to/photo.cr2')
expect(result).toBeNull()
})
})
describe('extractAndValidateRawPreview', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('returns previewPath, dimensions, orientation, and cleanup for valid RAW using embedded JPEG first', async () => {
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
fs.writeFileSync(dest, 'jpeg-data')
})
mockRead.mockResolvedValueOnce({ Orientation: 6, ImageWidth: 4000, ImageHeight: 3000 } as never)
mockMetadata.mockResolvedValueOnce({ width: 4000, height: 3000 })
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(result!.previewPath)).toBe(true)
expect(result!.width).toBe(4000)
expect(result!.height).toBe(3000)
expect(result!.orientation).toBe(6)
expect(mockExecFile).not.toHaveBeenCalled()
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to dcraw_emu when embedded preview extraction fails', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
mockRead.mockResolvedValueOnce({
ImageWidth: 6000,
ImageHeight: 4000,
Orientation: 1,
} as never)
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 })
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(mockExecFile).toHaveBeenCalled()
expect(result!.width).toBe(6000)
expect(result!.height).toBe(4000)
expect(result!.rawWidth).toBe(6000)
expect(result!.rawHeight).toBe(4000)
expect(result!.orientation).toBeUndefined()
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('falls back to dcraw_emu when extracted embedded JPEG is corrupt', async () => {
let createdJpegPath = ''
mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => {
createdJpegPath = dest
fs.writeFileSync(dest, 'corrupt-jpeg')
})
mockRead.mockResolvedValueOnce({ Orientation: 1 } as never)
mockMetadata.mockRejectedValueOnce(new Error('Not a valid image'))
// Fallback dcraw_emu
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 })
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(fs.existsSync(createdJpegPath)).toBe(false) // Embedded jpeg was cleaned up
expect(fs.existsSync(result!.previewPath)).toBe(true)
result!.cleanup()
expect(fs.existsSync(result!.previewPath)).toBe(false)
})
it('returns null when both embedded extraction and dcraw_emu fail', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
mockRead.mockRejectedValueOnce(new Error('fail'))
mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => {
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(new Error('dcraw failed'))
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).toBeNull()
})
it('returns null and cleans up when decoded TIFF is corrupt', async () => {
mockExtractBinaryTag
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
.mockRejectedValueOnce(new Error('fail'))
let createdTiff = ''
mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never)
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
createdTiff = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(createdTiff, 'corrupt-tiff')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
mockMetadata.mockRejectedValueOnce(new Error('Invalid TIFF'))
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).toBeNull()
expect(fs.existsSync(createdTiff)).toBe(false)
})
it('falls back to dcraw_emu when embedded preview extraction times out', async () => {
const origEnv = process.env.EXIFTOOL_TIMEOUT_MS
try {
process.env.EXIFTOOL_TIMEOUT_MS = '20'
// Hang on extractBinaryTag
mockExtractBinaryTag.mockImplementationOnce(
() => new Promise((resolve) => setTimeout(resolve, 300)),
)
mockRead.mockResolvedValueOnce({
ImageWidth: 6000,
ImageHeight: 4000,
Orientation: 1,
} as never)
mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => {
const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1]
fs.writeFileSync(tiffDest, 'tiff-data')
const callback = typeof _opts === 'function' ? _opts : cb
if (callback) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
;(callback as any)(null, { stdout: '', stderr: '' })
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return {} as any
})
mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 })
const result = await extractAndValidateRawPreview('/path/to/photo.cr2')
expect(result).not.toBeNull()
expect(mockExecFile).toHaveBeenCalled()
expect(result!.width).toBe(6000)
expect(result!.height).toBe(4000)
result!.cleanup()
} finally {
if (origEnv !== undefined) {
process.env.EXIFTOOL_TIMEOUT_MS = origEnv
} else {
delete process.env.EXIFTOOL_TIMEOUT_MS
}
}
})
})
describe('withTimeout', () => {
it('resolves when promise settles before timeout', async () => {
const result = await withTimeout(Promise.resolve('ok'), 1000)
expect(result).toBe('ok')
})
it('rejects with timeout error when promise exceeds timeout', async () => {
const hanging = new Promise((resolve) => setTimeout(resolve, 500))
await expect(withTimeout(hanging, 20, 'TestTask')).rejects.toThrow(
'TestTask timed out after 20ms',
)
})
it('propagates underlying promise rejection before timeout', async () => {
const failing = Promise.reject(new Error('fail fast'))
await expect(withTimeout(failing, 1000, 'TestTask')).rejects.toThrow('fail fast')
})
it('prevents unhandled rejection if promise rejects after timeout', async () => {
const delayedReject = new Promise((_, reject) =>
setTimeout(() => reject(new Error('late error')), 40),
)
await expect(withTimeout(delayedReject, 10, 'TestTask')).rejects.toThrow(
'TestTask timed out after 10ms',
)
await new Promise((resolve) => setTimeout(resolve, 60))
})
})
describe('EXIF_ORIENTATION_TO_ROTATION', () => {
it('maps all 8 EXIF orientations', () => {
expect(Object.keys(EXIF_ORIENTATION_TO_ROTATION)).toHaveLength(8)
for (let i = 1; i <= 8; i++) {
expect(EXIF_ORIENTATION_TO_ROTATION[i]).toBeDefined()
}
})
it('has correct rotation for landscape (1)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[1]).toEqual({})
})
it('has correct rotation for 90° CW (6)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[6]).toEqual({ angle: 90 })
})
it('has correct rotation for 180° (3)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[3]).toEqual({ angle: 180 })
})
it('has correct rotation for 270° CW (8)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[8]).toEqual({ angle: 270 })
})
it('has correct rotation for mirror horizontal (2)', () => {
expect(EXIF_ORIENTATION_TO_ROTATION[2]).toEqual({ flop: true })
})
})
+389
View File
@@ -0,0 +1,389 @@
import { execFile } from 'child_process'
import { exiftool } from 'exiftool-vendored'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import sharp from 'sharp'
import { promisify } from 'util'
import { logger } from '@shumai/core/src/logger'
const execFileAsync = promisify(execFile)
export interface RawExtractResult {
previewPath: string
cleanup: () => void
/** EXIF orientation from the RAW container (1–8), or undefined if absent or already applied. */
orientation?: number
/** Original unoriented or oriented image dimensions from EXIF if available */
rawWidth?: number
rawHeight?: number
}
export interface RawPreviewResult {
previewPath: string
cleanup: () => void
width: number
height: number
orientation?: number
rawWidth?: number
rawHeight?: number
}
export interface DecodeRawOptions {
timeoutMs?: number
signal?: AbortSignal
}
/**
* Maps EXIF orientation values (1–8) to Sharp rotation operations.
* Follows the same mapping as Immich's ORIENTATION_TO_SHARP_ROTATION.
*/
export const EXIF_ORIENTATION_TO_ROTATION: Record<
number,
{ angle?: number; flip?: boolean; flop?: boolean }
> = {
1: {}, // Horizontal (normal)
2: { flop: true }, // Mirror horizontal
3: { angle: 180 }, // Rotate 180°
4: { angle: 180, flop: true }, // Mirror vertical
5: { angle: 270, flip: true }, // Mirror horizontal + rotate 270° CW
6: { angle: 90 }, // Rotate 90° CW
7: { angle: 90, flip: true }, // Mirror horizontal + rotate 90° CW
8: { angle: 270 }, // Rotate 270° CW
}
const DEFAULT_EXIFTOOL_TIMEOUT_MS = 15_000
function getExifToolTimeoutMs(): number {
const envVal = process.env.EXIFTOOL_TIMEOUT_MS
if (envVal) {
const parsed = parseInt(envVal, 10)
if (!isNaN(parsed) && parsed > 0) {
return parsed
}
}
return DEFAULT_EXIFTOOL_TIMEOUT_MS
}
/**
* Wraps a promise with a timeout. If the timeout triggers first,
* the returned promise rejects with a timeout error and attaches a no-op handler
* to the underlying promise to prevent unhandled rejections.
*/
export function withTimeout<T>(
promise: Promise<T>,
timeoutMs: number,
operationName = 'Operation',
): Promise<T> {
let timer: ReturnType<typeof setTimeout> | undefined
const timeoutPromise = new Promise<never>((_, reject) => {
timer = setTimeout(() => {
reject(new Error(`${operationName} timed out after ${timeoutMs}ms`))
}, timeoutMs)
})
promise.catch(() => {})
return Promise.race([promise, timeoutPromise]).finally(() => {
if (timer) clearTimeout(timer)
})
}
/**
* Attempt to extract an embedded JPEG preview from a camera RAW file to a temporary file.
*
* Tries tags in priority order (matching Immich, skipping PreviewJXL):
* 1. JpgFromRaw2 — secondary full-resolution JPEG (newer Sony/Canon/Nikon)
* 2. JpgFromRaw — primary full-size embedded JPEG
* 3. PreviewImage — smaller embedded preview fallback
*
* Also reads the EXIF orientation from the RAW container so callers can
* apply it to the extracted buffer/file (which often lacks its own orientation tag).
*
* Returns null if no usable embedded JPEG exists.
*/
export async function extractEmbeddedJpeg(
rawFilePath: string,
targetDir?: string,
): Promise<RawExtractResult | null> {
const tags = ['JpgFromRaw2', 'JpgFromRaw', 'PreviewImage'] as const
const ownTempDir = !targetDir
const workDir = targetDir ?? fs.mkdtempSync(path.join(os.tmpdir(), 'raw-extract-'))
const cleanup = () => {
try {
if (ownTempDir && fs.existsSync(workDir)) {
fs.rmSync(workDir, { recursive: true, force: true })
}
} catch (err) {
logger.warn({ workDir, err }, 'Failed to clean up raw extraction temporary directory')
}
}
const exiftoolTimeoutMs = getExifToolTimeoutMs()
for (const tag of tags) {
const candidatePath = path.join(workDir, `preview-${tag}-${Date.now()}.jpg`)
try {
await withTimeout(
exiftool.extractBinaryTag(tag, rawFilePath, candidatePath),
exiftoolTimeoutMs,
`exiftool.extractBinaryTag(${tag})`,
)
if (fs.existsSync(candidatePath) && fs.statSync(candidatePath).size > 0) {
logger.debug({ tag, rawFilePath, candidatePath }, 'Extracted embedded JPEG from RAW')
// Read orientation and container dimensions from the RAW container's EXIF metadata.
let orientation: number | undefined
let rawWidth: number | undefined
let rawHeight: number | undefined
try {
const exif = await withTimeout(
exiftool.read(rawFilePath),
exiftoolTimeoutMs,
'exiftool.read',
)
const rawOrientation = exif.Orientation
if (typeof rawOrientation === 'number' && rawOrientation >= 1 && rawOrientation <= 8) {
orientation = rawOrientation
}
const w =
typeof exif.ImageWidth === 'number'
? exif.ImageWidth
: typeof exif.ExifImageWidth === 'number'
? exif.ExifImageWidth
: undefined
const h =
typeof exif.ImageHeight === 'number'
? exif.ImageHeight
: typeof exif.ExifImageHeight === 'number'
? exif.ExifImageHeight
: undefined
if (w && h && w > 0 && h > 0) {
rawWidth = w
rawHeight = h
}
} catch {
logger.debug({ rawFilePath }, 'Could not read EXIF orientation from RAW container')
}
return { previewPath: candidatePath, cleanup, orientation, rawWidth, rawHeight }
}
} catch (err) {
const isTimeout = err instanceof Error && err.message.includes('timed out')
logger.debug({ tag, rawFilePath, err, isTimeout }, `Could not extract ${tag} from RAW file`)
try {
if (fs.existsSync(candidatePath)) {
fs.rmSync(candidatePath, { force: true })
}
} catch {
// ignore
}
if (isTimeout) {
logger.warn(
{ rawFilePath, tag, timeoutMs: exiftoolTimeoutMs },
'ExifTool extraction timed out; skipping remaining tags to avoid further delays',
)
break
}
}
}
cleanup()
logger.info({ rawFilePath }, 'No usable embedded JPEG found in RAW file')
return null
}
/**
* Fallback RAW decoder using dcraw_emu (from LibRaw).
*
* Used when no usable embedded JPEG preview exists in the RAW file.
*
* Decoding options:
* - Camera white balance and sRGB output (-w -o 1)
* - Output temporary TIFF (-T -Z <tempPath>)
* - Half-size processing (-h) enabled only when longest dimension > 8192px
*
* LibRaw auto-rotates the output image to upright orientation based on camera EXIF,
* so the returned orientation is undefined to avoid double-rotation in downstream Sharp.
*/
export async function decodeRawWithDcraw(
rawFilePath: string,
targetDir?: string,
options?: DecodeRawOptions,
): Promise<RawExtractResult | null> {
const ownTempDir = !targetDir
const workDir = targetDir ?? fs.mkdtempSync(path.join(os.tmpdir(), 'raw-decode-'))
const tempTiffPath = path.join(
workDir,
`decoded-${Date.now()}-${Math.random().toString(36).slice(2, 8)}.tiff`,
)
const cleanup = () => {
try {
if (ownTempDir && fs.existsSync(workDir)) {
fs.rmSync(workDir, { recursive: true, force: true })
} else if (fs.existsSync(tempTiffPath)) {
fs.rmSync(tempTiffPath, { force: true })
}
} catch (err) {
logger.warn({ workDir, tempTiffPath, err }, 'Failed to clean up dcraw temporary files')
}
}
// 1. Probe RAW dimensions and orientation via ExifTool to check for 8192px threshold
let useHalfSize = false
let rawWidth: number | undefined
let rawHeight: number | undefined
try {
const exiftoolTimeoutMs = getExifToolTimeoutMs()
const exif = await withTimeout(exiftool.read(rawFilePath), exiftoolTimeoutMs, 'exiftool.read')
const w =
typeof exif.ImageWidth === 'number'
? exif.ImageWidth
: typeof exif.ExifImageWidth === 'number'
? exif.ExifImageWidth
: undefined
const h =
typeof exif.ImageHeight === 'number'
? exif.ImageHeight
: typeof exif.ExifImageHeight === 'number'
? exif.ExifImageHeight
: undefined
if (w && h && w > 0 && h > 0) {
const longest = Math.max(w, h)
if (longest > 8192) {
useHalfSize = true
}
// Compute upright raw dimensions matching dcraw_emu's auto-rotation
const rawOrientation = exif.Orientation
const isSwapped =
typeof rawOrientation === 'number' && rawOrientation >= 5 && rawOrientation <= 8
rawWidth = isSwapped ? h : w
rawHeight = isSwapped ? w : h
}
} catch (err) {
logger.debug({ rawFilePath, err }, 'Could not read EXIF metadata for RAW dimension check')
}
// 2. Build dcraw_emu arguments
const args = ['-w', '-o', '1']
if (useHalfSize) {
args.push('-h')
}
args.push('-T', '-Z', tempTiffPath, rawFilePath)
const dcrawBin = process.env.DCRAW_EMU_PATH || 'dcraw_emu'
const envTimeout = process.env.DCRAW_EMU_TIMEOUT_MS
? parseInt(process.env.DCRAW_EMU_TIMEOUT_MS, 10)
: undefined
const timeoutMs = options?.timeoutMs ?? (envTimeout && !isNaN(envTimeout) ? envTimeout : 60_000)
try {
logger.debug({ dcrawBin, args, rawFilePath }, 'Executing dcraw_emu fallback')
await execFileAsync(dcrawBin, args, {
timeout: timeoutMs,
signal: options?.signal,
})
if (!fs.existsSync(tempTiffPath) || fs.statSync(tempTiffPath).size === 0) {
logger.warn({ rawFilePath, tempTiffPath }, 'dcraw_emu did not produce a valid output TIFF')
cleanup()
return null
}
return {
previewPath: tempTiffPath,
cleanup,
orientation: undefined, // dcraw_emu already auto-rotates upright based on camera EXIF
rawWidth,
rawHeight,
}
} catch (err) {
logger.warn({ rawFilePath, err }, 'dcraw_emu failed to decode RAW file')
cleanup()
return null
}
}
/**
* Validate that an extracted or decoded image file (JPEG, TIFF, etc.) is decodable.
* Returns dimensions if valid, null if corrupt/unusable.
*/
export async function validateExtractedImage(
filePath: string,
): Promise<{ width: number; height: number } | null> {
try {
const metadata = await sharp(filePath).metadata()
if (metadata.width && metadata.height && metadata.width > 0 && metadata.height > 0) {
return { width: metadata.width, height: metadata.height }
}
return null
} catch {
return null
}
}
/** Backward compatibility alias */
export const validateExtractedJpeg = validateExtractedImage
/**
* Extract and validate a preview from a RAW file to a temporary file.
*
* 1. Attempts ExifTool embedded JPEG extraction first.
* 2. If no embedded preview is available or extraction fails, falls back to dcraw_emu decoding.
*
* Returns a validated file path + dimensions + orientation + cleanup function, or null.
*/
export async function extractAndValidateRawPreview(
rawFilePath: string,
targetDir?: string,
): Promise<RawPreviewResult | null> {
// 1. Try ExifTool embedded JPEG extraction first
const extracted = await extractEmbeddedJpeg(rawFilePath, targetDir)
if (extracted) {
const dims = await validateExtractedImage(extracted.previewPath)
if (dims) {
return {
previewPath: extracted.previewPath,
cleanup: extracted.cleanup,
orientation: extracted.orientation,
rawWidth: extracted.rawWidth,
rawHeight: extracted.rawHeight,
...dims,
}
}
logger.warn(
{ rawFilePath },
'Extracted JPEG from RAW is not decodable, falling back to dcraw_emu',
)
extracted.cleanup()
} else {
logger.info(
{ rawFilePath },
'No usable embedded JPEG found in RAW file, falling back to dcraw_emu',
)
}
// 2. Fallback to dcraw_emu
const decoded = await decodeRawWithDcraw(rawFilePath, targetDir)
if (!decoded) {
return null
}
const dims = await validateExtractedImage(decoded.previewPath)
if (!dims) {
logger.warn({ rawFilePath }, 'Decoded TIFF from dcraw_emu is not decodable, skipping preview')
decoded.cleanup()
return null
}
return {
previewPath: decoded.previewPath,
cleanup: decoded.cleanup,
orientation: decoded.orientation,
rawWidth: decoded.rawWidth,
rawHeight: decoded.rawHeight,
...dims,
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+7 -3
View File
@@ -198,17 +198,19 @@ describe('UploadService', () => {
threads: 0,
sprite: true,
poster: true,
hlsEnabled: false,
},
})
})
it('should pass custom transcode.threads from team settings to transcode task', async () => {
it('should pass custom transcode.threads and videoResolutions from team settings to transcode task', async () => {
await prisma.team.update({
where: { id: teamId },
data: {
settings: {
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 12,
},
@@ -244,11 +246,13 @@ describe('UploadService', () => {
expect(workflowTask?.payload).toEqual({
projectId: projectId,
transcode: {
videoStrategy: 'all',
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p'],
hardwareAcceleration: 'auto',
threads: 12,
sprite: true,
poster: true,
hlsEnabled: false,
},
})
})
+5
View File
@@ -334,12 +334,17 @@ export class UploadService {
const threads = settings?.transcode?.threads ?? 0
if (isVideo) {
const strategy = settings?.transcode?.videoStrategy || 'best_match'
const videoResolutions = settings?.transcode?.videoResolutions
const hardwareAcceleration = settings?.transcode?.hardwareAcceleration || 'off'
const hlsEnabled = settings?.transcode?.hlsEnabled ?? false
const hlsResolutions = settings?.transcode?.hlsResolutions
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await new VideoTranscoder(tx as any, asset.id, team.id, projectId)
.setStrategy(strategy)
.setVideoResolutions(videoResolutions)
.setHardwareAcceleration(hardwareAcceleration)
.setThreads(threads)
.setHls(hlsEnabled, hlsResolutions)
.withSprite()
.withPoster()
.submit()
+24
View File
@@ -15,6 +15,30 @@ const EXTENSION_MIME_TYPES: Record<string, string> = {
'.txt': 'text/plain',
'.md': 'text/markdown',
'.markdown': 'text/markdown',
// Camera RAW formats
'.3fr': 'image/x-hasselblad-3fr',
'.arw': 'image/x-sony-arw',
'.cr2': 'image/x-canon-cr2',
'.cr3': 'image/x-canon-cr3',
'.crw': 'image/x-canon-crw',
'.dcr': 'image/x-kodak-dcr',
'.dng': 'image/x-adobe-dng',
'.erf': 'image/x-epson-erf',
'.fff': 'image/x-hasselblad-fff',
'.iiq': 'image/x-phaseone-iiq',
'.kdc': 'image/x-kodak-kdc',
'.nef': 'image/x-nikon-nef',
'.nrw': 'image/x-nikon-nrw',
'.orf': 'image/x-olympus-orf',
'.pef': 'image/x-pentax-pef',
'.raf': 'image/x-fuji-raf',
'.raw': 'image/x-panasonic-raw',
'.rw2': 'image/x-panasonic-rw2',
'.rwl': 'image/x-leica-rwl',
'.sr2': 'image/x-sony-sr2',
'.srf': 'image/x-sony-srf',
'.srw': 'image/x-samsung-srw',
'.x3f': 'image/x-sigma-x3f',
}
/**
+14
View File
@@ -92,6 +92,20 @@ describe('getProxyType', () => {
expect(getProxyType(null, 'design.psd')).toBe('image')
})
it('should detect image proxyType for RAW camera files', () => {
// RAW files typically upload with application/octet-stream or null mediaType
expect(getProxyType('application/octet-stream', 'photo.cr2')).toBe('image')
expect(getProxyType('application/octet-stream', 'photo.arw')).toBe('image')
expect(getProxyType(null, 'photo.dng')).toBe('image')
expect(getProxyType(null, 'photo.nef')).toBe('image')
expect(getProxyType(null, 'photo.raf')).toBe('image')
expect(getProxyType(null, 'photo.cr3')).toBe('image')
expect(getProxyType(null, 'photo.rw2')).toBe('image')
// Case-insensitive
expect(getProxyType(null, 'PHOTO.ARW')).toBe('image')
expect(getProxyType(null, 'Photo.DNG')).toBe('image')
})
it('should detect video proxyType', () => {
expect(getProxyType('video/mp4', 'clip.mp4')).toBe('video')
})
+8 -1
View File
@@ -1,3 +1,5 @@
import { isRawImage } from './raw'
export type ProxyType = 'image' | 'video' | 'audio' | 'pdf'
export function isOfficeDocument(mediaType?: string | null, filename?: string | null): boolean {
@@ -73,7 +75,12 @@ export function getProxyType(
const lowerMediaType = mediaType?.toLowerCase() || ''
const lowerFilename = filename?.toLowerCase() || ''
if (lowerMediaType.startsWith('image/') || lowerFilename.endsWith('.psd')) return 'image'
if (
lowerMediaType.startsWith('image/') ||
lowerFilename.endsWith('.psd') ||
isRawImage(lowerFilename)
)
return 'image'
if (lowerMediaType.startsWith('video/')) return 'video'
if (lowerMediaType.startsWith('audio/')) return 'audio'
+68
View File
@@ -0,0 +1,68 @@
import { describe, it, expect } from 'vitest'
import { isRawImage } from './raw'
describe('isRawImage', () => {
it('recognizes common RAW extensions (lowercase)', () => {
expect(isRawImage('photo.cr2')).toBe(true)
expect(isRawImage('photo.arw')).toBe(true)
expect(isRawImage('photo.dng')).toBe(true)
expect(isRawImage('photo.nef')).toBe(true)
expect(isRawImage('photo.raf')).toBe(true)
expect(isRawImage('photo.rw2')).toBe(true)
expect(isRawImage('photo.orf')).toBe(true)
expect(isRawImage('photo.pef')).toBe(true)
expect(isRawImage('photo.cr3')).toBe(true)
expect(isRawImage('photo.x3f')).toBe(true)
expect(isRawImage('photo.raw')).toBe(true)
expect(isRawImage('photo.3fr')).toBe(true)
expect(isRawImage('photo.iiq')).toBe(true)
expect(isRawImage('photo.kdc')).toBe(true)
expect(isRawImage('photo.dcr')).toBe(true)
expect(isRawImage('photo.erf')).toBe(true)
expect(isRawImage('photo.fff')).toBe(true)
expect(isRawImage('photo.crw')).toBe(true)
expect(isRawImage('photo.nrw')).toBe(true)
expect(isRawImage('photo.rwl')).toBe(true)
expect(isRawImage('photo.sr2')).toBe(true)
expect(isRawImage('photo.srf')).toBe(true)
expect(isRawImage('photo.srw')).toBe(true)
})
it('recognizes RAW extensions case-insensitively', () => {
expect(isRawImage('photo.CR2')).toBe(true)
expect(isRawImage('IMG_1234.ARW')).toBe(true)
expect(isRawImage('test.DNG')).toBe(true)
expect(isRawImage('photo.Nef')).toBe(true)
})
it('rejects non-RAW image files', () => {
expect(isRawImage('photo.jpg')).toBe(false)
expect(isRawImage('photo.png')).toBe(false)
expect(isRawImage('photo.webp')).toBe(false)
expect(isRawImage('photo.gif')).toBe(false)
expect(isRawImage('photo.svg')).toBe(false)
})
it('rejects non-image files', () => {
expect(isRawImage('design.psd')).toBe(false)
expect(isRawImage('video.mp4')).toBe(false)
expect(isRawImage('doc.pdf')).toBe(false)
expect(isRawImage('readme.txt')).toBe(false)
})
it('rejects files without extensions', () => {
expect(isRawImage('photo')).toBe(false)
expect(isRawImage('rawfile')).toBe(false)
})
it('handles filenames with multiple dots', () => {
expect(isRawImage('my.photo.cr2')).toBe(true)
expect(isRawImage('backup.2024.dng')).toBe(true)
expect(isRawImage('my.photo.jpg')).toBe(false)
})
it('handles paths with directories', () => {
expect(isRawImage('/home/user/photos/photo.cr2')).toBe(true)
expect(isRawImage('C:\\Users\\photos\\photo.arw')).toBe(true)
})
})
+37
View File
@@ -0,0 +1,37 @@
/**
* Centralized set of recognized camera RAW file extensions.
* Used only for routing files into the ExifTool extraction path.
*/
const RAW_EXTENSIONS: ReadonlySet<string> = new Set([
'.3fr',
'.arw',
'.cr2',
'.cr3',
'.crw',
'.dcr',
'.dng',
'.erf',
'.fff',
'.iiq',
'.kdc',
'.nef',
'.nrw',
'.orf',
'.pef',
'.raf',
'.raw',
'.rw2',
'.rwl',
'.sr2',
'.srf',
'.srw',
'.x3f',
])
/** Check whether a filename has a recognized camera RAW extension. */
export function isRawImage(filename: string): boolean {
const dotIndex = filename.lastIndexOf('.')
if (dotIndex === -1) return false
const ext = filename.slice(dotIndex).toLowerCase()
return RAW_EXTENSIONS.has(ext)
}
+1 -1
View File
@@ -323,7 +323,7 @@ export class WatermarkService {
if (isVideo || isImage) {
mediaAssetIds.push(asset.id)
}
} else if (asset.type === 'folder') {
} else if (asset.type === 'folder' || asset.type === 'version_stack') {
folderIds.push(asset.id)
} else if (asset.type === 'symlink' && asset.targetId) {
// Resolve nested symlinks (e.g. folders that contain their own symlinks)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/db",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -0,0 +1,16 @@
-- Migrate videoStrategy 'all' / 'full' to 'multi'
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoStrategy}', '"multi"')
WHERE "settings"->'transcode'->>'videoStrategy' IN ('all', 'full');
-- Normalize legacy videoStrategy values 'single' / 'disable' to 'best_match'
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoStrategy}', '"best_match"')
WHERE "settings"->'transcode'->>'videoStrategy' IN ('single', 'disable');
-- Backfill default videoResolutions where transcode settings exist without videoResolutions
UPDATE "teams"
SET "settings" = jsonb_set("settings", '{transcode,videoResolutions}', '["480p", "720p", "1080p", "1440p", "2160p"]'::jsonb)
WHERE "settings" IS NOT NULL
AND jsonb_typeof("settings"->'transcode') = 'object'
AND NOT ("settings"->'transcode' ? 'videoResolutions');
@@ -0,0 +1,2 @@
-- AlterEnum
ALTER TYPE "AssetStatus" ADD VALUE 'failed';
+1
View File
@@ -40,6 +40,7 @@ enum AssetStatus {
uploaded
processing
processed
failed
trashed
pending_purge
}
+1
View File
@@ -36,6 +36,7 @@ export const AssetStatus = {
uploaded: 'uploaded',
processing: 'processing',
processed: 'processed',
failed: 'failed',
trashed: 'trashed',
pending_purge: 'pending_purge'
} as const
File diff suppressed because one or more lines are too long
+21 -1
View File
@@ -48,7 +48,8 @@ declare global {
// ----------------------------------------------------------------------
// Team Settings
// ----------------------------------------------------------------------
export type VideoTranscodeStrategy = 'best_match' | 'all'
export type VideoTranscodeStrategy = 'best_match' | 'multi' | 'all'
export type VideoResolutionLadder = '480p' | '720p' | '1080p' | '1440p' | '2160p'
export type HardwareAcceleration = 'off' | 'auto'
export type HdrType = 'pq' | 'hlg' | 'dovi_p5' | 'dovi_p8' | 'sdr'
@@ -57,10 +58,15 @@ declare global {
// ----------------------------------------------------------------------
export type CollectionFilter = import('@shumai/dtos').CollectionFilter
export type HlsResolutionLadder = '480p' | '720p' | '1080p' | '1440p' | '2160p'
export interface TranscodeSettings {
videoStrategy: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
export interface MediaProviderConfig {
@@ -164,6 +170,8 @@ declare global {
isHdr?: boolean
hdrType?: HdrType
dvProfile?: number
videoStreamIndex?: number
audioStreamIndex?: number
format: unknown
}
@@ -178,6 +186,12 @@ declare global {
url?: string
}
export interface HlsInfo {
key: string
url?: string
resolutions: VideoTranscode[]
}
export interface MediaInfo {
duration: number
filesize: number
@@ -193,6 +207,9 @@ declare global {
finishedAt: string
metadata: Metadata | null
original: OriginalInfo | null
isHls?: boolean
hls?: HlsInfo
error?: string
}
// ----------------------------------------------------------------------
@@ -336,11 +353,14 @@ declare global {
// ----------------------------------------------------------------------
export interface TaskSpec {
videoStrategy?: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
thumbnail?: boolean
sprite?: boolean
poster?: boolean
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
export interface AgentTaskPayload {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/dtos",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+22
View File
@@ -105,6 +105,7 @@ export const assetInfoSchema = z.object({
width: z.number(),
height: z.number(),
size: z.number(),
resolution: z.string().optional(),
hdr: z.boolean().optional(),
}),
)
@@ -123,8 +124,28 @@ export const assetInfoSchema = z.object({
.optional(),
videoPreview: z.object({ url: z.string(), key: z.string().optional() }).optional(),
pdfTranscode: z.object({ url: z.string(), key: z.string().optional() }).optional(),
isHls: z.boolean().optional(),
hls: z
.object({
key: z.string(),
url: z.string().optional(),
resolutions: z
.array(
z.object({
width: z.number(),
height: z.number(),
resolution: z.string().optional(),
hdr: z.boolean().optional(),
key: z.string().optional(),
url: z.string().optional(),
}),
)
.optional(),
})
.optional(),
proxyType: z.enum(['image', 'video', 'audio', 'pdf']).nullable().optional(),
metadata: mediaMetadataSchema.optional(),
error: z.string().optional(),
})
.optional(),
ancestorFolders: z.array(ancestorFolderSchema).optional(),
@@ -338,6 +359,7 @@ export interface VideoTranscode {
width: number
height: number
size: number
resolution?: string
hdr?: boolean
}
+45
View File
@@ -69,22 +69,56 @@ export const listMembersQuerySchema = z.object({
export const VideoTranscodeStrategy = {
// eslint-disable-next-line @typescript-eslint/naming-convention
best_match: 'best_match',
multi: 'multi',
all: 'all',
} as const
export type VideoTranscodeStrategy =
(typeof VideoTranscodeStrategy)[keyof typeof VideoTranscodeStrategy]
export const VideoResolutionLadder = {
r480p: '480p',
r720p: '720p',
r1080p: '1080p',
r1440p: '1440p',
r2160p: '2160p',
} as const
export type VideoResolutionLadder =
(typeof VideoResolutionLadder)[keyof typeof VideoResolutionLadder]
export const DEFAULT_VIDEO_RESOLUTIONS: VideoResolutionLadder[] = [
'480p',
'720p',
'1080p',
'1440p',
'2160p',
]
export const HardwareAcceleration = {
off: 'off',
auto: 'auto',
} as const
export type HardwareAcceleration = (typeof HardwareAcceleration)[keyof typeof HardwareAcceleration]
export const HlsResolutionLadder = {
r480p: '480p',
r720p: '720p',
r1080p: '1080p',
r1440p: '1440p',
r2160p: '2160p',
} as const
export type HlsResolutionLadder = (typeof HlsResolutionLadder)[keyof typeof HlsResolutionLadder]
export const DEFAULT_HLS_RESOLUTIONS: HlsResolutionLadder[] = ['480p', '720p', '1080p']
export const updateTeamSettingsRequestSchema = z.union([
z.object({
key: z.literal('transcode.videoStrategy'),
value: z.nativeEnum(VideoTranscodeStrategy),
}),
z.object({
key: z.literal('transcode.videoResolutions'),
value: z.array(z.nativeEnum(VideoResolutionLadder)),
}),
z.object({
key: z.literal('transcode.hardwareAcceleration'),
value: z.nativeEnum(HardwareAcceleration),
@@ -93,6 +127,14 @@ export const updateTeamSettingsRequestSchema = z.union([
key: z.literal('transcode.threads'),
value: z.number().int().min(0).max(32),
}),
z.object({
key: z.literal('transcode.hlsEnabled'),
value: z.boolean(),
}),
z.object({
key: z.literal('transcode.hlsResolutions'),
value: z.array(z.nativeEnum(HlsResolutionLadder)),
}),
z.object({
key: z.literal('appearance.hideAgent'),
value: z.boolean(),
@@ -107,8 +149,11 @@ export interface TeamAppearanceSettings {
export interface TeamSettingsResponse {
transcode?: {
videoStrategy?: VideoTranscodeStrategy
videoResolutions?: VideoResolutionLadder[]
hardwareAcceleration?: HardwareAcceleration
threads?: number
hlsEnabled?: boolean
hlsResolutions?: HlsResolutionLadder[]
}
appearance?: TeamAppearanceSettings
semanticSearchEnabled?: boolean
Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/e2e",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"dependencies": {
@@ -214,5 +214,86 @@ describe.each(['local', 'temporal'] as const)(
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
it('should run transcodeMedia workflow for a RAW image asset (.dng) successfully', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: { name: 'E2E RAW Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E RAW Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/test.dng',
},
})
const asset = await prisma.asset.create({
data: {
name: 'test.dng',
type: 'file',
status: 'uploaded',
mediaType: 'image/x-adobe-dng',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from fixture test.dng
const dngPath = path.join(fixturesDir, 'test.dng')
const dngBuffer = fs.readFileSync(dngPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/test.dng',
dngBuffer,
dngBuffer.length,
'image/x-adobe-dng',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_image',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
thumbnail: true,
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E RAW Image Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
// 5. Verification
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
imageTranscodes: unknown[]
thumbnail: unknown
}
expect(mediaInfo).toBeDefined()
expect(mediaInfo.proxyType).toBe('image')
expect(mediaInfo.imageTranscodes).toBeDefined()
expect(mediaInfo.imageTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.thumbnail).toBeDefined()
}, 50000)
},
)
+957 -1
View File
@@ -1,17 +1,63 @@
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest'
import { prisma, AssetStatus } from '@shumai/db'
import { prisma, AssetStatus, AssetType } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { workflowService, TaskQueueTranscode } from '@shumai/workflow-core'
import { initTranscodeWorkflows } from '@shumai/transcode'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { fileURLToPath } from 'url'
import * as path from 'path'
import * as fs from 'fs'
import * as os from 'os'
import { execFile } from 'child_process'
import { promisify } from 'util'
const execFileAsync = promisify(execFile)
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const transcodeWorkflowsPath = path.resolve(currentDir, '../../../apps/transcode/src/workflows.ts')
const fixturesDir = path.resolve(currentDir, '../fixtures')
async function sampleFirstFrameRgb(
buffer: Buffer,
ext: string = 'mp4',
): Promise<{ r: number; g: number; b: number }> {
const tmpFile = path.join(
os.tmpdir(),
`sample-${Date.now()}-${Math.random().toString(36).slice(2)}.${ext}`,
)
try {
fs.writeFileSync(tmpFile, buffer)
const proc = await execFileAsync(
'ffmpeg',
[
'-y',
'-i',
tmpFile,
'-vframes',
'1',
'-s',
'1x1',
'-f',
'rawvideo',
'-pix_fmt',
'rgb24',
'pipe:1',
],
{ encoding: 'buffer' },
)
return {
r: proc.stdout[0],
g: proc.stdout[1],
b: proc.stdout[2],
}
} finally {
if (fs.existsSync(tmpFile)) {
fs.unlinkSync(tmpFile)
}
}
}
describe.each(['local', 'temporal'] as const)(
'Workflow E2E - transcodeVideoWorkflow (executor: %s)',
(mode) => {
@@ -221,5 +267,915 @@ describe.each(['local', 'temporal'] as const)(
// Audio proxy key should end with -audio-proxy.mp4
expect(mediaInfo.videoTranscodes[0].key).toContain('-audio-proxy.mp4')
}, 50000)
it('should run transcodeMedia workflow with HLS enabled and serve playlists for version stacks and symlinks', async () => {
// 1. Seed Database
const team = await prisma.team.create({
data: {
name: 'E2E HLS Video Transcode Team',
settings: {
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
const project = await prisma.project.create({
data: { name: 'E2E HLS Video Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: {
key: 'projects/e2e/video-hls.mp4',
},
})
const asset = await prisma.asset.create({
data: {
name: 'video-hls.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
// 2. Seed S3 Storage from Fixture (small-480p.mp4)
const mp4Path = path.join(fixturesDir, 'small-480p.mp4')
const mp4Buffer = fs.readFileSync(mp4Path)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/video-hls.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
// 3. Create Workflow Task
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
// 4. Wait for workflow to complete
console.log(
`Submitted E2E HLS Transcode Workflow Task. ID: ${task.id}. Awaiting completion...`,
)
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
// 5. Verification of HLS generation
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as PrismaJson.MediaInfo
expect(mediaInfo.isHls).toBe(true)
expect(mediaInfo.hls).toBeDefined()
expect(mediaInfo.hls?.key).toBe('projects/e2e/hls/master.m3u8')
expect(mediaInfo.hls?.resolutions).toBeDefined()
expect(mediaInfo.hls?.resolutions?.length).toBeGreaterThan(0)
expect(mediaInfo.hls?.resolutions?.[0]?.resolution).toBe('480p')
// Verify master playlist in S3
const masterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/hls/master.m3u8',
)
const masterPlaylist = masterObj.buffer.toString('utf-8')
expect(masterPlaylist).toContain('#EXTM3U')
expect(masterPlaylist).toContain('480p/index.m3u8')
// Verify variant playlist in S3
const variantObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/hls/480p/index.m3u8',
)
expect(variantObj.buffer.toString('utf-8')).toContain('#EXTM3U')
// 6. Test Version Stack & Symlink Resolution
const stack = await prisma.asset.create({
data: {
name: 'video-hls-stack',
type: AssetType.version_stack,
status: AssetStatus.processed,
projectId: project.id,
},
})
await prisma.asset.update({
where: { id: asset.id },
data: {
parentId: stack.id,
sortIndex: 'a',
},
})
const symlink = await prisma.asset.create({
data: {
name: 'video-hls-symlink',
type: AssetType.symlink,
status: AssetStatus.processed,
projectId: project.id,
targetId: stack.id,
},
})
// Calling getHlsMasterPlaylist with stack ID
const stackMaster = await assetService.getHlsMasterPlaylist({ assetId: stack.id })
expect(stackMaster).toContain('#EXTM3U')
expect(stackMaster).toContain('480p/index.m3u8')
// Calling getHlsMasterPlaylist with symlink ID
const symlinkMaster = await assetService.getHlsMasterPlaylist({ assetId: symlink.id })
expect(symlinkMaster).toContain('#EXTM3U')
expect(symlinkMaster).toContain('480p/index.m3u8')
// Calling getHlsVariantPlaylist with symlink ID
const symlinkVariant = await assetService.getHlsVariantPlaylist({
assetId: symlink.id,
resolution: '480p',
})
expect(symlinkVariant).toContain('#EXTM3U')
}, 60000)
it('Case 1: should ignore attached picture cover art (Red) and select primary video stream (Green)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Cover Art Video Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Cover Art Video Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case1/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case1-'))
const videoPath = path.join(tmpDir, 'case1.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=green:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-pix_fmt:v:0',
'yuv420p',
'-c:v:1',
'mjpeg',
'-pix_fmt:v:1',
'yuvj420p',
'-c:a',
'aac',
'-disposition:v:1',
'attached_pic',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case1/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
sprite: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
sprite: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
expect(mediaInfo.metadata.videoStreamIndex).toBe(0)
// Verify poster is Green (not Red attached pic)
expect(mediaInfo.poster?.key).toBeDefined()
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.g).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.b).toBeLessThan(50)
// Verify MP4 proxy is Green
expect(mediaInfo.videoTranscodes.length).toBeGreaterThan(0)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.g).toBeGreaterThan(100)
expect(videoColor.r).toBeLessThan(50)
expect(videoColor.b).toBeLessThan(50)
// Verify sprite sheet exists
expect(mediaInfo.sprite?.key).toBeDefined()
await s3Service.headObject('shumai-e2e-test-bucket-transcode', mediaInfo.sprite.key)
}, 60000)
it('Case 2: should select higher bitrate stream (Blue) when both streams have default flag', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Bitrate Comparison Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Bitrate Comparison Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case2/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case2-'))
const videoPath = path.join(tmpDir, 'case2.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-b:v:0',
'200k',
'-minrate:v:0',
'200k',
'-maxrate:v:0',
'200k',
'-bufsize:v:0',
'200k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'2000k',
'-minrate:v:1',
'2000k',
'-maxrate:v:1',
'2000k',
'-bufsize:v:1',
'2000k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'default',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case2/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 1 (Blue, 2000k) should be selected over Stream 0 (Red, 200k)
expect(mediaInfo.metadata.videoStreamIndex).toBe(1)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.b).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.g).toBeLessThan(50)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.b).toBeGreaterThan(100)
expect(videoColor.r).toBeLessThan(50)
expect(videoColor.g).toBeLessThan(50)
}, 60000)
it('Case 3: should select stream with default flag (Yellow) over higher bitrate stream without default flag (Blue)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Default Flag Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Default Flag Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case3/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case3-'))
const videoPath = path.join(tmpDir, 'case3.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=yellow:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:a',
'-c:v:0',
'libx264',
'-b:v:0',
'200k',
'-minrate:v:0',
'200k',
'-maxrate:v:0',
'200k',
'-bufsize:v:0',
'200k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'2000k',
'-minrate:v:1',
'2000k',
'-maxrate:v:1',
'2000k',
'-bufsize:v:1',
'2000k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'0',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case3/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 0 (Yellow, default: 1) should win over Stream 1 (Blue, 10x higher bitrate but default: 0)
expect(mediaInfo.metadata.videoStreamIndex).toBe(0)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.r).toBeGreaterThan(100)
expect(posterColor.g).toBeGreaterThan(100)
expect(posterColor.b).toBeLessThan(50)
const videoObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
const videoColor = await sampleFirstFrameRgb(videoObj.buffer, 'mp4')
expect(videoColor.r).toBeGreaterThan(100)
expect(videoColor.g).toBeGreaterThan(100)
expect(videoColor.b).toBeLessThan(50)
}, 60000)
it('Case 4: should handle audio-only MP4 with cover art by downgrading to audio proxy (catches audio bug)', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Audio Cover Art Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Audio Cover Art Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case4/audio-cover.mp4' },
})
// Uploaded as video/mp4 because of .mp4 container, even though it only has audio + cover art
const asset = await prisma.asset.create({
data: {
name: 'audio-cover.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case4-'))
const videoPath = path.join(tmpDir, 'case4.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-f',
'lavfi',
'-i',
'color=c=magenta:s=320x240:d=2',
'-map',
'0:a',
'-map',
'1:v',
'-c:a',
'aac',
'-c:v',
'mjpeg',
'-pix_fmt:v:0',
'yuvj420p',
'-disposition:v:0',
'attached_pic',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case4/audio-cover.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
// Spec requested poster, sprite, videoStrategy as video upload would
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
poster: true,
sprite: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
// Must complete without error (previously failed trying to generate poster/sprite for audio)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
poster?: unknown
sprite?: unknown
videoTranscodes: { key: string }[]
}
expect(mediaInfo.proxyType).toBe('audio')
expect(mediaInfo.poster).toBeUndefined()
expect(mediaInfo.sprite).toBeUndefined()
expect(mediaInfo.videoTranscodes.length).toBeGreaterThan(0)
expect(mediaInfo.videoTranscodes[0].key).toContain('-audio-proxy.mp4')
await s3Service.headObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.videoTranscodes[0].key,
)
}, 60000)
it('Case 5: should rank 3 video streams with HLS enabled and select highest bitrate stream (Blue)', async () => {
const team = await prisma.team.create({
data: {
name: 'E2E Multi Stream HLS Team',
settings: {
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
},
},
},
})
const project = await prisma.project.create({
data: { name: 'E2E Multi Stream HLS Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/case5/video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'e2e-case5-'))
const videoPath = path.join(tmpDir, 'case5.mp4')
try {
await execFileAsync('ffmpeg', [
'-y',
'-f',
'lavfi',
'-i',
'color=c=red:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=green:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'color=c=blue:s=320x240:d=2',
'-f',
'lavfi',
'-i',
'sine=f=1000:d=2',
'-map',
'0:v',
'-map',
'1:v',
'-map',
'2:v',
'-map',
'3:a',
'-c:v:0',
'libx264',
'-b:v:0',
'100k',
'-minrate:v:0',
'100k',
'-maxrate:v:0',
'100k',
'-bufsize:v:0',
'100k',
'-nal-hrd',
'cbr',
'-disposition:v:0',
'default',
'-c:v:1',
'libx264',
'-b:v:1',
'500k',
'-minrate:v:1',
'500k',
'-maxrate:v:1',
'500k',
'-bufsize:v:1',
'500k',
'-nal-hrd',
'cbr',
'-disposition:v:1',
'default',
'-c:v:2',
'libx264',
'-b:v:2',
'1500k',
'-minrate:v:2',
'1500k',
'-maxrate:v:2',
'1500k',
'-bufsize:v:2',
'1500k',
'-nal-hrd',
'cbr',
'-disposition:v:2',
'default',
'-c:a',
'aac',
videoPath,
])
const mp4Buffer = fs.readFileSync(videoPath)
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/case5/video.mp4',
mp4Buffer,
mp4Buffer.length,
'video/mp4',
)
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p'],
poster: true,
},
},
},
})
const completedTask = await workflowService.executeWait(task, 45000)
expect(completedTask.status).toBe('completed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.processed)
const mediaInfo = updatedAsset?.media as unknown as {
proxyType: string
metadata: { videoStreamIndex: number }
poster: { key: string }
isHls: boolean
hls: { key: string; resolutions: { resolution: string }[] }
}
expect(mediaInfo.proxyType).toBe('video')
// Stream 2 (Blue, 1500k) should be selected
expect(mediaInfo.metadata.videoStreamIndex).toBe(2)
const posterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.poster.key,
)
const posterColor = await sampleFirstFrameRgb(posterObj.buffer, 'webp')
expect(posterColor.b).toBeGreaterThan(100)
expect(posterColor.r).toBeLessThan(50)
expect(posterColor.g).toBeLessThan(50)
expect(mediaInfo.isHls).toBe(true)
const masterObj = await s3Service.getObject(
'shumai-e2e-test-bucket-transcode',
mediaInfo.hls.key,
)
expect(masterObj.buffer.toString('utf-8')).toContain('480p/index.m3u8')
}, 60000)
it('should transition both task and asset to failed status with media.error when transcoding fails', async () => {
const team = await prisma.team.create({
data: { name: 'E2E Failed Transcode Team' },
})
const project = await prisma.project.create({
data: { name: 'E2E Failed Transcode Project', teamId: team.id },
})
const storageKey = await prisma.storageKey.create({
data: { key: 'projects/e2e/corrupt-video.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'corrupt-video.mp4',
type: 'file',
status: 'uploaded',
mediaType: 'video/mp4',
projectId: project.id,
storageKeyId: storageKey.id,
},
})
const corruptBuffer = Buffer.from('not a valid mp4 media stream at all')
await s3Service.putObject(
'shumai-e2e-test-bucket-transcode',
'projects/e2e/corrupt-video.mp4',
corruptBuffer,
corruptBuffer.length,
'video/mp4',
)
const task = await prisma.workflowTask.create({
data: {
type: 'transcode_video',
status: 'pending',
assetId: asset.id,
projectId: project.id,
teamId: team.id,
payload: {
projectId: project.id,
transcode: {
videoStrategy: 'best_match',
},
},
},
})
await expect(workflowService.executeWait(task, 45000)).rejects.toThrow()
const failedTask = await prisma.workflowTask.findUnique({
where: { id: task.id },
})
expect(failedTask?.status).toBe('failed')
const updatedAsset = await prisma.asset.findUnique({
where: { id: asset.id },
})
expect(updatedAsset?.status).toBe(AssetStatus.failed)
const media = updatedAsset?.media as PrismaJson.MediaInfo
expect(media?.error).toBeDefined()
expect(media?.error?.length).toBeGreaterThan(0)
}, 45000)
},
)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/tableprinter",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode",
"version": "0.4.9",
"version": "0.5.1",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -217,6 +217,86 @@ describe('Transcode Activities', () => {
)
})
it('should downgrade proxyType to audio when getVideoInfo throws No video stream found but audio exists', async () => {
const asset = await prisma.asset.create({
data: { name: 'audio-with-art.mp4', type: 'file', status: 'uploaded' },
})
vi.mocked(transcodeService.getVideoInfo).mockRejectedValueOnce(
new Error('No video stream found'),
)
vi.mocked(transcodeService.getAudioInfo).mockResolvedValueOnce({
originalWidth: 0,
originalHeight: 0,
duration: 42,
bitRate: 192000,
frameRate: 0,
totalFrames: 0,
hasAudio: true,
audioCodec: 'aac',
audioChannels: 2,
audioSampleRate: 44100,
audioBitDepth: 16,
audioStreamIndex: 0,
mimeType: 'audio/mp4',
})
const result = await getMediaInfoActivity({
assetId: asset.id,
filePath: '/tmp/audio-with-art.mp4',
proxyType: 'video',
mediaType: 'video/mp4',
})
expect(transcodeService.getVideoInfo).toHaveBeenCalledWith('/tmp/audio-with-art.mp4')
expect(transcodeService.getAudioInfo).toHaveBeenCalledWith('/tmp/audio-with-art.mp4')
expect(result.proxyType).toBe('audio')
expect(result.duration).toBe(42)
expect(result.metadata?.hasAudio).toBe(true)
expect(result.metadata?.audioStreamIndex).toBe(0)
expect(result.metadata?.videoStreamIndex).toBeUndefined()
expect(metadataService.updateAssetMetadata).toHaveBeenCalledWith(
asset.id,
expect.arrayContaining([
{ key: 'file_type', value: 'audio' },
{ key: 'proxy_type', value: 'audio' },
{ key: 'duration', value: 42 },
]),
true,
)
})
it('should populate videoStreamIndex and audioStreamIndex in mediaInfo.metadata', async () => {
const asset = await prisma.asset.create({
data: { name: 'dual-stream.mp4', type: 'file', status: 'uploaded' },
})
vi.mocked(transcodeService.getVideoInfo).mockResolvedValueOnce({
originalWidth: 1920,
originalHeight: 1080,
duration: 15,
bitRate: 2000000,
frameRate: 30,
totalFrames: 450,
hasAudio: true,
videoCodec: 'h264',
audioCodec: 'aac',
videoStreamIndex: 1,
audioStreamIndex: 2,
mimeType: 'video/mp4',
})
const result = await getMediaInfoActivity({
assetId: asset.id,
filePath: '/tmp/dual-stream.mp4',
proxyType: 'video',
mediaType: 'video/mp4',
})
expect(result.metadata?.videoStreamIndex).toBe(1)
expect(result.metadata?.audioStreamIndex).toBe(2)
})
it('should call getImageInfo and set file_type to image', async () => {
const asset = await prisma.asset.create({
data: { name: 'i.png', type: 'file', status: 'uploaded' },
@@ -703,6 +783,28 @@ describe('Transcode Activities', () => {
)
})
it('transcodeVideoActivity forwards streamIndex and audioStreamIndex to transcodeService', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
await transcodeVideoActivity({
assetKey: 'v.mp4',
filePath: '/tmp/v.mp4',
videoSpec: { resolution: '720p', width: 1280, height: 720 },
duration: 10,
originalFps: 30,
streamIndex: 2,
audioStreamIndex: 1,
})
expect(transcodeService.transcodeVideo).toHaveBeenCalledWith(
expect.objectContaining({
inputFile: '/tmp/v.mp4',
streamIndex: 2,
audioStreamIndex: 1,
}),
)
})
it('transcodeVideoActivity appends -hdr.mp4 suffix and forwards HDR metadata', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
@@ -896,6 +998,24 @@ describe('Transcode Activities', () => {
).rejects.toThrowError(/Image transcoding failed/)
})
it('should throw non-retryable ApplicationFailure when transcodeImageActivity fails with dcraw error', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
vi.mocked(transcodeService.transcodeImage).mockRejectedValue(
new Error('dcraw_emu failed to decode RAW file'),
)
await expect(
transcodeImageActivity({
assetKey: 'photo.cr2',
filePath: '/tmp/photo.cr2',
imageSpec: { width: 800, height: 600, quality: 90, format: 'webp' },
}),
).rejects.toMatchObject({
message: expect.stringContaining('Image transcoding failed'),
nonRetryable: true,
})
})
it('should throw non-retryable ApplicationFailure when generateSpriteActivity fails with error', async () => {
vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found'))
vi.mocked(transcodeService.generateSprite).mockRejectedValue(new Error('FFmpeg failed'))
@@ -946,6 +1066,44 @@ describe('Transcode Activities', () => {
hdrType: 'pq',
colorTransfer: 'smpte2084',
},
undefined,
)
})
it('generateSpriteActivity forwards videoStreamIndex to transcodeService.generateSprite', async () => {
vi.mocked(transcodeService.generateSprite).mockImplementation(
async (_in, spriteOut, posterOut) => {
const fs = await import('fs')
fs.writeFileSync(spriteOut, 'fake-sprite')
fs.writeFileSync(posterOut, 'fake-poster')
},
)
await generateSpriteActivity({
assetKey: 'files/asset-1/video.mp4',
filePath: '/tmp/video.mp4',
mediaInfo: {
duration: 10,
metadata: {
videoStreamIndex: 1,
},
} as unknown as PrismaJson.MediaInfo,
spriteSpec: { key: 'files/asset-1/sprite.webp', frames: 100, tileX: 10, tileY: 10 },
posterSpec: { key: 'files/asset-1/poster.webp' },
})
expect(transcodeService.generateSprite).toHaveBeenCalledWith(
'/tmp/video.mp4',
expect.any(String),
expect.any(String),
10,
undefined,
{
isHdr: undefined,
hdrType: undefined,
colorTransfer: undefined,
},
1,
)
})
+313 -48
View File
@@ -1,10 +1,16 @@
import { AssetStatus, prisma, WorkflowTaskType, WorkflowTaskStatus } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService, type HdrType } from '@shumai/core/src/transcode/transcode'
import {
transcodeService,
buildHlsMasterPlaylist,
calculateEffectiveBitrateBps,
type HdrType,
} from '@shumai/core/src/transcode/transcode'
import { metadataService } from '@shumai/core/src/metadata/metadata'
import { getDerivedArtifactDirectory, stemFromKey } from '@shumai/core/src/utils/filename'
import { gotenbergService } from '@shumai/core/src/gotenberg/gotenberg'
import { parseCsvContent } from '@shumai/core/src/transcode/transcode'
import { resolutionToDimensions } from '../workflows/transcode-utils'
import {
getProxyType,
isCsvDocument,
@@ -124,54 +130,110 @@ export async function getMediaInfoActivity(params: {
}
if (isVideo) {
const info = await transcodeService.getVideoInfo(params.filePath)
mediaInfo.duration = info.duration
mediaInfo.frames = info.totalFrames
mediaInfo.metadata = {
originalWidth: info.originalWidth,
originalHeight: info.originalHeight,
duration: info.duration,
bitRate: info.bitRate,
videoBitRate: info.videoBitRate,
frameRate: info.frameRate,
totalFrames: info.totalFrames,
startTimecode: info.startTimecode || '00:00:00:00',
hasAudio: info.hasAudio,
videoCodec: info.videoCodec,
audioCodec: info.audioCodec,
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
colorTransfer: info.colorTransfer,
colorPrimaries: info.colorPrimaries,
colorSpace: info.colorSpace,
isHdr: info.isHdr,
hdrType: info.hdrType,
dvProfile: info.dvProfile,
rotation: info.rotation,
format: {},
let isDowngradedToAudio = false
let videoInfo: Awaited<ReturnType<typeof transcodeService.getVideoInfo>> | null = null
try {
videoInfo = await transcodeService.getVideoInfo(params.filePath)
} catch (err) {
if (err instanceof Error && err.message.includes('No video stream found')) {
try {
const audioInfo = await transcodeService.getAudioInfo(params.filePath)
isDowngradedToAudio = true
mediaInfo.proxyType = 'audio'
mediaInfo.duration = audioInfo.duration
mediaInfo.metadata = {
originalWidth: 0,
originalHeight: 0,
duration: audioInfo.duration,
bitRate: audioInfo.bitRate,
frameRate: 0,
totalFrames: 0,
startTimecode: '00:00:00:00',
hasAudio: true,
videoCodec: undefined,
audioCodec: audioInfo.audioCodec,
audioChannels: audioInfo.audioChannels,
audioSampleRate: audioInfo.audioSampleRate,
audioBitDepth: audioInfo.audioBitDepth,
audioStreamIndex: audioInfo.audioStreamIndex,
format: {},
}
const ftIdx = metadataUpdates.findIndex((u) => u.key === 'file_type')
if (ftIdx !== -1) metadataUpdates[ftIdx].value = 'audio'
const ptIdx = metadataUpdates.findIndex((u) => u.key === 'proxy_type')
if (ptIdx !== -1) metadataUpdates[ptIdx].value = 'audio'
metadataUpdates.push(
{ key: 'duration', value: audioInfo.duration },
{ key: 'bitRate', value: audioInfo.bitRate / 1000 },
)
if (audioInfo.audioCodec)
metadataUpdates.push({ key: 'audio_codec', value: audioInfo.audioCodec })
if (audioInfo.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: audioInfo.audioChannels })
if (audioInfo.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: audioInfo.audioSampleRate })
if (audioInfo.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: audioInfo.audioBitDepth })
} catch {
throw err
}
} else {
throw err
}
}
if (!isDowngradedToAudio && videoInfo) {
const info = videoInfo
mediaInfo.duration = info.duration
mediaInfo.frames = info.totalFrames
mediaInfo.metadata = {
originalWidth: info.originalWidth,
originalHeight: info.originalHeight,
duration: info.duration,
bitRate: info.bitRate,
videoBitRate: info.videoBitRate,
frameRate: info.frameRate,
totalFrames: info.totalFrames,
startTimecode: info.startTimecode || '00:00:00:00',
hasAudio: info.hasAudio,
videoCodec: info.videoCodec,
audioCodec: info.audioCodec,
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
colorTransfer: info.colorTransfer,
colorPrimaries: info.colorPrimaries,
colorSpace: info.colorSpace,
isHdr: info.isHdr,
hdrType: info.hdrType,
dvProfile: info.dvProfile,
rotation: info.rotation,
videoStreamIndex: info.videoStreamIndex,
audioStreamIndex: info.audioStreamIndex,
format: {},
}
metadataUpdates.push(
{ key: 'resolution_width', value: info.originalWidth },
{ key: 'resolution_height', value: info.originalHeight },
{ key: 'duration', value: info.duration },
{ key: 'bitRate', value: info.bitRate / 1000 },
{ key: 'frame_rate', value: info.frameRate },
)
if (info.rotation !== undefined)
metadataUpdates.push({ key: 'rotation', value: info.rotation })
if (info.isHdr !== undefined)
metadataUpdates.push({ key: 'is_hdr', value: info.isHdr ? 1 : 0 })
if (info.colorTransfer)
metadataUpdates.push({ key: 'color_transfer', value: info.colorTransfer })
if (info.videoCodec) metadataUpdates.push({ key: 'video_codec', value: info.videoCodec })
if (info.audioCodec) metadataUpdates.push({ key: 'audio_codec', value: info.audioCodec })
if (info.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: info.audioChannels })
if (info.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: info.audioSampleRate })
if (info.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: info.audioBitDepth })
}
metadataUpdates.push(
{ key: 'resolution_width', value: info.originalWidth },
{ key: 'resolution_height', value: info.originalHeight },
{ key: 'duration', value: info.duration },
{ key: 'bitRate', value: info.bitRate / 1000 },
{ key: 'frame_rate', value: info.frameRate },
)
if (info.rotation !== undefined)
metadataUpdates.push({ key: 'rotation', value: info.rotation })
if (info.isHdr !== undefined)
metadataUpdates.push({ key: 'is_hdr', value: info.isHdr ? 1 : 0 })
if (info.colorTransfer)
metadataUpdates.push({ key: 'color_transfer', value: info.colorTransfer })
if (info.videoCodec) metadataUpdates.push({ key: 'video_codec', value: info.videoCodec })
if (info.audioCodec) metadataUpdates.push({ key: 'audio_codec', value: info.audioCodec })
if (info.audioChannels !== undefined)
metadataUpdates.push({ key: 'audio_channels', value: info.audioChannels })
if (info.audioSampleRate !== undefined)
metadataUpdates.push({ key: 'audio_sample_rate', value: info.audioSampleRate })
if (info.audioBitDepth !== undefined)
metadataUpdates.push({ key: 'audio_bit_depth', value: info.audioBitDepth })
} else if (isAudio) {
const info = await transcodeService.getAudioInfo(params.filePath)
mediaInfo.duration = info.duration
@@ -189,6 +251,7 @@ export async function getMediaInfoActivity(params: {
audioChannels: info.audioChannels,
audioSampleRate: info.audioSampleRate,
audioBitDepth: info.audioBitDepth,
audioStreamIndex: info.audioStreamIndex,
format: {},
}
metadataUpdates.push(
@@ -271,6 +334,9 @@ export interface VideoActivityParams {
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
sourceRotation?: number
streamIndex?: number
audioStreamIndex?: number
}
export async function transcodeVideoActivity(
@@ -332,6 +398,9 @@ export async function transcodeVideoActivity(
sourceColorTransfer: params.sourceColorTransfer,
sourceColorPrimaries: params.sourceColorPrimaries,
sourceColorSpace: params.sourceColorSpace,
sourceRotation: params.sourceRotation,
streamIndex: params.streamIndex,
audioStreamIndex: params.audioStreamIndex,
})
const stat = fs.statSync(outputFile)
@@ -380,11 +449,199 @@ export async function transcodeVideoActivity(
}
}
export interface HlsActivityParams {
taskId: string
assetKey: string
filePath: string
ladders: PrismaJson.HlsResolutionLadder[]
originalWidth: number
originalHeight: number
duration: number
originalFps: number
hardwareAcceleration?: 'off' | 'auto'
sourceVideoBitrate?: number
threads?: number
sourceIsHdr?: boolean
sourceHdrType?: HdrType
sourceColorTransfer?: string
sourceColorPrimaries?: string
sourceColorSpace?: string
sourceRotation?: number
streamIndex?: number
audioStreamIndex?: number
}
export async function transcodeHlsActivity(params: HlsActivityParams): Promise<PrismaJson.HlsInfo> {
const bucket = process.env.S3_BUCKET || 'shumai'
const assetDir = path.posix.dirname(params.assetKey)
const hlsDirKey = path.posix.join(assetDir, 'hls')
const masterKey = path.posix.join(hlsDirKey, 'master.m3u8')
const signal = getActivityCancellationSignal(params.taskId)
if (signal?.aborted) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
})
}
const hlsTmpDir = path.join(path.dirname(params.filePath), `hls-${ulid()}`)
try {
const renditionSpecs: Array<{
resolution: string
width: number
height: number
bitrateBps: number
isHdr?: boolean
}> = []
const mediaResolutions: PrismaJson.VideoTranscode[] = []
for (const ladder of params.ladders) {
if (signal?.aborted) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
})
}
const [width, height] = resolutionToDimensions(
ladder,
params.originalWidth,
params.originalHeight,
)
const targetBps = calculateEffectiveBitrateBps(
height,
width,
params.sourceVideoBitrate,
params.originalFps,
)
const ladderDir = path.join(hlsTmpDir, ladder)
await transcodeService.transcodeHlsRendition({
inputFile: params.filePath,
outputDir: ladderDir,
width,
height,
frameRate: params.originalFps,
hardwareAcceleration: params.hardwareAcceleration,
sourceVideoBitrate: params.sourceVideoBitrate,
threads: params.threads,
signal,
hdr: Boolean(params.sourceIsHdr),
sourceIsHdr: params.sourceIsHdr,
sourceHdrType: params.sourceHdrType,
sourceColorTransfer: params.sourceColorTransfer,
sourceColorPrimaries: params.sourceColorPrimaries,
sourceColorSpace: params.sourceColorSpace,
sourceRotation: params.sourceRotation,
streamIndex: params.streamIndex,
audioStreamIndex: params.audioStreamIndex,
})
renditionSpecs.push({
resolution: ladder,
width,
height,
bitrateBps: targetBps,
isHdr: params.sourceIsHdr,
})
mediaResolutions.push({
resolution: ladder,
width,
height,
hdr: Boolean(params.sourceIsHdr),
})
await ensureAssetNotPurging(params.assetKey)
const files = fs.readdirSync(ladderDir)
for (const file of files) {
const localFilePath = path.join(ladderDir, file)
const stat = fs.statSync(localFilePath)
const s3Key = path.posix.join(hlsDirKey, ladder, file)
let contentType = 'application/octet-stream'
if (file.endsWith('.m3u8')) {
contentType = 'application/vnd.apple.mpegurl'
} else if (file.endsWith('.mp4')) {
contentType = 'video/mp4'
} else if (file.endsWith('.m4s')) {
contentType = 'video/iso.segment'
}
const stream = Bun.file(localFilePath).stream()
await s3Service.putObject(bucket, s3Key, stream, stat.size, contentType)
}
}
const masterContent = buildHlsMasterPlaylist(renditionSpecs)
const masterBuffer = Buffer.from(masterContent, 'utf-8')
await ensureAssetNotPurging(params.assetKey)
await s3Service.putObject(
bucket,
masterKey,
masterBuffer,
masterBuffer.length,
'application/vnd.apple.mpegurl',
)
return {
key: masterKey,
resolutions: mediaResolutions,
}
} catch (err) {
const { code, message } = getErrorDetails(err)
const lowerMsg = message.toLowerCase()
if (
signal?.aborted ||
code === 'ABORT_ERR' ||
message.includes('aborted') ||
lowerMsg.includes('abort')
) {
throw ApplicationFailure.create({
message: 'HLS transcoding cancelled',
nonRetryable: true,
cause: err instanceof Error ? err : undefined,
})
}
if (
code === 'ENOENT' ||
lowerMsg.includes('enoent') ||
lowerMsg.includes('ffmpeg') ||
lowerMsg.includes('ffprobe') ||
lowerMsg.includes('spawn') ||
lowerMsg.includes('format') ||
lowerMsg.includes('no video stream found') ||
lowerMsg.includes('zscale') ||
lowerMsg.includes('tonemap')
) {
throw ApplicationFailure.create({
message: `HLS transcoding failed: ${message}`,
nonRetryable: true,
cause: err instanceof Error ? err : undefined,
})
}
throw err
} finally {
if (fs.existsSync(hlsTmpDir)) {
try {
fs.rmSync(hlsTmpDir, { recursive: true, force: true })
} catch {
// ignore cleanup error
}
}
}
}
export interface AudioActivityParams {
taskId?: string
assetKey: string
filePath: string
threads?: number
audioStreamIndex?: number
}
export async function transcodeAudioActivity(
@@ -419,6 +676,7 @@ export async function transcodeAudioActivity(
bitrate: '128k',
threads: params.threads,
signal,
audioStreamIndex: params.audioStreamIndex,
})
const stat = fs.statSync(outputFile)
@@ -518,6 +776,8 @@ export async function transcodeImageActivity(
lowerMsg.includes('imagemagick') ||
lowerMsg.includes('magick') ||
lowerMsg.includes('convert') ||
lowerMsg.includes('dcraw') ||
lowerMsg.includes('raw') ||
lowerMsg.includes('spawn') ||
lowerMsg.includes('format')
) {
@@ -590,6 +850,7 @@ export async function generateSpriteActivity(params: GenerateSpriteActivityParam
hdrType: params.mediaInfo.metadata?.hdrType,
colorTransfer: params.mediaInfo.metadata?.colorTransfer,
},
params.mediaInfo.metadata?.videoStreamIndex,
)
}
@@ -894,6 +1155,7 @@ export interface ExtractPosterActivityParams {
isHdr?: boolean
hdrType?: HdrType
colorTransfer?: string
streamIndex?: number
}
export async function extractPosterActivity(
@@ -925,6 +1187,7 @@ export async function extractPosterActivity(
hdrType: params.hdrType,
colorTransfer: params.colorTransfer,
signal,
streamIndex: params.streamIndex,
})
const posterBuffer = fs.readFileSync(posterFile)
@@ -1241,6 +1504,8 @@ export async function transcodeVideoChunkActivity(
params.startTime.toString(),
'-i',
inputSource,
'-map',
'0:V:0',
'-t',
(params.endTime - params.startTime).toString(),
'-vf',
@@ -45,26 +45,37 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
},
}))
vi.mock('@shumai/core/src/transcode/transcode', () => ({
transcodeService: {
getVideoInfo: vi.fn(),
getImageInfo: vi.fn(),
createTempDir: vi.fn().mockReturnValue('/tmp'),
removeDir: vi.fn(),
renderSvgToPng: vi.fn().mockResolvedValue(Buffer.from('fake-overlay-png')),
downscaleImageToPng: vi.fn().mockResolvedValue({
buffer: Buffer.from('fake-block-png'),
width: 32,
height: 32,
}),
compositeOverlayToWebpFile: vi.fn().mockImplementation(async (_in, _overlay, out) => {
fs.writeFileSync(out, 'fake-webp')
}),
transcodeVideo: vi.fn().mockImplementation(async (params) => {
fs.writeFileSync(params.outputFile, 'fake-mp4')
}),
},
}))
vi.mock('@shumai/core/src/transcode/transcode', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/core/src/transcode/transcode')>()
return {
...actual,
transcodeService: {
...actual.transcodeService,
getVideoInfo: vi.fn(),
getImageInfo: vi.fn(),
createTempDir: vi.fn().mockReturnValue('/tmp'),
removeDir: vi.fn(),
renderSvgToPng: vi.fn().mockResolvedValue(Buffer.from('fake-overlay-png')),
downscaleImageToPng: vi.fn().mockResolvedValue({
buffer: Buffer.from('fake-block-png'),
width: 32,
height: 32,
}),
compositeOverlayToWebpFile: vi.fn().mockImplementation(async (_in, _overlay, out) => {
fs.writeFileSync(out, 'fake-webp')
}),
transcodeVideo: vi.fn().mockImplementation(async (params) => {
fs.writeFileSync(params.outputFile, 'fake-mp4')
}),
transcodeHlsRendition: vi.fn().mockImplementation(async (params) => {
fs.mkdirSync(params.outputDir, { recursive: true })
fs.writeFileSync(path.join(params.outputDir, 'init.mp4'), 'fake-init')
fs.writeFileSync(path.join(params.outputDir, 'segment_000.m4s'), 'fake-segment')
fs.writeFileSync(path.join(params.outputDir, 'index.m3u8'), 'fake-index')
}),
},
}
})
describe('Watermark Activities', () => {
setupTestDbHooks()
@@ -524,6 +535,89 @@ describe('Watermark Activities', () => {
fs.rmSync(sdrOut, { force: true })
fs.rmSync(hdrOut, { force: true })
})
it('produces watermarked HLS renditions and updates mediaInfo.hls when original is HLS', async () => {
const originalMedia: PrismaJson.MediaInfo = {
proxyType: 'video',
duration: 10,
filesize: 1000,
frames: 300,
videoTranscodes: [
{ key: 'files/asset/video-1080p.mp4', width: 1920, height: 1080, resolution: '1080p' },
],
imageTranscodes: [],
videoPreview: { width: 1920, height: 1080 },
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
hasAudio: false,
format: {},
bitRate: 1000,
},
original: { key: 'files/asset/raw.mp4', filesizeInBytes: 1000, codec: 'h264' },
finishedAt: new Date().toISOString(),
isHls: true,
hls: {
key: 'files/asset/hls/master.m3u8',
resolutions: [{ resolution: '1080p', width: 1920, height: 1080 }],
},
}
const storageKey = await prisma.storageKey.create({
data: { key: 'files/asset/raw.mp4' },
})
const asset = await prisma.asset.create({
data: {
name: 'wm-hls.mp4',
type: 'file',
status: 'processed',
storageKeyId: storageKey.id,
mediaType: 'video/mp4',
media: originalMedia,
},
})
const config = await prisma.watermarkConfig.create({
data: {
hash: 'hls-hash-' + Math.random(),
config: {
blocks: [
{
id: 'b1',
type: 'text',
text: 'HLS Confidential',
opacity: 0.5,
size: 0.05,
color: '#FFFFFF',
x: 0.5,
y: 0.5,
rotation: 0,
},
],
},
},
})
const media = await transcodeWatermarkMediaActivity({
assetId: asset.id,
watermarkConfigId: config.id,
})
expect(media.isHls).toBe(true)
expect(media.hls).toBeDefined()
expect(media.hls?.key).toContain(`hls-watermark-${config.id}/master.m3u8`)
expect(media.hls?.resolutions).toHaveLength(1)
expect(media.hls?.resolutions?.[0]?.resolution).toBe('1080p')
expect(transcodeService.transcodeHlsRendition).toHaveBeenCalledWith(
expect.objectContaining({
width: 1920,
height: 1080,
}),
)
})
})
describe('completeWatermarkFileActivity', () => {
+120 -1
View File
@@ -1,7 +1,11 @@
import { prisma, WatermarkFileStatus } from '@shumai/db'
import '@shumai/db/src/prisma-json-types'
import { s3Service } from '@shumai/core/src/s3/s3'
import { transcodeService } from '@shumai/core/src/transcode/transcode'
import {
transcodeService,
buildHlsMasterPlaylist,
calculateEffectiveBitrateBps,
} from '@shumai/core/src/transcode/transcode'
import {
generateWatermarkSvg,
RenderBlockImageData,
@@ -366,6 +370,8 @@ export async function transcodeWatermarkMediaActivity(
sourceColorTransfer: colorTransfer ?? originalMedia?.metadata?.colorTransfer,
sourceColorPrimaries: colorPrimaries ?? originalMedia?.metadata?.colorPrimaries,
sourceColorSpace: colorSpace ?? originalMedia?.metadata?.colorSpace,
streamIndex: originalMedia?.metadata?.videoStreamIndex,
audioStreamIndex: originalMedia?.metadata?.audioStreamIndex,
})
const stat = fs.statSync(outFilePath)
@@ -388,6 +394,119 @@ export async function transcodeWatermarkMediaActivity(
hdr: isVtHdr,
})
}
if (originalMedia?.isHls && originalMedia?.hls?.resolutions?.length) {
const hlsWatermarkDirKey = path.posix.join(
path.posix.dirname(assetKey),
`hls-watermark-${params.watermarkConfigId}`,
)
const masterKey = path.posix.join(hlsWatermarkDirKey, 'master.m3u8')
const hlsTmpDir = path.join(tmpDir, `hls-watermark-${params.watermarkConfigId}`)
fs.mkdirSync(hlsTmpDir, { recursive: true })
const renditionSpecs: Array<{
resolution: string
width: number
height: number
bitrateBps: number
isHdr?: boolean
}> = []
const hlsResolutions: PrismaJson.VideoTranscode[] = []
for (const vt of originalMedia.hls.resolutions) {
const ladder = vt.resolution || `${vt.height}p`
const targetWidth = vt.width || originalWidth
const targetHeight = vt.height || originalHeight
const isVtHdr = Boolean(vt.hdr)
const svgString = generateWatermarkSvg(config, targetWidth, targetHeight, blockImagesMap)
const overlayPngBuffer = await transcodeService.renderSvgToPng(svgString)
const overlayPngPath = path.join(tmpDir, `watermarkOverlay-hls-${ladder}.png`)
fs.writeFileSync(overlayPngPath, overlayPngBuffer)
const ladderDir = path.join(hlsTmpDir, ladder)
await transcodeService.transcodeHlsRendition({
inputFile: rawFilePath,
outputDir: ladderDir,
width: targetWidth,
height: targetHeight,
overlayFile: overlayPngPath,
frameRate,
hardwareAcceleration,
sourceVideoBitrate,
threads,
hdr: isVtHdr,
disableAudio: !hasAudio,
sourceIsHdr: isHdr ?? originalMedia?.metadata?.isHdr,
sourceHdrType: hdrType ?? originalMedia?.metadata?.hdrType,
sourceColorTransfer: colorTransfer ?? originalMedia?.metadata?.colorTransfer,
sourceColorPrimaries: colorPrimaries ?? originalMedia?.metadata?.colorPrimaries,
sourceColorSpace: colorSpace ?? originalMedia?.metadata?.colorSpace,
streamIndex: originalMedia?.metadata?.videoStreamIndex,
audioStreamIndex: originalMedia?.metadata?.audioStreamIndex,
})
const targetBps = calculateEffectiveBitrateBps(
targetHeight,
targetWidth,
sourceVideoBitrate,
frameRate,
)
renditionSpecs.push({
resolution: ladder,
width: targetWidth,
height: targetHeight,
bitrateBps: targetBps,
isHdr: isVtHdr,
})
const ladderFiles = fs.readdirSync(ladderDir)
for (const file of ladderFiles) {
const filePath = path.join(ladderDir, file)
const stat = fs.statSync(filePath)
totalSize += stat.size
const contentType = file.endsWith('.m3u8')
? 'application/vnd.apple.mpegurl'
: file.endsWith('.m4s')
? 'video/iso.segment'
: 'video/mp4'
const s3Key = path.posix.join(hlsWatermarkDirKey, ladder, file)
await s3Service.putObject(
bucket,
s3Key,
Bun.file(filePath).stream(),
stat.size,
contentType,
)
}
hlsResolutions.push({
key: path.posix.join(hlsWatermarkDirKey, ladder, 'index.m3u8'),
width: targetWidth,
height: targetHeight,
resolution: ladder,
hdr: isVtHdr,
})
}
const masterM3u8Content = buildHlsMasterPlaylist(renditionSpecs)
const masterM3u8Buffer = Buffer.from(masterM3u8Content, 'utf-8')
await s3Service.putObject(
bucket,
masterKey,
masterM3u8Buffer,
masterM3u8Buffer.length,
'application/vnd.apple.mpegurl',
)
mediaInfo.isHls = true
mediaInfo.hls = {
key: masterKey,
resolutions: hlsResolutions,
}
}
mediaInfo.filesize = totalSize
}
+17
View File
@@ -22,6 +22,13 @@ export class VideoTranscoder {
return this
}
setVideoResolutions(resolutions?: PrismaJson.VideoResolutionLadder[]): this {
if (resolutions) {
this.spec.videoResolutions = resolutions
}
return this
}
setHardwareAcceleration(hardwareAcceleration?: PrismaJson.HardwareAcceleration): this {
if (hardwareAcceleration) {
this.spec.hardwareAcceleration = hardwareAcceleration
@@ -46,6 +53,16 @@ export class VideoTranscoder {
return this
}
setHls(enabled?: boolean, resolutions?: PrismaJson.HlsResolutionLadder[]): this {
if (enabled !== undefined) {
this.spec.hlsEnabled = enabled
}
if (resolutions) {
this.spec.hlsResolutions = resolutions
}
return this
}
async submit(): Promise<string> {
const task = await this.db.workflowTask.create({
data: {
+20 -3
View File
@@ -38,14 +38,31 @@ export async function completeTask(
})
}
export async function failTask(workerQueue: string, taskId: string, err: unknown) {
const { updateTaskStatusActivity } = getActivities()
export async function failTask(
workerQueue: string,
taskId: string,
err: unknown,
assetId?: string,
) {
const { updateTaskStatusActivity, updateAssetStatusActivity } = getActivities()
if (workerQueue) {
const errorMsg = err instanceof Error ? err.message : String(err)
await executeActivity(workerQueue, updateTaskStatusActivity, {
taskId,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
output: { error: errorMsg },
})
if (assetId) {
try {
await executeActivity(workerQueue, updateAssetStatusActivity, {
assetId,
status: 'failed',
error: errorMsg,
})
} catch (assetErr) {
console.error('Failed to update asset status to failed:', assetErr)
}
}
}
}
@@ -155,4 +155,73 @@ describe('transcodeImageWorkflow', () => {
projectId: 'proj-1',
})
})
it('should handle RAW image without usable preview by skipping transcodes and marking processed', async () => {
const task: WorkflowTask = {
id: 'task-raw-nopreview',
assetId: 'asset-raw',
type: WorkflowTaskType.transcode_image,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
thumbnail: true,
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-raw',
storageKey: { key: 'photo.cr2' },
mediaType: 'image/x-canon-cr2',
})
// RAW file without usable preview returns originalWidth: 0, originalHeight: 0
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'image',
metadata: {
originalWidth: 0,
originalHeight: 0,
duration: 0,
frameRate: 0,
totalFrames: 0,
startTimecode: undefined,
bitRate: 0,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
await transcodeImageWorkflow(task)
// transcodeImageActivity should NOT be called since hasUsablePreview is false
expect(mockActivities.transcodeImageActivity).not.toHaveBeenCalled()
// Asset media should still be updated
expect(mockActivities.updateAssetMediaActivity).toHaveBeenCalledWith({
assetId: 'asset-raw',
mediaInfo: expect.objectContaining({
imageTranscodes: [],
}),
})
// Asset should still be marked as processed
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-raw',
status: AssetStatus.processed,
})
})
})
@@ -12,6 +12,7 @@ import {
export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -54,9 +55,10 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
}
const metadata = mediaInfo.metadata
const hasUsablePreview = metadata && metadata.originalWidth > 0 && metadata.originalHeight > 0
const isImage = mediaInfo.proxyType === 'image'
if (isImage && metadata) {
if (isImage && hasUsablePreview && metadata) {
const imageSpec: PrismaJson.ImageTranscode = {
width: metadata.originalWidth,
height: metadata.originalHeight,
@@ -71,7 +73,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
mediaInfo.imageTranscodes.push(imageTranscode)
}
if (spec.thumbnail) {
if (spec.thumbnail && hasUsablePreview) {
const thumbTranscode = await executeActivity(workerQueue, transcodeImageActivity, {
assetKey: key,
filePath,
@@ -89,6 +91,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -105,7 +108,7 @@ export async function transcodeImageWorkflow(task: WorkflowTask): Promise<void>
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodeImageWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
@@ -12,6 +12,7 @@ import {
export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -101,6 +102,7 @@ export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -117,7 +119,7 @@ export async function transcodePdfWorkflow(task: WorkflowTask): Promise<void> {
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodePdfWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
@@ -0,0 +1,173 @@
import { describe, it, expect } from 'vitest'
import {
getTargetHlsResolutions,
resolutionToDimensions,
getTargetVideoResolutions,
isMimePsd,
} from './transcode-utils'
describe('transcode-utils', () => {
describe('getTargetHlsResolutions', () => {
it('uses default ladders when configuredLadders is undefined or empty', () => {
// 1080p source video (1920x1080)
const res1 = getTargetHlsResolutions(undefined, 1920, 1080)
expect(res1).toEqual(['1080p', '720p', '480p'])
const res2 = getTargetHlsResolutions([], 1920, 1080)
expect(res2).toEqual(['1080p', '720p', '480p'])
})
it('caps ladders up to source resolution (no upscaling)', () => {
// 1080p source with 4K configured ladders -> should cap to 1080p
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 1920, 1080)
expect(res).toEqual(['1080p', '720p', '480p'])
})
it('includes higher ladders when source resolution supports them', () => {
// 4K source (3840x2160)
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 3840, 2160)
expect(res).toEqual(['2160p', '1440p', '1080p', '720p', '480p'])
// 1440p source (2560x1440)
const res1440 = getTargetHlsResolutions(
['480p', '720p', '1080p', '1440p', '2160p'],
2560,
1440,
)
expect(res1440).toEqual(['1440p', '1080p', '720p', '480p'])
})
it('handles vertical videos correctly by using the longest side', () => {
// 1080x1920 (vertical 1080p)
const res = getTargetHlsResolutions(['480p', '720p', '1080p', '1440p', '2160p'], 1080, 1920)
expect(res).toEqual(['1080p', '720p', '480p'])
})
it('falls back to lowest selected ladder when source is smaller than smallest ladder', () => {
// 640x360 source is smaller than 480p (854 long side) -> falls back to lowest selected (480p)
const res = getTargetHlsResolutions(undefined, 640, 360)
expect(res).toEqual(['480p'])
// When user selected 720p and 1440p, and source is 480p (854 long side) -> falls back to lowest selected (720p)
const resCustom = getTargetHlsResolutions(['720p', '1440p'], 854, 480)
expect(resCustom).toEqual(['720p'])
})
it('respects selective subset of configured ladders', () => {
// User only selected 720p and 1440p
const res = getTargetHlsResolutions(['720p', '1440p'], 2560, 1440)
expect(res).toEqual(['1440p', '720p'])
})
})
describe('resolutionToDimensions', () => {
it('calculates dimensions correctly for 1440p and 480p', () => {
const [w1440, h1440] = resolutionToDimensions('1440p', 3840, 2160)
expect(w1440).toBe(2560)
expect(h1440).toBe(1440)
const [w480, h480] = resolutionToDimensions('480p', 1920, 1080)
expect(w480).toBe(854)
expect(h480).toBe(480)
})
it('ensures even dimensions for odd calculations', () => {
const [w, h] = resolutionToDimensions('720p', 1921, 1081)
expect(w % 2).toBe(0)
expect(h % 2).toBe(0)
})
it('returns [0, 0] for unknown resolution', () => {
expect(resolutionToDimensions('invalid', 1920, 1080)).toEqual([0, 0])
})
})
describe('getTargetVideoResolutions', () => {
describe('best_match strategy', () => {
it('returns best match resolution for 4K video', () => {
const resolutions = getTargetVideoResolutions('best_match', 3840, 2160)
expect(resolutions).toEqual(['180p', '2160p'])
})
it('returns best match resolution for 1080p video', () => {
const resolutions = getTargetVideoResolutions('best_match', 1920, 1080)
expect(resolutions).toEqual(['180p', '1080p'])
})
it('returns best match resolution for 720p video', () => {
const resolutions = getTargetVideoResolutions('best_match', 1280, 720)
expect(resolutions).toEqual(['180p', '720p'])
})
it('falls back to 480p for smaller videos (460p and 230p)', () => {
const res460 = getTargetVideoResolutions('best_match', 818, 460)
expect(res460).toEqual(['180p', '480p'])
const res230 = getTargetVideoResolutions('best_match', 408, 230)
expect(res230).toEqual(['180p', '480p'])
})
})
describe('multi strategy', () => {
it('returns all supported resolutions up to source for 4K video', () => {
const resolutions = getTargetVideoResolutions('multi', 3840, 2160)
expect(resolutions).toEqual(['180p', '2160p', '1440p', '1080p', '720p', '480p'])
})
it('returns resolutions up to 720p for 1280x720 video', () => {
const resolutions = getTargetVideoResolutions('multi', 1280, 720)
expect(resolutions).toEqual(['180p', '720p', '480p'])
})
it('returns resolutions up to 1080p for 2276x1280 video', () => {
const resolutions = getTargetVideoResolutions('multi', 2276, 1280)
expect(resolutions).toEqual(['180p', '1080p', '720p', '480p'])
})
it('falls back to lowest selected resolution (480p) for 460p and 230p videos', () => {
const res460 = getTargetVideoResolutions('multi', 818, 460)
expect(res460).toEqual(['180p', '480p'])
const res230 = getTargetVideoResolutions('multi', 408, 230)
expect(res230).toEqual(['180p', '480p'])
})
it('respects user-selected ladders and falls back to lowest selected ladder', () => {
const customLadders = ['720p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[]
// 4K video: matches 2160p, 1440p, 720p (1080p was not selected)
const res4k = getTargetVideoResolutions('multi', 3840, 2160, customLadders)
expect(res4k).toEqual(['180p', '2160p', '1440p', '720p'])
// 230p video: nothing matches <= 408 -> falls back to lowest selected (720p)
const res230 = getTargetVideoResolutions('multi', 408, 230, customLadders)
expect(res230).toEqual(['180p', '720p'])
})
it('normalizes legacy "all" and "full" strategy values to multi', () => {
const resAll = getTargetVideoResolutions(
'all' as unknown as PrismaJson.VideoTranscodeStrategy,
1280,
720,
)
expect(resAll).toEqual(['180p', '720p', '480p'])
const resFull = getTargetVideoResolutions(
'full' as unknown as PrismaJson.VideoTranscodeStrategy,
1280,
720,
)
expect(resFull).toEqual(['180p', '720p', '480p'])
})
})
})
describe('isMimePsd', () => {
it('identifies PSD mime types', () => {
expect(isMimePsd('image/vnd.adobe.photoshop')).toBe(true)
expect(isMimePsd('image/x-photoshop')).toBe(true)
expect(isMimePsd('image/psd')).toBe(true)
expect(isMimePsd('image/png')).toBe(false)
})
})
})
@@ -13,13 +13,67 @@ export function isMimePsd(mimeType: string): boolean {
const RESOLUTION_LONG_SIDES: Record<string, number> = {
'2160p': 3840,
'1440p': 2560,
'1080p': 1920,
'720p': 1280,
'540p': 960,
'480p': 854,
'360p': 640,
'180p': 320,
}
export const HLS_LADDER_ORDER: PrismaJson.HlsResolutionLadder[] = [
'2160p',
'1440p',
'1080p',
'720p',
'480p',
]
export const VIDEO_LADDER_ORDER: PrismaJson.VideoResolutionLadder[] = [
'2160p',
'1440p',
'1080p',
'720p',
'480p',
]
export function matchResolutionsByLongSide<T extends string>(
selectedLadders: T[],
ladderOrder: T[],
rawLongSide: number,
): T[] {
const orderedSelected = ladderOrder.filter((ladder) => selectedLadders.includes(ladder))
if (orderedSelected.length === 0) return []
const matched = orderedSelected.filter((ladder) => {
const targetLongSide = RESOLUTION_LONG_SIDES[ladder]
return targetLongSide !== undefined && targetLongSide <= rawLongSide
})
if (matched.length === 0) {
// Falls back to the lowest resolution selected if nothing matched
return [orderedSelected[orderedSelected.length - 1]]
}
return matched
}
export function getTargetHlsResolutions(
configuredLadders: PrismaJson.HlsResolutionLadder[] | undefined,
originalWidth: number,
originalHeight: number,
): PrismaJson.HlsResolutionLadder[] {
const selectedLadders =
configuredLadders && configuredLadders.length > 0
? configuredLadders
: (['480p', '720p', '1080p'] as PrismaJson.HlsResolutionLadder[])
const rawLongSide = Math.max(originalWidth, originalHeight)
return matchResolutionsByLongSide(selectedLadders, HLS_LADDER_ORDER, rawLongSide)
}
export function resolutionToDimensions(
resolution: string,
originalWidth: number,
@@ -45,31 +99,11 @@ export function resolutionToDimensions(
return [width, height]
}
const TARGET_RESOLUTIONS = [
{ name: '2160p', longSide: 3840 },
{ name: '1080p', longSide: 1920 },
{ name: '720p', longSide: 1280 },
{ name: '540p', longSide: 960 },
{ name: '360p', longSide: 640 },
]
function getBestMatchResolution(
originalWidth: number,
originalHeight: number,
): { name: string; longSide: number } {
const rawLongSide = Math.max(originalWidth, originalHeight)
const lower = TARGET_RESOLUTIONS.filter((r) => r.longSide <= rawLongSide)
if (lower.length > 0) {
lower.sort((a, b) => b.longSide - a.longSide)
return lower[0]
}
return TARGET_RESOLUTIONS[TARGET_RESOLUTIONS.length - 1]
}
export function getTargetVideoResolutions(
strategy: PrismaJson.VideoTranscodeStrategy,
originalWidth: number,
originalHeight: number,
configuredResolutions?: PrismaJson.VideoResolutionLadder[],
): string[] {
const resolutions = ['180p']
@@ -77,21 +111,27 @@ export function getTargetVideoResolutions(
const stratStr = strategy as string
if (stratStr === 'single' || stratStr === 'disable') {
normalizedStrategy = 'best_match'
} else if (stratStr === 'full') {
normalizedStrategy = 'all'
} else if (stratStr === 'full' || stratStr === 'all' || stratStr === 'multi') {
normalizedStrategy = 'multi'
}
const bestMatch = getBestMatchResolution(originalWidth, originalHeight)
const selectedLadders =
configuredResolutions && configuredResolutions.length > 0
? configuredResolutions
: (['480p', '720p', '1080p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[])
const rawLongSide = Math.max(originalWidth, originalHeight)
if (normalizedStrategy === 'best_match') {
resolutions.push(bestMatch.name)
} else if (normalizedStrategy === 'all') {
const bestMatchIndex = TARGET_RESOLUTIONS.findIndex((r) => r.name === bestMatch.name)
if (bestMatchIndex !== -1) {
for (let i = bestMatchIndex; i < TARGET_RESOLUTIONS.length; i++) {
resolutions.push(TARGET_RESOLUTIONS[i].name)
}
}
const matched = matchResolutionsByLongSide(
['480p', '720p', '1080p', '1440p', '2160p'] as PrismaJson.VideoResolutionLadder[],
VIDEO_LADDER_ORDER,
rawLongSide,
)
resolutions.push(matched[0])
} else if (normalizedStrategy === 'multi') {
const matched = matchResolutionsByLongSide(selectedLadders, VIDEO_LADDER_ORDER, rawLongSide)
resolutions.push(...matched)
}
return resolutions
@@ -25,6 +25,7 @@ describe('transcodeVideoWorkflow', () => {
getAssetActivity: Object.assign(vi.fn(), { _activityName: 'getAssetActivity' }),
getMediaInfoActivity: Object.assign(vi.fn(), { _activityName: 'getMediaInfoActivity' }),
transcodeVideoActivity: Object.assign(vi.fn(), { _activityName: 'transcodeVideoActivity' }),
transcodeHlsActivity: Object.assign(vi.fn(), { _activityName: 'transcodeHlsActivity' }),
transcodeAudioActivity: Object.assign(vi.fn(), { _activityName: 'transcodeAudioActivity' }),
transcodeImageActivity: Object.assign(vi.fn(), { _activityName: 'transcodeImageActivity' }),
generateSpriteActivity: Object.assign(vi.fn(), { _activityName: 'generateSpriteActivity' }),
@@ -702,4 +703,300 @@ describe('transcodeVideoWorkflow', () => {
status: WorkflowTaskStatus.completed,
})
})
it('should trigger HLS transcoding when hlsEnabled is true', async () => {
const task: WorkflowTask = {
id: 'task-hls',
assetId: 'asset-hls',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
videoStrategy: 'best_match',
hlsEnabled: true,
hlsResolutions: ['480p', '720p', '1080p'],
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid-hls',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-hls',
storageKey: { key: 'files/asset-hls/video.mp4' },
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
videoBitRate: 850000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockResolvedValue({
key: 'files/asset-hls/video-1080p.mp4',
width: 1920,
height: 1080,
resolution: '1080p',
})
mockActivities.transcodeHlsActivity.mockResolvedValue({
key: 'files/asset-hls/hls/master.m3u8',
resolutions: ['1080p', '720p', '480p'],
})
await transcodeVideoWorkflow(task)
expect(mockActivities.transcodeHlsActivity).toHaveBeenCalledWith(
expect.objectContaining({
taskId: 'task-hls',
assetKey: 'files/asset-hls/video.mp4',
filePath: '/tmp/video.mp4',
ladders: ['1080p', '720p', '480p'],
originalWidth: 1920,
originalHeight: 1080,
}),
)
expect(mockActivities.updateAssetMediaActivity).toHaveBeenCalledWith({
assetId: 'asset-hls',
mediaInfo: expect.objectContaining({
isHls: true,
hls: {
key: 'files/asset-hls/hls/master.m3u8',
resolutions: ['1080p', '720p', '480p'],
},
}),
})
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-hls',
status: WorkflowTaskStatus.completed,
})
})
it('should transcode selected multi resolutions when videoStrategy is multi', () => {
const task: WorkflowTask = {
id: 'task-multi',
assetId: 'asset-multi',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: {
videoStrategy: 'multi',
videoResolutions: ['720p', '1080p', '2160p'],
},
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-multi',
storageKey: { key: 'files/asset-multi/video.mp4' },
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockImplementation(async (params) => ({
key: `files/asset-multi/${params.videoSpec.resolution}.mp4`,
width: params.videoSpec.width,
height: params.videoSpec.height,
resolution: params.videoSpec.resolution,
}))
return transcodeVideoWorkflow(task).then(() => {
// 180p preview + 1080p + 720p (2160p is skipped since > 1920)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledTimes(3)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '180p' }),
}),
)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '1080p' }),
}),
)
expect(mockActivities.transcodeVideoActivity).toHaveBeenCalledWith(
expect.objectContaining({
videoSpec: expect.objectContaining({ resolution: '720p' }),
}),
)
})
})
it('should mark task and asset as failed when an activity fails', async () => {
const task: WorkflowTask = {
id: 'task-fail',
assetId: 'asset-fail',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: { videoStrategy: 'best_match' },
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-fail',
name: 'fail.mp4',
storageKey: { key: 'files/asset-fail/fail.mp4' },
status: AssetStatus.uploading,
})
mockActivities.getMediaInfoActivity.mockRejectedValue(new Error('FFprobe failed to read media'))
await expect(transcodeVideoWorkflow(task)).rejects.toThrow('FFprobe failed to read media')
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-fail',
status: 'failed',
output: { error: 'FFprobe failed to read media' },
})
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-fail',
status: 'failed',
error: 'FFprobe failed to read media',
})
})
it('should mark task as failed but keep asset as processed when post-transcode activity fails', async () => {
const task: WorkflowTask = {
id: 'task-post-fail',
assetId: 'asset-post-fail',
type: WorkflowTaskType.transcode_video,
status: WorkflowTaskStatus.pending,
sessionId: null,
output: null,
payload: {
projectId: 'proj-1',
transcode: { videoStrategy: 'best_match' },
},
createdAt: new Date(),
updatedAt: new Date(),
heartbeat: null,
teamId: 'team-1',
projectId: 'proj-1',
uid: 'task-uid',
model: null,
inputTokens: 0,
outputTokens: 0,
}
mockActivities.getAssetActivity.mockResolvedValue({
id: 'asset-post-fail',
name: 'video.mp4',
storageKey: { key: 'files/asset-post-fail/video.mp4' },
status: AssetStatus.uploading,
mediaType: 'video/mp4',
})
mockActivities.getMediaInfoActivity.mockResolvedValue({
proxyType: 'video',
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
startTimecode: '00:00:00:00',
bitRate: 1000,
hasAudio: false,
format: {},
},
videoTranscodes: [],
imageTranscodes: [],
})
mockActivities.transcodeVideoActivity.mockResolvedValue({
key: 'v.mp4',
width: 1920,
height: 1080,
})
mockActivities.createEmbeddingTaskIfEnabledActivity.mockRejectedValue(
new Error('Embedding creation failed'),
)
await expect(transcodeVideoWorkflow(task)).rejects.toThrow('Embedding creation failed')
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: 'task-post-fail',
status: 'failed',
output: { error: 'Embedding creation failed' },
})
expect(mockActivities.updateAssetStatusActivity).toHaveBeenCalledWith({
assetId: 'asset-post-fail',
status: 'processed',
})
expect(mockActivities.updateAssetStatusActivity).not.toHaveBeenCalledWith(
expect.objectContaining({
status: 'failed',
}),
)
})
})
@@ -8,11 +8,16 @@ import {
failTask,
cleanupTmpDir,
} from './common'
import { getTargetVideoResolutions, resolutionToDimensions } from './transcode-utils'
import {
getTargetVideoResolutions,
getTargetHlsResolutions,
resolutionToDimensions,
} from './transcode-utils'
export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void> {
let tmpDir: string | undefined
let workerQueue = ''
let mediaProcessed = false
try {
workerQueue = await getWorkerQueueAndStartTask(task)
@@ -21,6 +26,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
updateAssetStatusActivity,
getMediaInfoActivity,
transcodeVideoActivity,
transcodeHlsActivity,
transcodeAudioActivity,
transcodeImageActivity,
extractPosterActivity,
@@ -59,7 +65,9 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
codec: '',
}
if (spec.poster) {
const isVideo = mediaInfo.proxyType === 'video'
if (spec.poster && isVideo) {
const lastSlashIndex = key.lastIndexOf('/')
const assetDir = lastSlashIndex === -1 ? '' : key.substring(0, lastSlashIndex)
const posterSpec: PrismaJson.PosterInfo = {
@@ -72,6 +80,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
isHdr: mediaInfo.metadata?.isHdr,
hdrType: mediaInfo.metadata?.hdrType,
colorTransfer: mediaInfo.metadata?.colorTransfer,
streamIndex: mediaInfo.metadata?.videoStreamIndex,
})
mediaInfo.poster = posterResult.poster
await executeActivity(workerQueue, updateAssetMediaActivity, {
@@ -82,7 +91,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
// Generate the sprite (and its poster fallback) before the expensive proxy transcodes so the
// file list can show a poster/sprite preview while the asset is still processing.
if (spec.sprite || (spec.poster && !mediaInfo.poster)) {
if ((spec.sprite || (spec.poster && !mediaInfo.poster)) && isVideo) {
const lastSlashIndex = key.lastIndexOf('/')
const assetDir = lastSlashIndex === -1 ? '' : key.substring(0, lastSlashIndex)
@@ -119,11 +128,12 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
const metadata = mediaInfo.metadata
if (mediaInfo.proxyType === 'video' && metadata) {
if (isVideo && metadata) {
const videoResolutions = getTargetVideoResolutions(
spec.videoStrategy || 'best_match',
metadata.originalWidth,
metadata.originalHeight,
spec.videoResolutions,
)
for (const res of videoResolutions) {
const [width, height] = resolutionToDimensions(
@@ -157,6 +167,9 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
sourceRotation: metadata.rotation ?? 0,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoPreview = videoTranscode
@@ -185,10 +198,46 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
// rotation is undefined when the probed source has none
sourceRotation: metadata.rotation ?? 0,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoTranscodes.push(videoTranscode)
}
if (spec.hlsEnabled) {
const hlsLadders = getTargetHlsResolutions(
spec.hlsResolutions,
metadata.originalWidth,
metadata.originalHeight,
)
if (hlsLadders.length > 0) {
const hlsResult = await executeActivity(workerQueue, transcodeHlsActivity, {
taskId: task.id,
assetKey: key,
filePath,
ladders: hlsLadders,
originalWidth: metadata.originalWidth,
originalHeight: metadata.originalHeight,
originalFps: metadata.frameRate,
sourceVideoBitrate: metadata.videoBitRate || metadata.bitRate,
hardwareAcceleration: spec.hardwareAcceleration,
threads: spec.threads,
sourceIsHdr: metadata.isHdr,
sourceHdrType: metadata.hdrType,
sourceColorTransfer: metadata.colorTransfer,
sourceColorPrimaries: metadata.colorPrimaries,
sourceColorSpace: metadata.colorSpace,
sourceRotation: metadata.rotation ?? 0,
streamIndex: metadata.videoStreamIndex,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.isHls = true
mediaInfo.hls = hlsResult
}
}
}
const isAudio = mediaInfo.proxyType === 'audio'
@@ -198,11 +247,12 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
assetKey: key,
filePath,
threads: spec.threads,
audioStreamIndex: metadata.audioStreamIndex,
})
mediaInfo.videoTranscodes.push(audioTranscode)
}
if (spec.thumbnail) {
if (spec.thumbnail && isVideo) {
const thumbTranscode = await executeActivity(workerQueue, transcodeImageActivity, {
assetKey: key,
filePath,
@@ -220,6 +270,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
assetId: asset.id,
status: 'processed',
})
mediaProcessed = true
await executeActivity(workerQueue, createEmbeddingTaskIfEnabledActivity, {
assetId: asset.id,
@@ -236,7 +287,7 @@ export async function transcodeVideoWorkflow(task: WorkflowTask): Promise<void>
await completeTask(workerQueue, task.id)
} catch (err) {
console.error(`transcodeVideoWorkflow failed for task ${task.id}:`, err)
await failTask(workerQueue, task.id, err)
await failTask(workerQueue, task.id, err, mediaProcessed ? undefined : task.assetId)
throw err
} finally {
await cleanupTmpDir(workerQueue, tmpDir)
+7 -2
View File
@@ -51,9 +51,14 @@ export function AuthLayout({ children }: { children: ReactNode }) {
{/* Footer Label */}
<div className="relative pt-4 border-t border-zinc-200/30 dark:border-zinc-800/30">
<p className="text-xs text-zinc-500 dark:text-zinc-400 font-medium">
<a
href="https://discord.gg/sUzhujK6DD"
target="_blank"
rel="noreferrer"
className="text-xs text-zinc-500 hover:text-zinc-800 dark:text-zinc-400 dark:hover:text-zinc-200 font-medium transition-colors hover:underline inline-flex items-center gap-1"
>
{m.auth_community_cta()}
</p>
</a>
</div>
</div>
@@ -1,6 +1,6 @@
// @vitest-environment happy-dom
import React from 'react'
import { cleanup, render, screen } from '@testing-library/react'
import { cleanup, fireEvent, render, screen } from '@testing-library/react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { BreadcrumbNav } from './breadcrumb-nav'
@@ -160,4 +160,107 @@ describe('BreadcrumbNav component', () => {
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
expect(triggerButton).toBeDefined()
})
it('displays 480p instead of 360p for 480p video transcode in download menu', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-1"
currentAsset={{ id: 'file-1', name: 'sample.mov', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'sample-480p.mp4',
width: 854,
height: 480,
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('displays explicit resolution when provided in download menu', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-1"
currentAsset={{ id: 'file-1', name: 'sample.mov', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'sample-480p.mp4',
width: 854,
height: 480,
resolution: '480p',
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'sample.mov' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('correctly labels legacy portrait video transcode in download menu using long side fallback', () => {
render(
<QueryClientProvider client={queryClient}>
<BreadcrumbNav
{...baseProps}
fileId="file-portrait"
currentAsset={{ id: 'file-portrait', name: 'portrait.mp4', type: 'file', version: 1 }}
allowDownload={true}
downloadInfo={{
originalKey: 'raw.mp4',
videoTranscodes: [
{
key: 'portrait-legacy.mp4',
width: 1080,
height: 1920,
hdr: false,
},
],
}}
/>
</QueryClientProvider>,
)
const triggerButton = screen.getByRole('button', { name: 'portrait.mp4' })
fireEvent.pointerDown(triggerButton, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('1080p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('1920p'))).toBe(false)
})
})
+3 -8
View File
@@ -48,6 +48,7 @@ import { useState } from 'react'
import { useDualSidebarStore } from '@/ui/stores/dual-sidebar'
import { ManageVersionsDialog } from './manage-versions-dialog'
import { AssetLinkedTasksDialog } from './kanban/asset-linked-tasks-dialog'
import { getVideoResolutionLabel } from '@/ui/lib/media'
interface BreadcrumbNavProps {
teamId: string
@@ -84,6 +85,7 @@ interface BreadcrumbNavProps {
key: string
width: number
height: number
resolution?: string
hdr?: boolean
}>
}
@@ -287,14 +289,7 @@ export function BreadcrumbNav({
<DropdownMenuSubContent className="w-48">
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
{downloadInfo?.videoTranscodes?.map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const resolution = getVideoResolutionLabel(t)
const label = t.hdr ? `${resolution} (HDR)` : resolution
return (
<DropdownMenuItem
@@ -66,6 +66,9 @@ export function CompareControlBar({
showFrames: false,
currentResolution: v.currentResolution,
isCurrentHdr: v.isCurrentHdr,
activeAutoResolution: v.activeAutoResolution,
isHlsManualSupported: v.isHlsManualSupported,
isLoading: v.isLoading,
}
// The active-side asset carries the metadata used for timecode display and
+13 -3
View File
@@ -1,10 +1,17 @@
import type { VideoTranscode } from '@shumai/dtos'
export type PaneKind = 'image' | 'video' | 'unsupported'
export type CompareSide = 'left' | 'right'
export type DisplayTranscode = VideoTranscode & { resolution: string }
export interface DisplayTranscode {
id?: string
url?: string
key?: string
width: number
height: number
size?: number
hdr?: boolean
resolution: string
}
/**
* Reactive state a pane reports upward so the shared control bar can render.
@@ -26,6 +33,9 @@ export interface PaneReportedState {
isCurrentHdr?: boolean
resolutions: DisplayTranscode[]
buffered: number
activeAutoResolution?: string
isHlsManualSupported?: boolean
isLoading?: boolean
}
}
@@ -1258,7 +1258,8 @@ export function FileBrowser({
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={() => {
onClick={(e) => {
e.preventDefault()
emptyTrash()
}}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
@@ -382,4 +382,63 @@ describe('FileCard', () => {
// The duration badge appears once processing is done.
expect(screen.getByText('00:10')).toBeTruthy()
})
it('renders failed state with alert icon and shows creator at bottom when status is failed without a preview', () => {
const failedItem: AssetInfo = {
...fileItem,
status: 'failed',
creator: { id: 'u1', name: 'Alice' },
} as AssetInfo
renderComponent({ item: failedItem })
expect(screen.queryByTestId('file-card-preparing-circle')).toBeNull()
const failedState = screen.getByTestId('file-card-failed-state')
expect(failedState).toBeTruthy()
expect(screen.getByText(/Processing failed|处理失败/i)).toBeTruthy()
expect(screen.getByText(/Alice/i)).toBeTruthy()
})
it('renders thumbnail with failed badge in top-right corner, shows error in tooltip on hover, and shows creator at bottom', () => {
vi.useFakeTimers()
const failedItemWithPreview: AssetInfo = {
...fileItem,
status: 'failed',
creator: { id: 'u1', name: 'Alice' },
media: {
error: 'FFmpeg transcode failed: exit code 1',
},
preview: {
proxyType: 'video',
thumbnailUrl: 'https://example.com/poster.webp',
duration: 10,
},
} as AssetInfo
renderComponent({ item: failedItemWithPreview })
const media = screen.getByTestId('file-card-preview-media')
expect(media.className).not.toContain('animate-pulse')
const badge = screen.getByTestId('file-card-failed-badge')
expect(badge).toBeTruthy()
expect(badge.textContent).toMatch(/Transcoding Failed|转码失败/i)
expect(badge.querySelector('svg')).toBeNull()
// Failed badge is in the top-right container
const rightTopContainer = badge.closest('.absolute.right-2.top-2')
expect(rightTopContainer).toBeTruthy()
// Creator is still shown at the bottom of the card
expect(screen.getByText(/Alice/i)).toBeTruthy()
// Hovering the badge reveals the error message in tooltip
fireEvent.pointerMove(badge, { pointerType: 'mouse' })
act(() => {
vi.advanceTimersByTime(200)
})
const tooltip = screen.getByRole('tooltip')
expect(tooltip).toBeTruthy()
expect(tooltip.textContent).toContain('FFmpeg transcode failed: exit code 1')
vi.useRealTimers()
})
})
@@ -27,6 +27,7 @@ import { cn } from '@/ui/lib/utils'
import { useUploadStore } from '@/ui/stores/upload'
import { useDraggable, useDroppable } from '@dnd-kit/react'
import {
AlertCircle,
Clock,
Download,
Edit,
@@ -131,15 +132,21 @@ export function FileCard({
enabled: shouldPoll,
refetchInterval: (query: unknown) => {
const data = (query as { state: { data: { status?: string } } }).state.data
return data?.status === 'processed' ? false : 1000
return data?.status === 'processed' || data?.status === 'failed' || data?.status === 'error'
? false
: 1000
},
})
const displayItem = useMemo(() => {
if (item.status === 'processed' || item.status === 'error') {
if (item.status === 'processed' || item.status === 'error' || item.status === 'failed') {
return item
}
if (polledItem?.status === 'processed' || polledItem?.status === 'error') {
if (
polledItem?.status === 'processed' ||
polledItem?.status === 'error' ||
polledItem?.status === 'failed'
) {
return polledItem
}
return polledItem || item
@@ -263,10 +270,11 @@ export function FileCard({
// A preview (poster thumbnail and/or sprite) can be available before transcoding finishes,
// since the poster/sprite are generated and persisted ahead of the proxy transcodes.
const hasPreview = Boolean(displayItem.preview?.thumbnailUrl || displayItem.preview?.spriteUrl)
const isFailed = displayItem.status === 'failed'
const isProcessing = displayItem.status === 'processing' || displayItem.status === 'uploaded'
// While uploading/transcoding, the creator row is replaced by a short status label so the real
// date/author only appears once the asset is ready.
// date/author only appears once the asset is ready or failed.
const statusText =
displayItem.status === 'uploading'
? m.uploading()
@@ -274,6 +282,8 @@ export function FileCard({
? m.preparing()
: null
const failedErrorMessage = displayItem.media?.error || m.processing_failed()
const previewBadges =
daysLeft !== null ||
(typeof displayItem.commentsCount === 'number' && displayItem.commentsCount > 0) ? (
@@ -322,7 +332,7 @@ export function FileCard({
showDropFeedback && 'border-primary outline-1 outline-primary',
)}
>
<div className="absolute left-2 top-2 z-10">
<div className="absolute left-2 top-2 z-10 flex items-center gap-1.5">
<Checkbox
checked={isChecked}
onCheckedChange={() => {}}
@@ -351,6 +361,27 @@ export function FileCard({
displayItem.versionStack.versions.length}
</Badge>
)}
{isFailed && (
<TooltipProvider delayDuration={100}>
<Tooltip>
<TooltipTrigger asChild>
<span
data-testid="file-card-failed-badge"
className="select-none rounded bg-destructive/90 px-1.5 py-0.5 text-xs font-medium text-destructive-foreground shadow-sm cursor-help"
>
{m.transcoding_failed()}
</span>
</TooltipTrigger>
<TooltipContent
side="bottom"
data-testid="file-card-failed-tooltip"
className="max-w-xs break-words text-xs"
>
{failedErrorMessage}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)}
</div>
<div className="relative aspect-square overflow-hidden bg-muted/30">
@@ -364,7 +395,7 @@ export function FileCard({
data-testid="file-card-preview-media"
className={cn('h-full w-full', isProcessing && 'animate-pulse')}
>
<FilePreview item={displayItem} showDuration={!isProcessing} />
<FilePreview item={displayItem} showDuration={!isProcessing && !isFailed} />
</div>
{previewBadges}
</>
@@ -374,7 +405,17 @@ export function FileCard({
{m.failed_to_upload()}
</span>
</div>
) : displayItem.status === 'processing' || displayItem.status === 'uploaded' ? (
) : isFailed ? (
<div
data-testid="file-card-failed-state"
className="flex flex-col h-full w-full items-center justify-center gap-2 p-3 text-center bg-muted/40"
>
<AlertCircle className="h-10 w-10 text-destructive/80 shrink-0" />
<span className="z-10 text-xs font-semibold text-destructive px-2">
{m.processing_failed()}
</span>
</div>
) : isProcessing ? (
<div className="flex h-full w-full items-center justify-center">
{/* Same geometry as the upload progress ring, but empty inside and no percentage. */}
<svg
@@ -431,7 +472,7 @@ export function FileCard({
}}
>
<TooltipTrigger asChild>
<p ref={creatorRef} className="text-sm text-muted-foreground line-clamp-2 h-[2lh]">
<p ref={creatorRef} className="text-sm line-clamp-2 h-[2lh] text-muted-foreground">
{statusText ?? creatorText}
</p>
</TooltipTrigger>
@@ -134,4 +134,17 @@ describe('FileListItem', () => {
expect(onDoubleClick).toHaveBeenCalled()
})
it('renders (Processing failed) label when status is failed', () => {
const failedItem: AssetInfo = {
...mockItem,
status: 'failed',
} as AssetInfo
renderComponent({ item: failedItem })
const failedLabel = screen.getByTestId('file-list-item-failed')
expect(failedLabel).toBeTruthy()
expect(failedLabel.textContent).toMatch(/Processing failed|处理失败/i)
})
})
@@ -1,9 +1,10 @@
import type { AssetInfo, FieldValueInfo } from '@shumai/dtos'
import { client } from '@/ui/api/client'
import { m } from '@/ui/paraglide/messages.js'
import { useQuery } from '@tanstack/react-query'
import { useDraggable } from '@dnd-kit/react'
import { File, Folder, MoreVertical, AudioLines } from 'lucide-react'
import { AlertCircle, File, Folder, MoreVertical, AudioLines } from 'lucide-react'
import React, { useEffect, useMemo, useRef, useState } from 'react'
import { cn } from '@/ui/lib/utils'
import { formatSize } from '@/ui/lib/format'
@@ -90,15 +91,21 @@ export function FileListItem({
enabled: shouldPoll,
refetchInterval: (query: unknown) => {
const data = (query as { state: { data: { status?: string } } }).state.data
return data?.status === 'processed' ? false : 1000
return data?.status === 'processed' || data?.status === 'failed' || data?.status === 'error'
? false
: 1000
},
})
const displayItem = useMemo(() => {
if (item.status === 'processed' || item.status === 'error') {
if (item.status === 'processed' || item.status === 'error' || item.status === 'failed') {
return item
}
if (polledItem?.status === 'processed' || polledItem?.status === 'error') {
if (
polledItem?.status === 'processed' ||
polledItem?.status === 'error' ||
polledItem?.status === 'failed'
) {
return polledItem
}
return polledItem || item
@@ -215,6 +222,8 @@ export function FileListItem({
<div className="w-8 h-8 shrink-0 flex items-center justify-center bg-muted rounded overflow-hidden">
{displayItem.status === 'uploading' ? (
<ProgressCircle progress={uploadPercent} className="w-7 h-7" />
) : displayItem.status === 'failed' && !displayItem.preview?.thumbnailUrl ? (
<AlertCircle className="h-4 w-4 text-destructive" />
) : displayItem.preview?.thumbnailUrl ? (
<img
src={displayItem.preview.thumbnailUrl}
@@ -240,7 +249,15 @@ export function FileListItem({
/>
{displayItem.status === 'error' && (
<span className="text-xs text-destructive font-semibold shrink-0">
(Failed to upload)
{m.failed_to_upload_parenthesized()}
</span>
)}
{displayItem.status === 'failed' && (
<span
data-testid="file-list-item-failed"
className="text-xs text-destructive font-semibold shrink-0"
>
{m.processing_failed_parenthesized()}
</span>
)}
@@ -654,7 +654,10 @@ export function MobileFileBrowser({
<AlertDialogFooter>
<AlertDialogCancel>{m.cancel()}</AlertDialogCancel>
<AlertDialogAction
onClick={() => emptyTrash()}
onClick={(e) => {
e.preventDefault()
emptyTrash()
}}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
disabled={isEmptyingTrash}
>
@@ -145,6 +145,7 @@ export function MobileFileDetail({
key: t.key,
width: t.width,
height: t.height,
resolution: t.resolution,
hdr: t.hdr,
})),
}
@@ -119,4 +119,69 @@ describe('MobileFileHeader', () => {
expect(menuBtn).toBeDefined()
fireEvent.click(menuBtn)
})
it('displays 480p instead of 360p for 480p video transcode in download menu', () => {
render(
<MobileFileHeader
fileName="scene1.mp4"
activeFileId="file-1"
allowDownload={true}
onBack={vi.fn()}
downloadInfo={{
originalKey: 'raw.mov',
videoTranscodes: [
{
key: 'scene1-480p.mp4',
width: 854,
height: 480,
resolution: '480p',
hdr: false,
},
],
}}
/>,
)
const menuBtn = screen.getByLabelText(/More options/i)
fireEvent.pointerDown(menuBtn, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('480p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('360p'))).toBe(false)
})
it('correctly labels legacy portrait video transcode in download menu using long side fallback', () => {
render(
<MobileFileHeader
fileName="portrait.mp4"
activeFileId="file-portrait"
allowDownload={true}
onBack={vi.fn()}
downloadInfo={{
originalKey: 'raw.mp4',
videoTranscodes: [
{
key: 'portrait-legacy.mp4',
width: 1080,
height: 1920,
hdr: false,
},
],
}}
/>,
)
const menuBtn = screen.getByLabelText(/More options/i)
fireEvent.pointerDown(menuBtn, { button: 0, ctrlKey: false })
const downloadSubTrigger = screen.getByText('Download')
fireEvent.click(downloadSubTrigger)
const items = screen.getAllByRole('menuitem')
expect(items.some((i) => i.textContent?.includes('1080p'))).toBe(true)
expect(items.some((i) => i.textContent?.includes('1920p'))).toBe(false)
})
})
@@ -26,6 +26,7 @@ import {
Trash2,
} from 'lucide-react'
import React from 'react'
import { getVideoResolutionLabel } from '@/ui/lib/media'
export interface MobileFileHeaderProps {
fileName: string
@@ -46,6 +47,7 @@ export interface MobileFileHeaderProps {
key: string
width: number
height: number
resolution?: string
hdr?: boolean
}>
}
@@ -238,14 +240,7 @@ export function MobileFileHeader({
<DropdownMenuSubContent className="w-48 z-50">
<DropdownMenuLabel className="text-xs">{m.download()}</DropdownMenuLabel>
{downloadInfo?.videoTranscodes?.map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const resolution = getVideoResolutionLabel(t)
const label = t.hdr ? `${resolution} (HDR)` : resolution
return (
<DropdownMenuItem
@@ -136,13 +136,19 @@ export const NotificationCard = ({ notification }: NotificationCardProps) => {
const hasPreview = asset?.preview && (asset.proxyType === 'image' || asset.proxyType === 'video')
const handleClick = () => {
const handleClick = () => {
if (kanbanTask?.id && team?.id) {
navigate({
to: '/teams/$teamId/kanban',
params: { teamId: team.id },
search: { taskId: kanbanTask.id },
})
}else if(project?.id && asset?.id){
//TODO: Perhaps it should be marked as read too when redirecting to it.
navigate({
to: '/projects/$projectId/files/$fileId',
params: { projectId: project.id, fileId: asset.id },
})
}
}
@@ -440,6 +440,7 @@ export function PublicShareManager({
key: t.key,
width: t.width,
height: t.height,
resolution: t.resolution,
hdr: t.hdr,
})),
}
@@ -0,0 +1,46 @@
// @vitest-environment happy-dom
import { cleanup, render, screen } from '@testing-library/react'
import React from 'react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { DefaultViewer } from './default-viewer'
import type { AssetInfo } from '@shumai/dtos'
vi.mock('@/ui/paraglide/messages.js', () => ({
m: new Proxy({}, { get: () => () => '' }),
}))
describe('DefaultViewer', () => {
afterEach(() => {
cleanup()
})
it('renders failed state when file status is failed', () => {
const file = {
id: 'failed-doc',
name: 'failed.docx',
status: 'failed',
media: {
error: 'PDF generation failed',
},
} as unknown as AssetInfo
render(<DefaultViewer file={file} />)
expect(screen.getByTestId('default-viewer-failed-state')).toBeDefined()
expect(screen.getByText('PDF generation failed')).toBeDefined()
})
it('renders standard download view when file is not failed', () => {
const file = {
id: 'file-1',
name: 'archive.zip',
status: 'processed',
sizeByte: 1024,
media: {
original: { key: 'raw/archive.zip' },
},
} as unknown as AssetInfo
render(<DefaultViewer file={file} />)
expect(screen.queryByTestId('default-viewer-failed-state')).toBeNull()
expect(screen.getByText('Download')).toBeDefined()
})
})
@@ -1,5 +1,6 @@
import { client } from '@/ui/api/client'
import { Download } from 'lucide-react'
import { m } from '@/ui/paraglide/messages.js'
import { Download, AlertCircle } from 'lucide-react'
import React, { useImperativeHandle } from 'react'
import { FileViewerProps, MediaController } from '../types'
@@ -37,12 +38,40 @@ export const DefaultViewer = React.forwardRef<MediaController, FileViewerProps>(
}
}
const isFailed = file.status === 'failed'
const errorMessage = file.media?.error
return (
<div className="flex flex-col flex-1 h-full overflow-hidden bg-gray-100 dark:bg-gray-950 relative">
<div className="flex-1 flex flex-col-reverse md:flex-row min-h-0 relative">
{children}
<div className="flex-1 flex items-center justify-center">
<p className="text-muted-foreground">Preview unavailable</p>
<div className="flex-1 flex items-center justify-center p-6 text-center">
{isFailed ? (
<div
data-testid="default-viewer-failed-state"
className="flex flex-col items-center justify-center gap-3 max-w-md"
>
<AlertCircle className="h-12 w-12 text-destructive/80" />
<div className="space-y-1">
<h3 className="text-base font-semibold text-foreground">
{m.media_processing_failed()}
</h3>
<p className="text-sm text-muted-foreground">
{m.media_processing_failed_desc()}
</p>
{errorMessage && (
<div
title={errorMessage}
className="mt-3 rounded-lg border border-destructive/20 bg-destructive/10 p-3 text-xs text-destructive text-left line-clamp-3 break-words"
>
{errorMessage}
</div>
)}
</div>
</div>
) : (
<p className="text-muted-foreground">Preview unavailable</p>
)}
</div>
</div>
<div className="relative px-4 py-3 bg-card border-t border-gray-200 dark:border-gray-700 z-10 flex items-center justify-end gap-2 transition-colors duration-200">
@@ -0,0 +1,502 @@
// @vitest-environment happy-dom
import { act, cleanup, render } from '@testing-library/react'
import React, { createRef } from 'react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { CompareVideoPane } from './compare-video-pane'
import type { ComparePaneHandle, PaneReportedState } from '../../compare/types'
import type { AssetInfo } from '@shumai/dtos'
const { mockHlsInstance, mockHlsConstructor } = vi.hoisted(() => {
const instance = {
attachMedia: vi.fn(),
loadSource: vi.fn(),
on: vi.fn(),
off: vi.fn(),
once: vi.fn(),
destroy: vi.fn(),
levels: [
{ height: 1080, width: 1920, bitrate: 5000000 },
{ height: 720, width: 1280, bitrate: 2500000 },
],
currentLevel: -1,
startLoad: vi.fn(),
recoverMediaError: vi.fn(),
}
const constructor = vi.fn(function () {
return instance
})
return { mockHlsInstance: instance, mockHlsConstructor: constructor }
})
vi.mock('hls.js', () => {
const MockHls = mockHlsConstructor
// @ts-expect-error adding static mock property
MockHls.isSupported = vi.fn(() => true)
// @ts-expect-error adding static mock property
MockHls.Events = {
MANIFEST_PARSED: 'hlsManifestParsed',
LEVEL_SWITCHED: 'hlsLevelSwitched',
ERROR: 'hlsError',
}
// @ts-expect-error adding static mock property
MockHls.ErrorTypes = {
NETWORK_ERROR: 'networkError',
MEDIA_ERROR: 'mediaError',
OTHER_ERROR: 'otherError',
}
return {
default: MockHls,
}
})
vi.mock('@/ui/components/drawing-canvas', () => ({
default: () => <div data-testid="drawing-canvas" />,
}))
vi.mock('@/ui/paraglide/messages.js', () => ({
m: new Proxy({}, { get: () => () => '' }),
}))
describe('CompareVideoPane', () => {
beforeEach(async () => {
vi.clearAllMocks()
mockHlsInstance.currentLevel = -1
const Hls = (await import('hls.js')).default as unknown as {
isSupported: ReturnType<typeof vi.fn>
}
Hls.isSupported.mockReturnValue(true)
})
afterEach(() => {
cleanup()
})
it('safely handles native HLS fallback without clearing video.src when manual resolution is invoked', async () => {
const Hls = (await import('hls.js')).default as unknown as {
isSupported: ReturnType<typeof vi.fn>
}
Hls.isSupported.mockReturnValue(false)
// Mock HTMLMediaElement.prototype.canPlayType to support native HLS
const originalCanPlayType = HTMLMediaElement.prototype.canPlayType
HTMLMediaElement.prototype.canPlayType = vi.fn((type: string) =>
type === 'application/vnd.apple.mpegurl' ? 'maybe' : '',
)
try {
const hlsVideo: AssetInfo = {
id: 'hls-compare',
name: 'hls-compare.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
const ref = createRef<ComparePaneHandle>()
const { container } = render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onStateChange={(state) => reportedStates.push(state)}
onActivate={vi.fn()}
/>,
)
const video = container.querySelector('video') as HTMLVideoElement
expect(video.src).toBe('https://cdn.example.com/master.m3u8')
// Check reported state: isHlsManualSupported should be false
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.isHlsManualSupported).toBe(false)
// Invoke changeResolution('1080p')
act(() => {
ref.current?.changeResolution('1080p')
})
// In native fallback, changeResolution should NO-OP and NOT clear video.src to empty string
expect(video.src).toBe('https://cdn.example.com/master.m3u8')
} finally {
HTMLMediaElement.prototype.canPlayType = originalCanPlayType
}
})
it('switches HLS resolution immediately using currentLevel and does not hijack subsequent seeks', async () => {
vi.useFakeTimers()
try {
const hlsVideo: AssetInfo = {
id: 'hls-compare-switch',
name: 'hls-compare-switch.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
const ref = createRef<ComparePaneHandle>()
const { container } = render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onStateChange={(state) => reportedStates.push(state)}
onActivate={vi.fn()}
/>,
)
const video = container.querySelector('video') as HTMLVideoElement
Object.defineProperty(video, 'paused', { value: true, configurable: true })
video.currentTime = 0
// In Hls.js supported mode, isHlsManualSupported should be true
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.isHlsManualSupported).toBe(true)
// Switch to 720p (index 1)
act(() => {
ref.current?.changeResolution('720p')
})
expect(mockHlsInstance.currentLevel).toBe(1)
// Advance timers past the 1500ms safety timeout without seeked event
act(() => {
vi.advanceTimersByTime(2000)
})
// Now user performs an unrelated seek to frame 150 (currentTime = 5.0)
video.currentTime = 5.0
act(() => {
video.dispatchEvent(new Event('seeked'))
})
// video.currentTime must remain at frame 150, NOT rewound to frame 0
expect(Math.floor(video.currentTime * 30 + 0.001)).toBe(150)
expect(video.currentTime).toBeCloseTo(5.0167, 3)
} finally {
vi.useRealTimers()
}
})
it('reports 2160p instead of 3840p for 4K vertical video when HLS switches level on Auto', () => {
mockHlsInstance.levels = [{ height: 3840, width: 2160, bitrate: 10000000 }]
const hlsVideo: AssetInfo = {
id: 'video-hls-compare-vertical-4k',
name: 'vertical-4k.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [{ width: 2160, height: 3840, resolution: '2160p' }],
},
metadata: {
originalWidth: 2160,
originalHeight: 3840,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const ref = createRef<ComparePaneHandle>()
const reportedStates: PaneReportedState[] = []
render(
<CompareVideoPane
ref={ref}
file={hlsVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={(s) => reportedStates.push(s)}
/>,
)
const levelSwitchedHandler = mockHlsInstance.on.mock.calls.find(
([event]) => event === 'hlsLevelSwitched',
)?.[1]
expect(levelSwitchedHandler).toBeDefined()
act(() => {
levelSwitchedHandler('hlsLevelSwitched', { level: 0 })
})
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.activeAutoResolution).toBe('2160p')
})
it('derives resolution from long side for legacy portrait MP4 transcode without explicit resolution', () => {
const legacyPortraitVideo: AssetInfo = {
id: 'legacy-portrait-compare-mp4',
name: 'portrait.mp4',
proxyType: 'video',
media: {
videoTranscodes: [
{
key: 'portrait-1080p.mp4',
url: 'https://cdn.example.com/portrait-1080p.mp4',
width: 1080,
height: 1920,
size: 5000000,
},
],
metadata: {
originalWidth: 1080,
originalHeight: 1920,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const reportedStates: PaneReportedState[] = []
render(
<CompareVideoPane
file={legacyPortraitVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={(s) => reportedStates.push(s)}
/>,
)
const lastState = reportedStates[reportedStates.length - 1]
expect(lastState?.video?.resolutions?.[0]?.resolution).toBe('1080p')
})
it('sets poster on video element from file.preview.thumbnailUrl', () => {
const videoWithThumb: AssetInfo = {
id: 'compare-thumb',
name: 'thumb.mp4',
proxyType: 'video',
preview: {
thumbnailUrl: 'https://cdn.example.com/compare-poster.jpg',
},
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(
<CompareVideoPane
file={videoWithThumb}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
/>,
)
const video = container.querySelector(
'[data-testid="compare-video-area"] video',
) as HTMLVideoElement
expect(video.getAttribute('poster')).toBe('https://cdn.example.com/compare-poster.jpg')
})
it('shows loading spinner initially, transitions on loadeddata, and blocks play while loading', () => {
const testVideo: AssetInfo = {
id: 'compare-loading-test',
name: 'loading.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const onRequestTogglePlay = vi.fn()
const { container } = render(
<CompareVideoPane
file={testVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
onRequestTogglePlay={onRequestTogglePlay}
/>,
)
const area = container.querySelector('[data-testid="compare-video-area"]') as HTMLDivElement
const video = area.querySelector('video') as HTMLVideoElement
// Initially loading spinner is visible
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).not.toBeNull()
expect(container.querySelector('[data-testid="compare-video-play-overlay"]')).toBeNull()
// Clicking while loading should not trigger play
act(() => {
area.click()
})
expect(onRequestTogglePlay).not.toHaveBeenCalled()
// Dispatch loadeddata
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
// Spinner gone, play overlay visible
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
expect(container.querySelector('[data-testid="compare-video-play-overlay"]')).not.toBeNull()
// Clicking now triggers toggle play
act(() => {
area.click()
})
expect(onRequestTogglePlay).toHaveBeenCalledTimes(1)
})
it('does not cancel waiting stall debounce when progress events fire during stall', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'compare-stall-progress-test',
name: 'compare-stall-progress.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/compare-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(
<CompareVideoPane
file={testVideo}
isActive={true}
annotations={[]}
onActivate={vi.fn()}
onStateChange={vi.fn()}
/>,
)
const video = container.querySelector(
'[data-testid="compare-video-area"] video',
) as HTMLVideoElement
// 1. Initial ready state
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 2. Play starts (paused === false)
Object.defineProperty(video, 'paused', { value: false, configurable: true, writable: true })
act(() => {
video.dispatchEvent(new Event('playing'))
})
// 3. Stalls: dispatches waiting
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Spinner not visible immediately (within 200ms debounce)
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 4. Progress event fires at 100ms while paused is false (data still trickling in)
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('progress'))
})
// Should NOT clear debounce or mark loading false
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
// 5. Advance past 200ms debounce (advance remaining 110ms)
act(() => {
vi.advanceTimersByTime(110)
})
// Spinner should now appear despite the progress event!
expect(
container.querySelector('[data-testid="compare-video-loading-spinner"]'),
).not.toBeNull()
// 6. Playback resumes: dispatch playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
// Spinner disappears
expect(container.querySelector('[data-testid="compare-video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
})
@@ -4,14 +4,14 @@ import { cn } from '@/ui/lib/utils'
import { useAnnotationStore } from '@/ui/stores/annotation-store'
import type { Annotation } from '@/ui/types'
import type { AssetInfo } from '@shumai/dtos'
import { Play, AudioLines } from 'lucide-react'
import { Play, AudioLines, Loader2 } from 'lucide-react'
import { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react'
import videojs from 'video.js'
import type Player from 'video.js/dist/types/player'
import Hls from 'hls.js'
import { useFramePlayer } from './use-frame-player'
import { resolveTotalFrames } from './utils'
import { calculateFrameCenterTime, resolveTotalFrames } from './utils'
import { clampFrame as clampFrameUtil } from '../../compare/compare-utils'
import type { ComparePaneHandle, DisplayTranscode, PaneReportedState } from '../../compare/types'
import { getVideoResolutionLabel } from '@/ui/lib/media'
interface CompareVideoPaneProps {
file: AssetInfo
@@ -32,17 +32,15 @@ interface CompareVideoPaneProps {
function computeResolutions(file: AssetInfo): DisplayTranscode[] {
// Only transcoded proxy versions are ever displayed; the raw original file
// is never used as a playback source.
return (file.media?.videoTranscodes ?? []).map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
return { ...t, resolution }
})
const isHls = Boolean(file.media?.isHls && file.media.hls?.url)
const baseTranscodes = isHls
? (file.media?.hls?.resolutions ?? [])
: (file.media?.videoTranscodes ?? [])
return baseTranscodes.map((t) => ({
...t,
resolution: getVideoResolutionLabel(t),
}))
}
function getInitialResolution(resolutions: DisplayTranscode[]): DisplayTranscode | null {
@@ -84,8 +82,7 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
},
ref,
) {
const playerRef = useRef<Player | null>(null)
const videoContainerRef = useRef<HTMLDivElement>(null)
const hlsRef = useRef<Hls | null>(null)
const containerRef = useRef<HTMLDivElement>(null)
const videoRef = useRef<HTMLVideoElement | null>(null)
@@ -110,6 +107,8 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const [playerVolume, setPlayerVolume] = useState(volume)
const [playerMuted, setPlayerMuted] = useState(muted)
const [isPlayerReady, setIsPlayerReady] = useState(false)
const [isLoading, setIsLoading] = useState(true)
const waitingTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const metadata = file.media?.metadata
const frameRate = metadata?.frameRate || 30
@@ -117,11 +116,17 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const containerDuration = metadata?.duration || 0
const totalFrames = resolveTotalFrames({ dbTotalFrames, containerDuration, frameRate })
const isHls = Boolean(file.media?.isHls && file.media.hls?.url)
const resolutions = computeResolutions(file)
const initialRes = getInitialResolution(resolutions)
const [currentResolution, setCurrentResolution] = useState(initialRes?.resolution ?? '')
const initialRes = isHls ? null : getInitialResolution(resolutions)
const [currentResolution, setCurrentResolution] = useState(
isHls ? 'Auto' : (initialRes?.resolution ?? ''),
)
const [isCurrentHdr, setIsCurrentHdr] = useState(initialRes?.hdr)
const currentSrcRef = useRef(initialRes?.url)
const [activeAutoResolution, setActiveAutoResolution] = useState<string | undefined>(undefined)
const pendingResolutionCorrectionRef = useRef<(() => void) | null>(null)
const isHlsManualSupported = !isHls || Boolean(Hls.isSupported())
const currentSrcRef = useRef(isHls ? file.media!.hls!.url : initialRes?.url)
const isAudio = file.proxyType === 'audio'
const { currentFrame, seekToFrame } = useFramePlayer(videoRef, frameRate, totalFrames, isAudio)
@@ -160,78 +165,220 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
// if the parent renders this pane without a per-asset `key`. Runs before the
// video.js init effect below (declaration order) so the ref is fresh.
useEffect(() => {
const res = getInitialResolution(resolutions)
setCurrentResolution(res?.resolution ?? '')
const res = isHls ? null : getInitialResolution(resolutions)
setCurrentResolution(isHls ? 'Auto' : (res?.resolution ?? ''))
setIsCurrentHdr(res?.hdr)
currentSrcRef.current = res?.url
setActiveAutoResolution(undefined)
currentSrcRef.current = isHls ? file.media!.hls!.url : res?.url
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsLoading(true)
setIsPlayerReady(false)
setHasStartedPlaying(false)
}, [file.id])
}, [file.id, Boolean(isHls ? file.media?.hls?.url : initialRes?.url)])
// Initialize video.js
// Initialize player (Hls.js or native HTML5 video)
useEffect(() => {
if (!videoContainerRef.current) return
const videoElement = document.createElement('video-js')
videoElement.classList.add('vjs-big-play-centered', '!h-full', '!w-full')
videoElement.style.pointerEvents = 'none'
videoContainerRef.current.appendChild(videoElement)
const video = videoRef.current
if (!video) return
const player = (playerRef.current = videojs(videoElement, {
controls: false,
autoplay: false,
preload: 'auto',
playsinline: true,
sources: [{ src: currentSrcRef.current ?? '', type: 'video/mp4' }],
}))
const targetSrc = isHls ? file.media!.hls!.url! : (currentSrcRef.current ?? '')
if (!targetSrc) return
const htmlVid = videoElement.querySelector('video')
if (htmlVid) {
videoRef.current = htmlVid
} else {
player.ready(() => {
const techEl = player.tech({ iWillNotUseThisInPlugins: true })?.el() as HTMLVideoElement
if (techEl) {
videoRef.current = techEl
}
})
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
player.on('play', () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
let hls: Hls | null = null
if (isHls) {
if (Hls.isSupported()) {
hls = new Hls({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
})
hlsRef.current = hls
hls.attachMedia(video)
hls.loadSource(targetSrc)
hls.on(Hls.Events.LEVEL_SWITCHED, (_event, eventData) => {
const level = hls?.levels[eventData.level]
if (level) {
const matched = resolutions.find(
(r) => r.width === level.width && r.height === level.height,
)
if (matched?.resolution) {
setActiveAutoResolution(matched.resolution)
} else {
const shortSide =
level.width && level.height
? Math.min(level.width, level.height)
: level.height || level.width
setActiveAutoResolution(shortSide ? `${shortSide}p` : undefined)
}
}
})
hls.on(Hls.Events.ERROR, (_event, eventData) => {
if (eventData.fatal) {
switch (eventData.type) {
case Hls.ErrorTypes.NETWORK_ERROR:
hls?.startLoad()
break
case Hls.ErrorTypes.MEDIA_ERROR:
hls?.recoverMediaError()
break
default:
hls?.destroy()
break
}
}
})
} else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = targetSrc
}
} else {
video.src = targetSrc
}
const clearWaitingTimeout = () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
}
const handlePlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setIsPlaying(true)
setHasStartedPlaying(true)
onPlay?.()
})
player.on('pause', () => setIsPlaying(false))
player.on('ended', () => setIsPlaying(false))
player.on('timeupdate', () => {
setBuffered(player.bufferedPercent())
})
player.on('loadedmetadata', () => {
}
const handlePause = () => setIsPlaying(false)
const handleEnded = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlaying(false)
}
const handleLoadedData = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
})
player.on('loadstart', () => setIsPlayerReady(false))
player.on('volumechange', () => {
setPlayerVolume(player.volume() || 0)
setPlayerMuted(player.muted() || false)
})
player.on('ratechange', () => setPlaybackRate(player.playbackRate() || 1))
}
const handleCanPlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
}
const handlePlaying = () => {
clearWaitingTimeout()
setIsLoading(false)
}
const handleWaiting = () => {
clearWaitingTimeout()
waitingTimeoutRef.current = setTimeout(() => {
setIsLoading(true)
}, 200)
}
const handleLoadStart = () => {
clearWaitingTimeout()
setIsLoading(true)
setIsPlayerReady(false)
}
const handleProgress = () => {
const vidDuration = video.duration || containerDuration || 0
if (vidDuration > 0 && video.buffered.length > 0) {
let bufferedEnd = 0
for (let i = 0; i < video.buffered.length; i++) {
if (
video.buffered.start(i) <= video.currentTime &&
video.currentTime <= video.buffered.end(i)
) {
bufferedEnd = video.buffered.end(i)
break
}
}
if (bufferedEnd === 0) {
bufferedEnd = video.buffered.end(video.buffered.length - 1)
}
setBuffered((bufferedEnd / vidDuration) * 100)
}
}
const handleVolumeChange = () => {
setPlayerVolume(video.volume || 0)
setPlayerMuted(video.muted || false)
}
const handleRateChange = () => setPlaybackRate(video.playbackRate || 1)
if (video.readyState >= 2) {
setIsLoading(false)
setIsPlayerReady(true)
}
video.addEventListener('play', handlePlay)
video.addEventListener('pause', handlePause)
video.addEventListener('ended', handleEnded)
video.addEventListener('loadeddata', handleLoadedData)
video.addEventListener('canplay', handleCanPlay)
video.addEventListener('playing', handlePlaying)
video.addEventListener('waiting', handleWaiting)
video.addEventListener('loadstart', handleLoadStart)
video.addEventListener('timeupdate', handleProgress)
video.addEventListener('progress', handleProgress)
video.addEventListener('volumechange', handleVolumeChange)
video.addEventListener('ratechange', handleRateChange)
return () => {
if (player && !player.isDisposed()) {
player.dispose()
playerRef.current = null
videoRef.current = null
clearWaitingTimeout()
video.removeEventListener('play', handlePlay)
video.removeEventListener('pause', handlePause)
video.removeEventListener('ended', handleEnded)
video.removeEventListener('loadeddata', handleLoadedData)
video.removeEventListener('canplay', handleCanPlay)
video.removeEventListener('playing', handlePlaying)
video.removeEventListener('waiting', handleWaiting)
video.removeEventListener('loadstart', handleLoadStart)
video.removeEventListener('timeupdate', handleProgress)
video.removeEventListener('progress', handleProgress)
video.removeEventListener('volumechange', handleVolumeChange)
video.removeEventListener('ratechange', handleRateChange)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
}
}, [file.id])
// Apply muted / volume (audio routing: only active side unmuted)
useEffect(() => {
const player = playerRef.current
if (!player || !isPlayerReady) return
player.muted(muted)
player.volume(volume)
const video = videoRef.current
if (!video || !isPlayerReady) return
video.muted = muted
video.volume = volume
}, [muted, volume, isPlayerReady])
// Report state upward
@@ -252,6 +399,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
isCurrentHdr,
resolutions,
buffered,
activeAutoResolution,
isHlsManualSupported,
isLoading,
},
}
onStateChange(state)
@@ -268,6 +418,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
currentResolution,
isCurrentHdr,
buffered,
activeAutoResolution,
isHlsManualSupported,
isLoading,
])
// Propagate active playhead time
@@ -323,8 +476,81 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
const changeResolution = useCallback(
(resolution: string, hdr?: boolean) => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (isHls) {
if (!hlsRef.current) {
// Safari native HLS fallback does not support manual rendition switching
return
}
const hls = hlsRef.current
const wasPaused = video.paused
const targetFrame = currentFrameRef.current
if (resolution === 'Auto') {
if (currentResolution === 'Auto' && hls.currentLevel === -1) {
return
}
hls.currentLevel = -1
setCurrentResolution('Auto')
} else {
const target = resolutions.find((r) => r.resolution === resolution)
if (target) {
const targetIndex = hls.levels.findIndex(
(lvl) =>
lvl.height === target.height ||
Math.max(lvl.width, lvl.height) === Math.max(target.width, target.height),
)
if (targetIndex !== -1) {
if (targetIndex === hls.currentLevel && currentResolution === resolution) {
return
}
hls.currentLevel = targetIndex
setCurrentResolution(resolution)
setIsCurrentHdr(target.hdr)
}
}
}
if (wasPaused) {
let timer: ReturnType<typeof setTimeout> | null = null
const cleanup = () => {
if (timer) {
clearTimeout(timer)
timer = null
}
video.removeEventListener('seeked', onSeeked)
hls.off?.(Hls.Events.ERROR, cleanup)
if (pendingResolutionCorrectionRef.current === cleanup) {
pendingResolutionCorrectionRef.current = null
}
}
const onSeeked = () => {
cleanup()
const safeCenterTime = calculateFrameCenterTime(targetFrame, frameRate)
const frameDuration = 1 / frameRate
if (Math.abs(video.currentTime - safeCenterTime) > frameDuration / 4) {
video.currentTime = safeCenterTime
}
}
// Safety timeout (1.5s): if no seeked event fires from rendition switch, cleanly detach
timer = setTimeout(cleanup, 1500)
pendingResolutionCorrectionRef.current = cleanup
video.addEventListener('seeked', onSeeked)
hls.once?.(Hls.Events.ERROR, cleanup)
}
return
}
const target =
hdr !== undefined
? resolutions.find(
@@ -332,21 +558,24 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
)
: resolutions.find((r) => r.resolution === resolution)
if (!target) return
const wasPlaying = !player.paused()
const currentT = player.currentTime()
const wasPlaying = !video.paused
const currentT = video.currentTime
setCurrentResolution(resolution)
setIsCurrentHdr(target.hdr)
currentSrcRef.current = target.url
player.src({ type: 'video/mp4', src: target.url })
player.one('loadedmetadata', () => {
player.currentTime(currentT)
video.src = target.url || ''
const onLoadedMetadata = () => {
video.removeEventListener('loadedmetadata', onLoadedMetadata)
video.currentTime = currentT
if (wasPlaying) {
const p = player.play()
const p = video.play()
if (p !== undefined) p.catch(() => {})
}
})
video.playbackRate = playbackRate
}
video.addEventListener('loadedmetadata', onLoadedMetadata)
},
[resolutions],
[resolutions, isHls, frameRate, playbackRate, currentResolution],
)
useImperativeHandle(
@@ -354,47 +583,71 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
(): ComparePaneHandle => ({
getKind: () => 'video',
play: () => {
const p = playerRef.current?.play()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
const p = videoRef.current?.play()
if (p !== undefined) p.catch(() => {})
},
pause: () => playerRef.current?.pause(),
pause: () => videoRef.current?.pause(),
togglePlay: () => {
const player = playerRef.current
if (!player) return
if (player.paused() || player.ended()) {
const p = player.play()
const video = videoRef.current
if (!video) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (video.paused || video.ended) {
const p = video.play()
if (p !== undefined) p.catch(() => {})
} else {
player.pause()
video.pause()
}
},
seekToFrame: (frame) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(clampFrame(frame))
},
seekToSecond: (second) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
const frame = Math.floor(second * frameRate + 0.45)
seekToFrame(clampFrame(frame))
},
stepFrame: (delta) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(clampFrame(currentFrameRef.current + delta))
},
setMuted: (m) => playerRef.current?.muted(m),
setMuted: (m) => {
if (videoRef.current) videoRef.current.muted = m
},
setVolume: (v) => {
const player = playerRef.current
if (!player) return
player.volume(v)
if (v > 0 && player.muted()) player.muted(false)
else if (v === 0 && !player.muted()) player.muted(true)
const video = videoRef.current
if (!video) return
video.volume = v
if (v > 0 && video.muted) video.muted = false
else if (v === 0 && !video.muted) video.muted = true
},
setPlaybackRate: (rate) => {
if (videoRef.current) videoRef.current.playbackRate = rate
},
setPlaybackRate: (rate) => playerRef.current?.playbackRate(rate),
toggleLoop: () => {
const player = playerRef.current
if (!player) return
const next = !player.loop()
player.loop(next)
const video = videoRef.current
if (!video) return
const next = !video.loop
video.loop = next
setIsLooping(next)
},
changeResolution,
@@ -426,7 +679,7 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
],
)
const displayAnnotations = [...annotations, ...(isActive ? draftAnnotations : [])]
const displayAnnotations = [...(annotations ?? []), ...(isActive ? draftAnnotations : [])]
const scale = zoom
const defaultPanX = (containerSize.width - vidW * scale) / 2
@@ -441,8 +694,9 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
onActivate?.()
return
}
if (isLoading) return
onRequestTogglePlay?.()
}, [isActive, onActivate, onRequestTogglePlay])
}, [isActive, isLoading, onActivate, onRequestTogglePlay])
if (!file.media?.metadata || resolutions.length === 0) {
return (
@@ -474,14 +728,22 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
transformOrigin: '0 0',
}}
>
<div
ref={videoContainerRef}
className="w-full h-full [&_.video-js]:!w-full [&_.video-js]:!h-full [&_video]:!w-full [&_video]:!h-full [&_video]:!block [&_video]:!object-contain"
<video
ref={videoRef}
className="w-full h-full block object-contain pointer-events-none"
playsInline
preload="auto"
poster={file.preview?.thumbnailUrl || undefined}
/>
</div>
)}
{isAudio && (
<div ref={videoContainerRef} className="absolute inset-0 pointer-events-none opacity-0" />
<video
ref={videoRef}
className="absolute inset-0 pointer-events-none opacity-0"
playsInline
preload="auto"
/>
)}
{isAudio ? (
@@ -512,8 +774,23 @@ export const CompareVideoPane = forwardRef<ComparePaneHandle, CompareVideoPanePr
)
)}
{!hasStartedPlaying && !isDrawing && (
<div className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200">
{/* Loading Spinner Overlay */}
{isLoading && (
<div
className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200"
data-testid="compare-video-loading-spinner"
>
<div className="flex h-14 w-14 items-center justify-center rounded-full border-2 border-white/30 bg-white/10 backdrop-blur-sm">
<Loader2 className="h-7 w-7 text-white animate-spin" />
</div>
</div>
)}
{!isLoading && !hasStartedPlaying && !isDrawing && (
<div
className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-black/20 transition-opacity duration-200"
data-testid="compare-video-play-overlay"
>
<div className="flex h-16 w-16 items-center justify-center rounded-full border-2 border-white/30 bg-white/10 backdrop-blur-sm">
<Play className="ml-1 h-8 w-8 fill-white text-white" />
</div>
@@ -24,7 +24,7 @@ import {
} from '../../ui/dropdown-menu'
import ProgressBar from './progress-bar'
import { formatTime, formatTimecode } from './utils'
import type { ControlBarProps } from './video-control-bar'
import type { ControlBarProps, DisplayTranscode } from './video-control-bar'
export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
state,
@@ -45,7 +45,9 @@ export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
floatOverlayInFullScreen = true,
onMouseEnter,
onMouseLeave,
isLoading: propIsLoading,
}) => {
const isLoading = propIsLoading ?? state.isLoading ?? false
const { videoTimeDisplayMode, setVideoTimeDisplayMode } = useUiStore()
if (!data.media?.metadata) {
@@ -78,214 +80,248 @@ export const MobileVideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter={onMouseEnter}
onMouseLeave={onMouseLeave}
>
{/* Progress Bar */}
<div className="mb-2 px-0.5">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
<div
className={cn(
'transition-opacity duration-200',
isLoading && 'pointer-events-none opacity-60 select-none',
)}
>
{/* Progress Bar */}
<div className="mb-2 px-0.5">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
{/* Control Row */}
<div className="flex items-center justify-between text-foreground gap-1 select-none">
{/* Left Side: Play, Mute, Time */}
<div className="flex items-center gap-2 min-w-0">
<button
onClick={togglePlay}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-5 h-5 fill-current" />
) : (
<Play className="w-5 h-5 fill-current" />
)}
</button>
<button
onClick={toggleMute}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
aria-label={state.isMuted ? 'Unmute' : 'Mute'}
>
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-5 h-5" />
) : (
<Volume2 className="w-5 h-5" />
)}
</button>
<div className="flex items-center gap-0.5 text-xs font-medium tabular-nums min-w-0">
<span
className="text-muted-foreground font-mono truncate text-[11px] sm:text-xs"
data-testid="time-readout"
{/* Control Row */}
<div className="flex items-center justify-between text-foreground gap-1 select-none">
{/* Left Side: Play, Mute, Time */}
<div className="flex items-center gap-2 min-w-0">
<button
onClick={togglePlay}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{displayString}
</span>
{state.isPlaying ? (
<Pause className="w-5 h-5 fill-current" />
) : (
<Play className="w-5 h-5 fill-current" />
)}
</button>
<button
onClick={toggleMute}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
aria-label={state.isMuted ? 'Unmute' : 'Mute'}
>
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-5 h-5" />
) : (
<Volume2 className="w-5 h-5" />
)}
</button>
<div className="flex items-center gap-0.5 text-xs font-medium tabular-nums min-w-0">
<span
className="text-muted-foreground font-mono truncate text-[11px] sm:text-xs"
data-testid="time-readout"
>
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="h-5 w-5 inline-flex items-center justify-center rounded hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none shrink-0"
aria-label={m.change_time_format()}
>
<ChevronDown className="h-3 w-3" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.format()}
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.standard_time()}</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.frames()}</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.timecode()}</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Settings (Quality + Speed) & Fullscreen */}
<div className="flex items-center gap-1 shrink-0">
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="h-5 w-5 inline-flex items-center justify-center rounded hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none shrink-0"
aria-label={m.change_time_format()}
className="flex items-center gap-1 rounded border border-border px-1.5 py-1 text-xs font-semibold hover:bg-muted transition-colors cursor-pointer"
title={m.settings()}
aria-label={m.settings()}
>
<ChevronDown className="h-3 w-3" />
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1 text-[11px]">
<span>
{!isAudio && state.currentResolution
? state.currentResolution === 'Auto'
? state.activeAutoResolution
? `${m.quality_auto()} (${state.activeAutoResolution})`
: m.quality_auto()
: state.currentResolution === 'Original'
? m.original()
: state.currentResolution
: `${state.playbackRate}x`}
</span>
{!isAudio && state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuContent align="end" className="w-48 z-30">
{!isAudio && resolutions.length > 0 && (
<>
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.quality()}
</DropdownMenuLabel>
{data.media?.isHls && (
<DropdownMenuItem
onClick={() =>
changeResolution({
resolution: 'Auto',
width: data.media?.metadata?.originalWidth ?? 0,
height: data.media?.metadata?.originalHeight ?? 0,
} as DisplayTranscode)
}
className={cn(
'flex w-full items-center justify-between text-xs cursor-pointer',
state.currentResolution === 'Auto'
? 'text-primary font-medium'
: 'text-foreground',
)}
>
<span>{m.quality_auto()}</span>
</DropdownMenuItem>
)}
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
const isManualDisabled =
Boolean(data.media?.isHls) && state.isHlsManualSupported === false
return (
<DropdownMenuItem
key={itemKey}
disabled={isManualDisabled}
onClick={() => !isManualDisabled && changeResolution(res)}
className={cn(
'flex w-full items-center justify-between text-xs',
isSelected ? 'text-primary font-medium' : 'text-foreground',
isManualDisabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
)}
>
<div className="flex items-center gap-1.5">
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-[10px] text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
<DropdownMenuSeparator />
</>
)}
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.format()}
{m.playback_speed()}
</DropdownMenuLabel>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'flex items-center justify-between text-xs cursor-pointer',
state.playbackRate === rate ? 'text-primary font-medium' : 'text-foreground',
)}
>
<span>{rate}x</span>
{state.playbackRate === rate && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
))}
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
onClick={toggleLoop}
className="flex items-center justify-between text-xs cursor-pointer"
>
<span>{m.standard_time()}</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.frames()}</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>{m.timecode()}</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
<span className="flex items-center gap-1.5">
<Repeat className="h-3.5 w-3.5" />
<span>{m.loop()}</span>
</span>
{state.isLooping && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Settings (Quality + Speed) & Fullscreen */}
<div className="flex items-center gap-1 shrink-0">
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button
className="flex items-center gap-1 rounded border border-border px-1.5 py-1 text-xs font-semibold hover:bg-muted transition-colors cursor-pointer"
title={m.settings()}
aria-label={m.settings()}
>
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1 text-[11px]">
<span>
{!isAudio && state.currentResolution
? state.currentResolution === 'Original'
? m.original()
: state.currentResolution
: `${state.playbackRate}x`}
</span>
{!isAudio && state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-48 z-30">
{!isAudio && resolutions.length > 0 && (
<>
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.quality()}
</DropdownMenuLabel>
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
return (
<DropdownMenuItem
key={itemKey}
onClick={() => changeResolution(res)}
className={cn(
'flex w-full items-center justify-between text-xs cursor-pointer',
isSelected ? 'text-primary font-medium' : 'text-foreground',
)}
>
<div className="flex items-center gap-1.5">
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[9px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-[10px] text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
<DropdownMenuSeparator />
</>
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
title={m.fullscreen()}
aria-label={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-5 w-5" />
) : (
<Maximize className="h-5 w-5" />
)}
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
{m.playback_speed()}
</DropdownMenuLabel>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'flex items-center justify-between text-xs cursor-pointer',
state.playbackRate === rate ? 'text-primary font-medium' : 'text-foreground',
)}
>
<span>{rate}x</span>
{state.playbackRate === rate && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
))}
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={toggleLoop}
className="flex items-center justify-between text-xs cursor-pointer"
>
<span className="flex items-center gap-1.5">
<Repeat className="h-3.5 w-3.5" />
<span>{m.loop()}</span>
</span>
{state.isLooping && <Check className="h-3.5 w-3.5 text-primary" />}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="p-1 hover:text-primary transition-colors cursor-pointer shrink-0"
title={m.fullscreen()}
aria-label={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-5 w-5" />
) : (
<Maximize className="h-5 w-5" />
)}
</button>
</button>
</div>
</div>
</div>
</div>
@@ -1,7 +1,7 @@
import { cn } from '@/ui/lib/utils'
import { m } from '@/ui/paraglide/messages.js'
import { useUiStore } from '@/ui/stores/ui'
import type { AssetInfo, VideoTranscode } from '@shumai/dtos'
import type { AssetInfo } from '@shumai/dtos'
import {
Check,
ChevronDown,
@@ -29,6 +29,7 @@ import {
import { Slider } from '../../ui/slider'
import ProgressBar from './progress-bar'
import { formatTime, formatTimecode } from './utils'
import { getVideoResolutionLabel } from '@/ui/lib/media'
export interface PlayerState {
isPlaying: boolean
@@ -44,9 +45,21 @@ export interface PlayerState {
currentResolution: string // 'Original' or '720p', '480p' etc.
currentSrc?: string
isCurrentHdr?: boolean
activeAutoResolution?: string
isHlsManualSupported?: boolean
isLoading?: boolean
}
export type DisplayTranscode = VideoTranscode & { resolution: string }
export interface DisplayTranscode {
id?: string
url?: string
key?: string
width: number
height: number
size?: number
hdr?: boolean
resolution: string
}
export interface ControlBarProps {
state: PlayerState
@@ -81,6 +94,7 @@ export interface ControlBarProps {
onMouseLeave?: () => void
/** When false, hides the download affordance. Defaults to true. */
allowDownload?: boolean
isLoading?: boolean
}
export const VideoControlBar: React.FC<ControlBarProps> = ({
@@ -108,7 +122,9 @@ export const VideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter,
onMouseLeave,
allowDownload = true,
isLoading: propIsLoading,
}) => {
const isLoading = propIsLoading ?? state.isLoading ?? false
const { videoTimeDisplayMode, setVideoTimeDisplayMode } = useUiStore()
if (!data.media?.metadata) {
@@ -144,301 +160,341 @@ export const VideoControlBar: React.FC<ControlBarProps> = ({
onMouseEnter={onMouseEnter}
onMouseLeave={onMouseLeave}
>
{/* Progress Bar */}
<div className="mb-4 px-1">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
<div
className={cn(
'transition-opacity duration-200',
isLoading && 'pointer-events-none opacity-60 select-none',
)}
>
{/* Progress Bar */}
<div className="mb-4 px-1">
<ProgressBar
totalFrames={totalFrames}
currentFrame={currentFrame}
fps={frameRate}
buffered={buffered}
previewUrl={data.media.videoPreview?.url}
onSeek={seekToFrame}
metadata={{
...data.media.metadata,
frameRate: frameRate,
}}
/>
</div>
{/* Lower Controls Row */}
<div className="flex items-center justify-between text-foreground">
{/* Left Side: Play, Vol, Time */}
<div className="flex items-center gap-4">
<button
onClick={togglePlay}
className="hover:text-primary transition-colors"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-6 h-6 fill-current" />
) : (
<Play className="w-6 h-6 fill-current" />
)}
</button>
<button
onClick={toggleLoop}
className={cn(
'transition-colors',
state.isLooping ? 'text-primary' : 'text-muted-foreground',
)}
title="Loop"
>
<Repeat className="w-5 h-5" />
</button>
<div className="flex items-center gap-3 group/vol">
<button onClick={toggleMute} className="hover:text-primary">
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-6 h-6" />
{/* Lower Controls Row */}
<div className="flex items-center justify-between text-foreground">
{/* Left Side: Play, Vol, Time */}
<div className="flex items-center gap-4">
<button
onClick={togglePlay}
className="hover:text-primary transition-colors"
data-testid="play-toggle"
data-playing={state.isPlaying}
aria-label={state.isPlaying ? 'Pause' : 'Play'}
>
{state.isPlaying ? (
<Pause className="w-6 h-6 fill-current" />
) : (
<Volume2 className="w-6 h-6" />
<Play className="w-6 h-6 fill-current" />
)}
</button>
<div className="w-0 overflow-hidden group-hover/vol:w-24 transition-all duration-300 ease-in-out">
<div className="w-24 px-1">
<Slider
value={state.isMuted ? [0] : [state.volume]}
onValueChange={(v: number[]) => handleVolumeChange(v[0])}
max={1}
step={0.05}
/>
<button
onClick={toggleLoop}
className={cn(
'transition-colors',
state.isLooping ? 'text-primary' : 'text-muted-foreground',
)}
title="Loop"
>
<Repeat className="w-5 h-5" />
</button>
<div className="flex items-center gap-3 group/vol">
<button onClick={toggleMute} className="hover:text-primary">
{state.isMuted || state.volume === 0 ? (
<VolumeX className="w-6 h-6" />
) : (
<Volume2 className="w-6 h-6" />
)}
</button>
<div className="w-0 overflow-hidden group-hover/vol:w-24 transition-all duration-300 ease-in-out">
<div className="w-24 px-1">
<Slider
value={state.isMuted ? [0] : [state.volume]}
onValueChange={(v: number[]) => handleVolumeChange(v[0])}
max={1}
step={0.05}
/>
</div>
</div>
</div>
</div>
<div className="flex items-center gap-1 text-sm font-medium tabular-nums select-none min-w-[100px]">
<span className="text-muted-foreground font-mono" data-testid="time-readout">
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button className="h-6 w-6 inline-flex items-center justify-center rounded-md hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none">
<ChevronDown className="h-3.5 w-3.5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
Format
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Standard Time</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Frames</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Timecode</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Zoom, Speed, Res, Download, Fullscreen */}
<div className="relative flex items-center gap-3">
{/* Zoom Controls */}
{!isAudio && (
<div className="flex items-center gap-1">
<div className="flex items-center gap-1 rounded-md bg-muted p-0.5">
<button
onClick={() => onZoomChange(zoom * 0.8)}
className="p-1 hover:text-primary rounded"
title={m.zoom_out()}
>
<Minus size={14} />
</button>
<span className="text-xs w-8 text-center tabular-nums">
{Math.round(zoom * 100)}%
</span>
<button
onClick={() => onZoomChange(zoom * 1.2)}
className="p-1 hover:text-primary rounded"
title={m.zoom_in()}
>
<Plus size={14} />
</button>
</div>
<button
onClick={onZoomReset}
className="text-xs font-medium px-2 py-1 rounded bg-muted hover:text-primary transition-colors"
>
{m.fit()}
</button>
</div>
)}
{/* Playback Rate */}
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="w-8 text-sm font-bold hover:text-primary">
{state.playbackRate}x
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'rounded py-1 px-2 text-left text-xs',
state.playbackRate === rate
? 'bg-primary text-primary-foreground focus:bg-primary focus:text-primary-foreground'
: 'text-foreground',
)}
>
{rate}x
</DropdownMenuItem>
))}
</DropdownMenuContent>
</DropdownMenu>
{/* Settings / Resolution */}
{!isAudio && resolutions.length > 0 && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="flex items-center gap-1.5 rounded border border-border px-2 py-0.5 text-sm font-semibold hover:bg-muted transition-colors">
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1">
<span>
{state.currentResolution === 'Original'
? m.original()
: state.currentResolution}
</span>
{state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.quality()}</DropdownMenuLabel>
{resolutions.map((res) => {
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
return (
<DropdownMenuItem
key={itemKey}
onClick={() => changeResolution(res)}
className={cn(
'flex w-full items-center justify-between',
isSelected ? 'text-primary font-medium' : 'text-foreground',
)}
>
<div className="flex items-center gap-1.5">
<span>{res.resolution === 'Original' ? m.original() : res.resolution}</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="pl-5 text-xs text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
</DropdownMenuContent>
</DropdownMenu>
)}
{/* Download */}
{allowDownload &&
(isAudio ? (
<button
onClick={() => handleDownload(data.media?.original?.key || '')}
className="transition-colors hover:text-primary"
title={m.download()}
disabled={!data.media?.original?.key}
>
<Download className="h-5 w-5" />
</button>
) : (
<DropdownMenu>
<div className="flex items-center gap-1 text-sm font-medium tabular-nums select-none min-w-[100px]">
<span className="text-muted-foreground font-mono" data-testid="time-readout">
{displayString}
</span>
<DropdownMenu modal={false}>
<DropdownMenuTrigger asChild>
<button className="transition-colors hover:text-primary" title={m.download()}>
<Download className="h-5 w-5" />
<button className="h-6 w-6 inline-flex items-center justify-center rounded-md hover:bg-muted text-muted-foreground hover:text-foreground transition-colors cursor-pointer focus:outline-none">
<ChevronDown className="h-3.5 w-3.5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" className="w-40 z-30">
<DropdownMenuLabel className="text-[10px] font-bold text-muted-foreground uppercase tracking-wider">
Format
</DropdownMenuLabel>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('standard')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Standard Time</span>
{videoTimeDisplayMode === 'standard' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('frames')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Frames</span>
{videoTimeDisplayMode === 'frames' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => setVideoTimeDisplayMode('timecode')}
className="flex items-center justify-between cursor-pointer text-xs"
>
<span>Timecode</span>
{videoTimeDisplayMode === 'timecode' && (
<Check className="h-3.5 w-3.5 text-primary" />
)}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
{/* Right Side: Zoom, Speed, Res, Download, Fullscreen */}
<div className="relative flex items-center gap-3">
{/* Zoom Controls */}
{!isAudio && (
<div className="flex items-center gap-1">
<div className="flex items-center gap-1 rounded-md bg-muted p-0.5">
<button
onClick={() => onZoomChange(zoom * 0.8)}
className="p-1 hover:text-primary rounded"
title={m.zoom_out()}
>
<Minus size={14} />
</button>
<span className="text-xs w-8 text-center tabular-nums">
{Math.round(zoom * 100)}%
</span>
<button
onClick={() => onZoomChange(zoom * 1.2)}
className="p-1 hover:text-primary rounded"
title={m.zoom_in()}
>
<Plus size={14} />
</button>
</div>
<button
onClick={onZoomReset}
className="text-xs font-medium px-2 py-1 rounded bg-muted hover:text-primary transition-colors"
>
{m.fit()}
</button>
</div>
)}
{/* Playback Rate */}
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="w-8 text-sm font-bold hover:text-primary">
{state.playbackRate}x
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
{[0.5, 1, 1.5, 2].map((rate) => (
<DropdownMenuItem
key={rate}
onClick={() => changePlaybackRate(rate)}
className={cn(
'rounded py-1 px-2 text-left text-xs',
state.playbackRate === rate
? 'bg-primary text-primary-foreground focus:bg-primary focus:text-primary-foreground'
: 'text-foreground',
)}
>
{rate}x
</DropdownMenuItem>
))}
</DropdownMenuContent>
</DropdownMenu>
{/* Settings / Resolution */}
{!isAudio && resolutions.length > 0 && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="flex items-center gap-1.5 rounded border border-border px-2 py-0.5 text-sm font-semibold hover:bg-muted transition-colors">
<Settings className="h-3.5 w-3.5" />
<span className="flex items-center gap-1">
<span>
{state.currentResolution === 'Auto'
? state.activeAutoResolution
? `${m.quality_auto()} (${state.activeAutoResolution})`
: m.quality_auto()
: state.currentResolution === 'Original'
? m.original()
: state.currentResolution}
</span>
{state.isCurrentHdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
<DropdownMenuLabel>{m.quality()}</DropdownMenuLabel>
{data.media?.isHls && (
<DropdownMenuItem
onClick={() =>
changeResolution({
resolution: 'Auto',
width: data.media?.metadata?.originalWidth ?? 0,
height: data.media?.metadata?.originalHeight ?? 0,
} as DisplayTranscode)
}
className={cn(
'flex w-full items-center justify-between cursor-pointer',
state.currentResolution === 'Auto'
? 'text-primary font-medium'
: 'text-foreground',
)}
>
<span>{m.quality_auto()}</span>
</DropdownMenuItem>
)}
{resolutions.map((res) => {
const itemKey = res.key || `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const itemKey = `${res.resolution}-${res.hdr ? 'hdr' : 'sdr'}`
const isSelected =
state.currentResolution === res.resolution &&
Boolean(state.isCurrentHdr) === Boolean(res.hdr)
const isManualDisabled =
Boolean(data.media?.isHls) && state.isHlsManualSupported === false
return (
<DropdownMenuItem
key={itemKey}
onClick={() => handleDownload(res.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground cursor-pointer"
disabled={isManualDisabled}
onClick={() => !isManualDisabled && changeResolution(res)}
className={cn(
'flex w-full items-center justify-between',
isSelected ? 'text-primary font-medium' : 'text-foreground',
isManualDisabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
)}
>
<div className="flex items-center gap-1.5">
<span>{res.resolution}</span>
<span>
{res.resolution === 'Original' ? m.original() : res.resolution}
</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-xs text-muted-foreground">MP4</span>
<span className="pl-5 text-xs text-muted-foreground">
{res.width}x{res.height}
</span>
</DropdownMenuItem>
)
})}
{data.media?.original?.key && (
<>
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => handleDownload(data.media?.original?.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground"
>
<span>{m.original()}</span>
<span className="text-xs text-muted-foreground">
{data.name?.split('.').pop()?.toUpperCase() || 'RAW'}
</span>
</DropdownMenuItem>
</>
)}
</DropdownMenuContent>
</DropdownMenu>
))}
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="transition-colors hover:text-primary"
title={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-6 w-6" />
) : (
<Maximize className="h-6 w-6" />
)}
</button>
{/* Download */}
{allowDownload &&
(isAudio ? (
<button
onClick={() => handleDownload(data.media?.original?.key || '')}
className="transition-colors hover:text-primary"
title={m.download()}
disabled={!data.media?.original?.key}
>
<Download className="h-5 w-5" />
</button>
) : (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button className="transition-colors hover:text-primary" title={m.download()}>
<Download className="h-5 w-5" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuLabel>{m.download()}</DropdownMenuLabel>
{(data.media?.videoTranscodes && data.media.videoTranscodes.length > 0
? data.media.videoTranscodes
: resolutions
).map((res) => {
const resLabel = getVideoResolutionLabel(res)
const itemKey = res.key || `${resLabel}-${res.hdr ? 'hdr' : 'sdr'}`
return (
<DropdownMenuItem
key={itemKey}
onClick={() => handleDownload(res.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground cursor-pointer"
>
<div className="flex items-center gap-1.5">
<span>{resLabel}</span>
{res.hdr && (
<span className="px-1 py-0.2 rounded text-[10px] font-bold bg-amber-500/20 text-amber-500 border border-amber-500/30">
{m.hdr()}
</span>
)}
</div>
<span className="text-xs text-muted-foreground">MP4</span>
</DropdownMenuItem>
)
})}
{data.media?.original?.key && (
<>
<DropdownMenuSeparator />
<DropdownMenuItem
onClick={() => handleDownload(data.media?.original?.key ?? '')}
className="flex w-full items-center justify-between px-3 py-2 text-left text-sm text-foreground"
>
<span>{m.original()}</span>
<span className="text-xs text-muted-foreground">
{data.name?.split('.').pop()?.toUpperCase() || 'RAW'}
</span>
</DropdownMenuItem>
</>
)}
</DropdownMenuContent>
</DropdownMenu>
))}
{/* Fullscreen */}
<button
onClick={toggleFullScreen}
className="transition-colors hover:text-primary"
title={m.fullscreen()}
>
{state.isFullScreen ? (
<Minimize className="h-6 w-6" />
) : (
<Maximize className="h-6 w-6" />
)}
</button>
</div>
</div>
</div>
</div>
@@ -1,28 +1,52 @@
// @vitest-environment happy-dom
import { act, cleanup, render } from '@testing-library/react'
import { act, cleanup, fireEvent, render } from '@testing-library/react'
import React from 'react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import VideoViewer from './video-viewer'
import type { AssetInfo } from '@shumai/dtos'
const videojsMock = vi.fn((...args: unknown[]) => {
void args
return {
const { mockHlsInstance, mockHlsConstructor } = vi.hoisted(() => {
const instance = {
attachMedia: vi.fn(),
loadSource: vi.fn(),
on: vi.fn(),
volume: vi.fn(),
duration: vi.fn(() => 10),
currentTime: vi.fn(() => 0),
bufferedEnd: vi.fn(() => 0),
playbackRate: vi.fn(() => 1),
ready: vi.fn(),
dispose: vi.fn(),
isDisposed: vi.fn(() => false),
off: vi.fn(),
once: vi.fn(),
destroy: vi.fn(),
levels: [
{ height: 1080, width: 1920, bitrate: 5000000 },
{ height: 720, width: 1280, bitrate: 2500000 },
],
currentLevel: -1,
startLoad: vi.fn(),
recoverMediaError: vi.fn(),
}
const constructor = vi.fn(function () {
return instance
})
return { mockHlsInstance: instance, mockHlsConstructor: constructor }
})
vi.mock('video.js', () => ({
default: (...args: unknown[]) => videojsMock(...args),
}))
vi.mock('hls.js', () => {
const MockHls = mockHlsConstructor
// @ts-expect-error adding static mock property
MockHls.isSupported = vi.fn(() => true)
// @ts-expect-error adding static mock property
MockHls.Events = {
MANIFEST_PARSED: 'hlsManifestParsed',
LEVEL_SWITCHED: 'hlsLevelSwitched',
ERROR: 'hlsError',
}
// @ts-expect-error adding static mock property
MockHls.ErrorTypes = {
NETWORK_ERROR: 'networkError',
MEDIA_ERROR: 'mediaError',
OTHER_ERROR: 'otherError',
}
return {
default: MockHls,
}
})
vi.mock('@/ui/components/drawing-canvas', () => ({
default: () => <div data-testid="drawing-canvas" />,
@@ -34,7 +58,8 @@ vi.mock('@/ui/paraglide/messages.js', () => ({
describe('VideoViewer', () => {
beforeEach(() => {
videojsMock.mockClear()
vi.clearAllMocks()
mockHlsInstance.currentLevel = -1
})
afterEach(() => {
@@ -88,39 +113,20 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
const { rerender } = render(<VideoViewer file={videoA} />)
const { container, rerender } = render(<VideoViewer file={videoA} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Initial render should initialize videojs with video A URL
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/videoA-720p.mp4',
type: 'video/mp4',
},
],
}),
)
// Initial render should set video element src with video A URL
expect(video.src).toBe('https://cdn.example.com/videoA-720p.mp4')
// Rerender with video B (simulating selecting another video from the carousel)
rerender(<VideoViewer file={videoB} />)
// VideoJS should be re-initialized with video B's URL
expect(videojsMock).toHaveBeenLastCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/videoB-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
// Native video element should now have video B's URL
expect(video.src).toBe('https://cdn.example.com/videoB-1080p.mp4')
})
it('initializes VideoJS when transcode URL becomes available after initially empty with same ID', () => {
it('initializes video when transcode URL becomes available after initially empty with same ID', () => {
const initialVideo: AssetInfo = {
id: 'video-pending',
name: 'pending.mp4',
@@ -136,7 +142,6 @@ describe('VideoViewer', () => {
videoTranscodes: [
{
resolution: '1080p',
// URL not yet presigned (e.g. from file list payload)
url: undefined as unknown as string,
width: 1920,
height: 1080,
@@ -160,27 +165,17 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
const { rerender } = render(<VideoViewer file={initialVideo} autoPlay={true} />)
const { container, rerender } = render(<VideoViewer file={initialVideo} autoPlay={true} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Initially targetSrc is empty, videojs should not be initialized
expect(videojsMock).not.toHaveBeenCalled()
// Initially targetSrc is empty, video.src should be empty
expect(video.src).toBe('')
// Rerender with detailedVideo (same id, but signed URL now available)
rerender(<VideoViewer file={detailedVideo} autoPlay={true} />)
// VideoJS should now be initialized with the new URL
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
autoplay: false,
sources: [
{
src: 'https://cdn.example.com/signed-pending-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
// Video element should now have the newly available signed URL
expect(video.src).toBe('https://cdn.example.com/signed-pending-1080p.mp4')
})
it('auto-selects HDR proxy when display supports HDR (dynamic-range: high)', () => {
@@ -227,19 +222,10 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
render(<VideoViewer file={hdrVideo} />)
const { container } = render(<VideoViewer file={hdrVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/video-1080p-hdr.mp4',
type: 'video/mp4',
},
],
}),
)
expect(video.src).toBe('https://cdn.example.com/video-1080p-hdr.mp4')
window.matchMedia = originalMatchMedia
})
@@ -288,39 +274,15 @@ describe('VideoViewer', () => {
},
} as unknown as AssetInfo
render(<VideoViewer file={hdrVideo} />)
const { container } = render(<VideoViewer file={hdrVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(videojsMock).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
sources: [
{
src: 'https://cdn.example.com/video-1080p.mp4',
type: 'video/mp4',
},
],
}),
)
expect(video.src).toBe('https://cdn.example.com/video-1080p.mp4')
window.matchMedia = originalMatchMedia
})
it('shows play button and darkened overlay initially, but never again after playing (even when paused or scrubbed back to start)', () => {
const listeners: Record<string, () => void> = {}
videojsMock.mockImplementation(() => ({
on: vi.fn((event: string, cb: () => void) => {
listeners[event] = cb
}),
volume: vi.fn(),
duration: vi.fn(() => 10),
currentTime: vi.fn(() => 0),
bufferedEnd: vi.fn(() => 0),
playbackRate: vi.fn(() => 1),
ready: vi.fn(),
dispose: vi.fn(),
isDisposed: vi.fn(() => false),
}))
const testVideo: AssetInfo = {
id: 'test-video-lifecycle',
name: 'test.mp4',
@@ -346,27 +308,503 @@ describe('VideoViewer', () => {
const { container } = render(<VideoViewer file={testVideo} />)
const videoArea = container.querySelector('[data-testid="video-area"]')
const video = videoArea?.querySelector('video') as HTMLVideoElement
expect(videoArea).toBeTruthy()
expect(video).toBeTruthy()
// 1. Initially (before playing): play button & 20% black tint are present
expect(videoArea?.querySelector('.bg-black\\/20')).not.toBeNull()
// 2. Start playback
act(() => {
listeners['play']?.()
video.dispatchEvent(new Event('play'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
// 3. Pause video: overlay must NOT show again
act(() => {
listeners['pause']?.()
video.dispatchEvent(new Event('pause'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
// 4. Drag seekbar / time back to 0: overlay must STILL NOT show
act(() => {
listeners['timeupdate']?.()
video.dispatchEvent(new Event('timeupdate'))
})
expect(videoArea?.querySelector('.bg-black\\/20')).toBeNull()
})
it('initializes Hls.js with capLevelToPlayerSize false when video is HLS', () => {
const hlsVideo: AssetInfo = {
id: 'video-hls',
name: 'video-hls.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(mockHlsConstructor).toHaveBeenCalledWith(
expect.objectContaining({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
}),
)
expect(mockHlsInstance.attachMedia).toHaveBeenCalledWith(video)
expect(mockHlsInstance.loadSource).toHaveBeenCalledWith('https://cdn.example.com/master.m3u8')
})
it('switches HLS resolution immediately using currentLevel', () => {
const hlsVideo: AssetInfo = {
id: 'video-hls-switch',
name: 'video-hls-switch.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
// Open settings / quality dropdown
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton).toBeTruthy()
fireEvent.pointerDown(settingsButton!, { pointerType: 'mouse', button: 0 })
fireEvent.click(settingsButton!)
// Click 720p (index 1 in levels: 1080p is index 0, 720p is index 1)
const items = document.querySelectorAll('[role="menuitem"]')
const item720p = Array.from(items).find((el) => el.textContent?.includes('720p'))
expect(item720p).toBeTruthy()
act(() => {
fireEvent.click(item720p!)
})
// currentLevel should be set immediately to 1 (720p)
expect(mockHlsInstance.currentLevel).toBe(1)
// Click Auto
const itemAuto = Array.from(document.querySelectorAll('[role="menuitem"]')).find(
(el) => el.textContent?.includes('Auto') || el.textContent === '',
)
if (itemAuto) {
act(() => {
fireEvent.click(itemAuto)
})
// Auto should reset currentLevel to -1
expect(mockHlsInstance.currentLevel).toBe(-1)
}
})
it('does not hijack subsequent seeks if paused HLS switch did not emit seeked', () => {
vi.useFakeTimers()
try {
const hlsVideo: AssetInfo = {
id: 'video-hls-stale-seek',
name: 'video-hls-stale-seek.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [
{ width: 1920, height: 1080, resolution: '1080p' },
{ width: 1280, height: 720, resolution: '720p' },
],
},
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Video is paused at 0s (frame 0)
Object.defineProperty(video, 'paused', { value: true, configurable: true })
video.currentTime = 0
// Open settings and click 720p
const settingsButton = container.querySelector('button:has(.lucide-settings)')
fireEvent.pointerDown(settingsButton!, { pointerType: 'mouse', button: 0 })
fireEvent.click(settingsButton!)
const item720p = Array.from(document.querySelectorAll('[role="menuitem"]')).find((el) =>
el.textContent?.includes('720p'),
)
act(() => {
fireEvent.click(item720p!)
})
// Simulate that no seeked event occurred within the timeout (e.g. 1500ms elapses)
act(() => {
vi.advanceTimersByTime(2000)
})
// Now user performs an unrelated seek to frame 150 (currentTime = 5.0)
video.currentTime = 5.0
act(() => {
fireEvent(video, new Event('seeked'))
})
// video.currentTime must remain at frame 150, NOT rewound to frame 0
expect(Math.floor(video.currentTime * 30 + 0.001)).toBe(150)
expect(video.currentTime).toBeCloseTo(5.0167, 3)
} finally {
vi.useRealTimers()
}
})
it('displays 2160p instead of 3840p for 4K vertical video when HLS switches level on Auto', () => {
mockHlsInstance.levels = [{ height: 3840, width: 2160, bitrate: 10000000 }]
const hlsVideo: AssetInfo = {
id: 'video-hls-vertical-4k',
name: 'vertical-4k.m3u8',
proxyType: 'video',
media: {
isHls: true,
hls: {
key: 'hls-key',
url: 'https://cdn.example.com/master.m3u8',
resolutions: [{ width: 2160, height: 3840, resolution: '2160p' }],
},
metadata: {
originalWidth: 2160,
originalHeight: 3840,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={hlsVideo} />)
const levelSwitchedHandler = mockHlsInstance.on.mock.calls.find(
([event]) => event === 'hlsLevelSwitched',
)?.[1]
expect(levelSwitchedHandler).toBeDefined()
act(() => {
levelSwitchedHandler('hlsLevelSwitched', { level: 0 })
})
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton?.textContent).toContain('2160p')
expect(settingsButton?.textContent).not.toContain('3840p')
})
it('derives resolution from long side for legacy portrait MP4 transcode without explicit resolution', () => {
const legacyPortraitVideo: AssetInfo = {
id: 'legacy-portrait-mp4',
name: 'portrait.mp4',
proxyType: 'video',
media: {
videoTranscodes: [
{
key: 'portrait-1080p.mp4',
url: 'https://cdn.example.com/portrait-1080p.mp4',
width: 1080,
height: 1920,
size: 5000000,
},
],
metadata: {
originalWidth: 1080,
originalHeight: 1920,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={legacyPortraitVideo} />)
const settingsButton = container.querySelector('button:has(.lucide-settings)')
expect(settingsButton?.textContent).toContain('1080p')
expect(settingsButton?.textContent).not.toContain('1920p')
})
it('sets poster on video element from file.preview.thumbnailUrl', () => {
const videoWithThumb: AssetInfo = {
id: 'video-thumb',
name: 'thumb.mp4',
proxyType: 'video',
preview: {
thumbnailUrl: 'https://cdn.example.com/poster.jpg',
},
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={videoWithThumb} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
expect(video.getAttribute('poster')).toBe('https://cdn.example.com/poster.jpg')
})
it('shows loading spinner initially and transitions to play overlay on loadeddata', () => {
const testVideo: AssetInfo = {
id: 'video-loading-test',
name: 'loading.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// 1. Initially loading spinner is visible, play overlay is not
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
expect(container.querySelector('[data-testid="video-play-overlay"]')).toBeNull()
// 2. Control bar inner wrapper is dimmed & non-interactive
const controlBarWrapper = container.querySelector('.pointer-events-none.opacity-60')
expect(controlBarWrapper).not.toBeNull()
// 3. Dispatch loadeddata
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
// Spinner is gone, play overlay is visible, controls are interactive
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
expect(container.querySelector('[data-testid="video-play-overlay"]')).not.toBeNull()
expect(container.querySelector('.pointer-events-none.opacity-60')).toBeNull()
})
it('debounces waiting stall with 200ms delay and cancels spinner on quick recovery', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'video-stall-test',
name: 'stall.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// Become ready first
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Stalls briefly (<200ms)
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Immediately after waiting, spinner should NOT be shown yet (debounced)
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Recovers quickly at 100ms
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('playing'))
})
act(() => {
vi.advanceTimersByTime(150)
})
// Spinner still not shown
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// Now a long stall (>200ms)
act(() => {
video.dispatchEvent(new Event('waiting'))
})
act(() => {
vi.advanceTimersByTime(250)
})
// Spinner should now appear!
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
// Resume playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
it('does not cancel waiting stall debounce when progress events fire during stall', () => {
vi.useFakeTimers()
try {
const testVideo: AssetInfo = {
id: 'video-stall-progress-test',
name: 'stall-progress.mp4',
proxyType: 'video',
media: {
metadata: {
originalWidth: 1920,
originalHeight: 1080,
duration: 10,
frameRate: 30,
totalFrames: 300,
},
videoTranscodes: [
{
resolution: '1080p',
url: 'https://cdn.example.com/video-1080p.mp4',
width: 1920,
height: 1080,
},
],
},
} as unknown as AssetInfo
const { container } = render(<VideoViewer file={testVideo} />)
const video = container.querySelector('[data-testid="video-area"] video') as HTMLVideoElement
// 1. Initial ready state
act(() => {
video.dispatchEvent(new Event('loadeddata'))
})
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 2. Play starts (paused === false)
Object.defineProperty(video, 'paused', { value: false, configurable: true, writable: true })
act(() => {
video.dispatchEvent(new Event('playing'))
})
// 3. Stalls: dispatches waiting
act(() => {
video.dispatchEvent(new Event('waiting'))
})
// Spinner not visible immediately (within 200ms debounce)
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 4. Progress event fires at 100ms while paused is false (data still trickling in)
act(() => {
vi.advanceTimersByTime(100)
video.dispatchEvent(new Event('progress'))
})
// Should NOT clear debounce or mark loading false
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
// 5. Advance past 200ms debounce (advance remaining 110ms)
act(() => {
vi.advanceTimersByTime(110)
})
// Spinner should now appear despite the progress event!
expect(container.querySelector('[data-testid="video-loading-spinner"]')).not.toBeNull()
// 6. Playback resumes: dispatch playing
act(() => {
video.dispatchEvent(new Event('playing'))
})
// Spinner disappears
expect(container.querySelector('[data-testid="video-loading-spinner"]')).toBeNull()
} finally {
vi.useRealTimers()
}
})
it('renders failed state when file has no media and status is failed', () => {
const failedVideo = {
id: 'failed-video',
name: 'failed-video.mp4',
proxyType: 'video',
status: 'failed',
media: {
error: 'Transcoding failed due to server error',
},
} as unknown as AssetInfo
const { getByTestId, getByText } = render(<VideoViewer file={failedVideo} />)
expect(getByTestId('video-viewer-failed-state')).toBeDefined()
expect(getByText('Transcoding failed due to server error')).toBeDefined()
})
})
@@ -1,16 +1,17 @@
import { client } from '@/ui/api/client'
import { m } from '@/ui/paraglide/messages.js'
import { cn } from '@/ui/lib/utils'
import { Play, AudioLines } from 'lucide-react'
import { Play, AudioLines, Loader2, AlertCircle } from 'lucide-react'
import React, { useCallback, useEffect, useRef, useState, useImperativeHandle } from 'react'
import videojs from 'video.js'
import type Player from 'video.js/dist/types/player'
import Hls from 'hls.js'
import { useFramePlayer } from './use-frame-player'
import { resolveTotalFrames } from './utils'
import { calculateFrameCenterTime, resolveTotalFrames } from './utils'
import { VideoControlBar, type PlayerState, type DisplayTranscode } from './video-control-bar'
import { MobileVideoControlBar } from './mobile-video-control-bar'
import { useIsMobile } from '@/ui/hooks/use-mobile'
import DrawingCanvas from '@/ui/components/drawing-canvas'
import { useAnnotationStore } from '@/ui/stores/annotation-store'
import { getVideoResolutionLabel } from '@/ui/lib/media'
import { FileViewerProps, MediaController } from '../types'
import { centeredPan, fitScale, zoomAtPoint } from '../pan-zoom'
import { usePanZoomGestures } from '../use-pan-zoom'
@@ -31,28 +32,22 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
},
ref,
) => {
const localPlayerRef = useRef<Player | null>(null)
const playerRef = localPlayerRef
const resolutions: DisplayTranscode[] = (data.media?.videoTranscodes ?? []).map((t) => {
const longSide = Math.max(t.width, t.height)
let resolution = `${t.height}p`
if (longSide >= 3840) resolution = '2160p'
else if (longSide >= 1920) resolution = '1080p'
else if (longSide >= 1280) resolution = '720p'
else if (longSide >= 960) resolution = '540p'
else if (longSide >= 640) resolution = '360p'
else if (longSide >= 320) resolution = '180p'
const hlsRef = useRef<Hls | null>(null)
const videoRef = useRef<HTMLVideoElement | null>(null)
const [activeAutoResolution, setActiveAutoResolution] = useState<string | undefined>(undefined)
const [isHlsManualSupported, setIsHlsManualSupported] = useState<boolean>(true)
const isHls = Boolean(data.media?.isHls && data.media.hls?.url)
const baseTranscodes = isHls
? (data.media?.hls?.resolutions ?? [])
: (data.media?.videoTranscodes ?? [])
return {
...t,
resolution,
}
})
const resolutions: DisplayTranscode[] = baseTranscodes.map((t) => ({
...t,
resolution: getVideoResolutionLabel(t),
}))
// Only transcoded proxy versions are ever displayed; the raw original file
// is never used as a playback source.
const hasMedia = resolutions.length > 0 && !!data.media?.metadata
// We use a container ref to manually append the video element
const videoContainerRef = useRef<HTMLDivElement>(null)
const hasMedia = (isHls || resolutions.length > 0) && !!data.media?.metadata
// Logic to select best resolution based on screen size and dynamic range
const getInitialResolution = (): DisplayTranscode | null => {
@@ -90,7 +85,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
return bestFit || sortedResolutions[sortedResolutions.length - 1]
}
const initialRes = getInitialResolution()
const initialRes = isHls ? null : getInitialResolution()
// State
const [state, setState] = useState<PlayerState>({
@@ -104,66 +99,75 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
playbackRate: 1,
isFullScreen: false,
showFrames: false,
currentResolution: initialRes?.resolution ?? '',
currentSrc: initialRes?.url ?? '',
currentResolution: isHls ? 'Auto' : (initialRes?.resolution ?? ''),
currentSrc: isHls ? data.media!.hls!.url : (initialRes?.url ?? ''),
isCurrentHdr: initialRes?.hdr ?? false,
})
const [hasStartedPlaying, setHasStartedPlaying] = useState(false)
const containerRef = useRef<HTMLDivElement>(null)
const rootRef = useRef<HTMLDivElement>(null)
const controlsTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const pendingResolutionCorrectionRef = useRef<(() => void) | null>(null)
useImperativeHandle(ref, () => ({
play: () => {
if (localPlayerRef.current) {
localPlayerRef.current.play()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
videoRef.current?.play().catch(() => {})
},
pause: () => {
if (localPlayerRef.current) {
localPlayerRef.current.pause()
}
videoRef.current?.pause()
},
seekTo: (second: number) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
if (localPlayerRef.current) {
localPlayerRef.current.currentTime(second)
if (videoRef.current) {
videoRef.current.currentTime = second
}
},
getCurrentTime: () => {
return (localPlayerRef.current ? localPlayerRef.current.currentTime() : 0) ?? 0
return (videoRef.current ? videoRef.current.currentTime : 0) ?? 0
},
getDuration: () => {
return (localPlayerRef.current ? localPlayerRef.current.duration() : 0) ?? 0
return (videoRef.current ? videoRef.current.duration : 0) ?? 0
},
}))
const isOverControlsRef = useRef(false)
const [videoHtmlEl, setVideoHtmlEl] = useState<HTMLVideoElement | undefined>(undefined)
const videoRef = useRef<HTMLVideoElement | null>(null)
const [zoom, setZoom] = useState(1)
const [pan, setPan] = useState({ x: 0, y: 0 })
const [hasManuallyZoomed, setHasManuallyZoomed] = useState(false)
const [containerSize, setContainerSize] = useState({ width: 0, height: 0 })
useEffect(() => {
const res = getInitialResolution()
const res = isHls ? null : getInitialResolution()
setState((prev) => ({
...prev,
isPlaying: false,
progress: 0,
currentTime: 0,
duration: data.media?.metadata?.duration || 0,
currentResolution: res?.resolution ?? '',
currentSrc: res?.url ?? '',
currentResolution: isHls ? 'Auto' : (res?.resolution ?? ''),
currentSrc: isHls ? (data.media?.hls?.url ?? '') : (res?.url ?? ''),
isCurrentHdr: res?.hdr ?? false,
}))
setActiveAutoResolution(undefined)
setHasManuallyZoomed(false)
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
setBuffered(0)
lastProcessedStartTimeRef.current = null
setHasStartedPlaying(false)
}, [data.id, Boolean(initialRes?.url)])
}, [data.id, Boolean(isHls ? data.media?.hls?.url : initialRes?.url)])
const vidW = data.media?.metadata?.originalWidth || 1920
const vidH = data.media?.metadata?.originalHeight || 1080
@@ -252,6 +256,8 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const [buffered, setBuffered] = useState(0)
const [isControlsVisible, setIsControlsVisible] = useState(true)
const [isPlayerReady, setIsPlayerReady] = useState(false)
const [isLoading, setIsLoading] = useState(true)
const waitingTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const lastProcessedStartTimeRef = useRef<number | null>(null)
// Frame-accurate hook and derived state
@@ -266,6 +272,10 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const handleSeekToFrame = useCallback(
(frame: number) => {
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setHasStartedPlaying(true)
return seekToFrame(frame)
},
@@ -281,6 +291,9 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
currentTime,
duration,
progress,
activeAutoResolution,
isHlsManualSupported,
isLoading,
}
// Trigger time update event reactively
@@ -290,146 +303,258 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
}
}, [currentFrame, frameRate, onTimeUpdate])
// Initialize Video.js
// Initialize player (Hls.js or native HTML5 video)
const targetSrc = isHls ? (data.media?.hls?.url ?? '') : (getInitialResolution()?.url ?? '')
useEffect(() => {
if (!videoContainerRef.current) return
const video = videoRef.current
if (!video) return
const initialRes = getInitialResolution()
const targetSrc = initialRes?.url ?? ''
if (!targetSrc) return
// Clean up previous player if exists
if (playerRef.current && !playerRef.current.isDisposed()) {
playerRef.current.dispose()
playerRef.current = null
videoRef.current = null
// Clean up previous HLS instance if exists
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
// 1. Create the video element manually
// This avoids conflicts with React Strict Mode where the DOM might not match React's virtual DOM expectations after Video.js modifies it.
const videoElement = document.createElement('video-js')
videoElement.classList.add('vjs-big-play-centered', '!h-full', '!w-full')
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
setIsPlayerReady(false)
setIsLoading(true)
setActiveAutoResolution(undefined)
// Keep it in DOM, enable visible rendering
videoElement.style.pointerEvents = 'none'
const startAutoPlay = () => {
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch(() => {
video.muted = true
setState((prev) => ({ ...prev, isMuted: true }))
video.play()?.catch(() => {})
})
}
}
// Append to our container
videoContainerRef.current.appendChild(videoElement)
let hls: Hls | null = null
// 2. Initialize the player
const player = (playerRef.current = videojs(videoElement, {
controls: false,
autoplay: false,
preload: 'auto',
playsinline: true,
sources: [
{
src: targetSrc,
type: 'video/mp4',
},
],
}))
if (isHls) {
if (Hls.isSupported()) {
setIsHlsManualSupported(true)
hls = new Hls({
autoStartLoad: true,
startLevel: -1,
capLevelToPlayerSize: false,
enableWorker: true,
})
hlsRef.current = hls
hls.attachMedia(video)
hls.loadSource(targetSrc)
if (autoPlay) {
const startAutoPlay = () => {
if (player.isDisposed()) return
const playPromise = player.play()
if (playPromise !== undefined) {
playPromise.catch(() => {
if (player.isDisposed()) return
player.muted(true)
setState((prev) => ({ ...prev, isMuted: true }))
player.play()?.catch(() => {})
})
hls.on(Hls.Events.MANIFEST_PARSED, () => {
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
video.addEventListener('canplay', startAutoPlay, { once: true })
}
}
})
hls.on(Hls.Events.LEVEL_SWITCHED, (_event, eventData) => {
const level = hls?.levels[eventData.level]
if (level) {
const matched = resolutions.find(
(r) => r.width === level.width && r.height === level.height,
)
if (matched?.resolution) {
setActiveAutoResolution(matched.resolution)
} else {
const shortSide =
level.width && level.height
? Math.min(level.width, level.height)
: level.height || level.width
setActiveAutoResolution(shortSide ? `${shortSide}p` : undefined)
}
}
})
hls.on(Hls.Events.ERROR, (_event, eventData) => {
if (eventData.fatal) {
switch (eventData.type) {
case Hls.ErrorTypes.NETWORK_ERROR:
hls?.startLoad()
break
case Hls.ErrorTypes.MEDIA_ERROR:
hls?.recoverMediaError()
break
default:
hls?.destroy()
break
}
}
})
} else if (video.canPlayType('application/vnd.apple.mpegurl')) {
// Native Safari HLS fallback
setIsHlsManualSupported(false)
video.src = targetSrc
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
video.addEventListener('canplay', startAutoPlay, { once: true })
}
}
}
player.ready(() => {
if (player.isDisposed()) return
if (player.readyState() >= 2) {
} else {
// Native MP4
video.src = targetSrc
if (autoPlay) {
if (video.readyState >= 2) {
startAutoPlay()
} else {
player.one('canplay', startAutoPlay)
video.addEventListener('canplay', startAutoPlay, { once: true })
}
})
}
}
// Extract video element for Konva
const htmlVid = videoElement.querySelector('video')
if (htmlVid) {
setVideoHtmlEl(htmlVid)
videoRef.current = htmlVid
} else {
// Fallback: try player.tech().el()
// Wait for ready?
player.ready(() => {
const techEl = player.tech({ iWillNotUseThisInPlugins: true })?.el() as HTMLVideoElement
if (techEl) {
setVideoHtmlEl(techEl)
videoRef.current = techEl
}
})
const clearWaitingTimeout = () => {
if (waitingTimeoutRef.current) {
clearTimeout(waitingTimeoutRef.current)
waitingTimeoutRef.current = null
}
}
player.on('play', () => {
const handlePlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
setState((p) => ({ ...p, isPlaying: true }))
setHasStartedPlaying(true)
onPlay?.()
})
player.on('pause', () => {
}
const handlePause = () => {
setState((p) => ({ ...p, isPlaying: false }))
onPause?.()
})
player.on('ended', () => setState((p) => ({ ...p, isPlaying: false })))
player.on('loadedmetadata', () => {
}
const handleEnded = () => {
clearWaitingTimeout()
setIsLoading(false)
setState((p) => ({ ...p, isPlaying: false }))
}
const handleLoadedData = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
})
player.on('loadstart', () => {
}
const handleCanPlay = () => {
clearWaitingTimeout()
setIsLoading(false)
setIsPlayerReady(true)
}
const handlePlaying = () => {
clearWaitingTimeout()
setIsLoading(false)
}
const handleWaiting = () => {
clearWaitingTimeout()
waitingTimeoutRef.current = setTimeout(() => {
setIsLoading(true)
}, 200)
}
const handleLoadStart = () => {
clearWaitingTimeout()
setIsLoading(true)
setIsPlayerReady(false)
})
player.on('timeupdate', () => {
const playerDuration = player.duration() || data.media?.metadata?.duration || 0
const bufferedEnd = player.bufferedEnd()
if (playerDuration > 0) {
setBuffered((bufferedEnd / playerDuration) * 100)
}
})
player.on('volumechange', () => {
const vol = player.volume()
const muted = player.muted()
setState((prev) => ({
...prev,
volume: vol || 0,
isMuted: muted || false,
}))
})
player.on('ratechange', () => {
setState((prev) => ({
...prev,
playbackRate: player.playbackRate() || 1,
}))
})
player.on('error', () => {
console.error('VideoJS Error:', player.error())
})
// Set initial state
player.volume(state.volume)
// 4. Cleanup
return () => {
if (player && !player.isDisposed()) {
player.dispose()
playerRef.current = null
videoRef.current = null
}
const handleProgress = () => {
const vidDuration = video.duration || data.media?.metadata?.duration || 0
if (vidDuration > 0 && video.buffered.length > 0) {
let bufferedEnd = 0
for (let i = 0; i < video.buffered.length; i++) {
if (
video.buffered.start(i) <= video.currentTime &&
video.currentTime <= video.buffered.end(i)
) {
bufferedEnd = video.buffered.end(i)
break
}
}
if (bufferedEnd === 0) {
bufferedEnd = video.buffered.end(video.buffered.length - 1)
}
setBuffered((bufferedEnd / vidDuration) * 100)
}
}
}, [data.id, Boolean(initialRes?.url), autoPlay])
const handleVolumeChange = () => {
setState((prev) => ({
...prev,
volume: video.volume,
isMuted: video.muted,
}))
}
const handleRateChange = () => {
setState((prev) => ({
...prev,
playbackRate: video.playbackRate,
}))
}
const handleError = () => {
clearWaitingTimeout()
setIsLoading(false)
console.error('Video Error:', video.error)
}
if (video.readyState >= 2) {
setIsLoading(false)
setIsPlayerReady(true)
}
video.addEventListener('play', handlePlay)
video.addEventListener('pause', handlePause)
video.addEventListener('ended', handleEnded)
video.addEventListener('loadeddata', handleLoadedData)
video.addEventListener('canplay', handleCanPlay)
video.addEventListener('playing', handlePlaying)
video.addEventListener('waiting', handleWaiting)
video.addEventListener('loadstart', handleLoadStart)
video.addEventListener('timeupdate', handleProgress)
video.addEventListener('progress', handleProgress)
video.addEventListener('volumechange', handleVolumeChange)
video.addEventListener('ratechange', handleRateChange)
video.addEventListener('error', handleError)
// Set initial state
video.volume = state.volume
// Cleanup
return () => {
clearWaitingTimeout()
video.removeEventListener('play', handlePlay)
video.removeEventListener('pause', handlePause)
video.removeEventListener('ended', handleEnded)
video.removeEventListener('loadeddata', handleLoadedData)
video.removeEventListener('canplay', handleCanPlay)
video.removeEventListener('playing', handlePlaying)
video.removeEventListener('waiting', handleWaiting)
video.removeEventListener('loadstart', handleLoadStart)
video.removeEventListener('timeupdate', handleProgress)
video.removeEventListener('progress', handleProgress)
video.removeEventListener('volumechange', handleVolumeChange)
video.removeEventListener('ratechange', handleRateChange)
video.removeEventListener('error', handleError)
if (hlsRef.current) {
hlsRef.current.destroy()
hlsRef.current = null
}
}
}, [data.id, targetSrc, isHls, autoPlay])
// Handle changes to startTime (e.g., clicking different chunks in search results)
useEffect(() => {
@@ -503,14 +628,20 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
// -- Event Handlers --
const togglePlay = useCallback(() => {
if (isLoading) return
// Disable click-to-play if drawing
if (useAnnotationStore.getState().isDrawing) return
const player = playerRef.current
if (!player) return
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (player.paused() || player.ended()) {
const playPromise = player.play()
const video = videoRef.current
if (!video) return
if (video.paused || video.ended) {
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch((error) => {
// Ignore AbortError which happens when pausing rapidly after playing
@@ -519,9 +650,9 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
})
}
} else {
player.pause()
video.pause()
}
}, [])
}, [isLoading])
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
@@ -565,10 +696,10 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
}, [togglePlay])
const toggleLoop = () => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const newLoop = !state.isLooping
player.loop(newLoop)
video.loop = newLoop
setState((prev) => ({ ...prev, isLooping: newLoop }))
}
@@ -580,48 +711,121 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
isMuted: newVolume === 0, // If dragging to 0, consider it muted
}))
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
// Set volume first so if we unmute, it is at the correct level
player.volume(newVolume)
video.volume = newVolume
// Manage mute state based on volume
if (newVolume > 0 && player.muted()) {
player.muted(false)
} else if (newVolume === 0 && !player.muted()) {
player.muted(true)
if (newVolume > 0 && video.muted) {
video.muted = false
} else if (newVolume === 0 && !video.muted) {
video.muted = true
}
}
const toggleMute = () => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const isMuted = player.muted()
const isMuted = video.muted
if (isMuted) {
player.muted(false)
video.muted = false
// If volume was 0 (e.g. user dragged to 0), restore to default 0.5 so they hear something
if (player.volume() === 0) {
player.volume(0.5)
if (video.volume === 0) {
video.volume = 0.5
}
} else {
player.muted(true)
video.muted = true
}
}
const changePlaybackRate = (rate: number) => {
const player = playerRef.current
if (!player) return
player.playbackRate(rate)
const video = videoRef.current
if (!video) return
video.playbackRate = rate
setState((prev) => ({ ...prev, playbackRate: rate }))
}
const changeResolution = (res: DisplayTranscode) => {
const player = playerRef.current
if (!player) return
const video = videoRef.current
if (!video) return
const wasPlaying = !player.paused()
const currentT = player.currentTime()
if (pendingResolutionCorrectionRef.current) {
pendingResolutionCorrectionRef.current()
pendingResolutionCorrectionRef.current = null
}
if (isHls && hlsRef.current) {
const hls = hlsRef.current
const wasPaused = video.paused
const targetFrame = currentFrame
if (res.resolution === 'Auto') {
if (state.currentResolution === 'Auto' && hls.currentLevel === -1) {
return
}
hls.currentLevel = -1
setState((prev) => ({
...prev,
currentResolution: 'Auto',
}))
} else {
const targetIndex = hls.levels.findIndex(
(lvl) =>
lvl.height === res.height ||
Math.max(lvl.width, lvl.height) === Math.max(res.width, res.height),
)
if (targetIndex !== -1) {
if (targetIndex === hls.currentLevel && state.currentResolution === res.resolution) {
return
}
hls.currentLevel = targetIndex
setState((prev) => ({
...prev,
currentResolution: res.resolution,
isCurrentHdr: res.hdr ?? false,
}))
}
}
// Guarantee frame accuracy if paused: re-nudge to safe center time once fragment is decoded
if (wasPaused) {
let timer: ReturnType<typeof setTimeout> | null = null
const cleanup = () => {
if (timer) {
clearTimeout(timer)
timer = null
}
video.removeEventListener('seeked', onSeeked)
hls.off?.(Hls.Events.ERROR, cleanup)
if (pendingResolutionCorrectionRef.current === cleanup) {
pendingResolutionCorrectionRef.current = null
}
}
const onSeeked = () => {
cleanup()
const safeCenterTime = calculateFrameCenterTime(targetFrame, frameRate)
const frameDuration = 1 / frameRate
if (Math.abs(video.currentTime - safeCenterTime) > frameDuration / 4) {
video.currentTime = safeCenterTime
}
}
// Safety timeout (1.5s): if no seeked event fires from rendition switch, cleanly detach
timer = setTimeout(cleanup, 1500)
pendingResolutionCorrectionRef.current = cleanup
video.addEventListener('seeked', onSeeked)
hls.once?.(Hls.Events.ERROR, cleanup)
}
return
}
const wasPlaying = !video.paused
const currentT = video.currentTime
setState((prev) => ({
...prev,
@@ -630,26 +834,27 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
isCurrentHdr: res.hdr ?? false,
}))
player.src({ type: 'video/mp4', src: res.url })
video.src = res.url || ''
player.one('loadedmetadata', () => {
player.currentTime(currentT)
const onLoadedMetadata = () => {
video.removeEventListener('loadedmetadata', onLoadedMetadata)
video.currentTime = currentT
if (wasPlaying) {
const playPromise = player.play()
const playPromise = video.play()
if (playPromise !== undefined) {
playPromise.catch((error) => {
if (error.name === 'AbortError') return
console.error('Play after seek failed:', error)
if (error.name !== 'AbortError') console.error('Play after seek failed:', error)
})
}
}
player.playbackRate(state.playbackRate)
})
video.playbackRate = state.playbackRate
}
video.addEventListener('loadedmetadata', onLoadedMetadata)
}
const toggleFullScreen = () => {
const rootEl = rootRef.current
const videoEl = videoHtmlEl || videoRef.current
const videoEl = videoRef.current
// 1. Standard W3C Fullscreen API (Desktop Chrome/Firefox/Safari, Android, iPadOS)
if (rootEl && typeof rootEl.requestFullscreen === 'function') {
@@ -727,7 +932,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
document.addEventListener('fullscreenchange', onFsChange)
// iOS WebKit fullscreen events on <video>
const videoEl = videoHtmlEl || videoRef.current
const videoEl = videoRef.current
const onWebkitBeginFs = () => {
setState((prev) => ({ ...prev, isFullScreen: true }))
}
@@ -746,7 +951,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
videoEl.removeEventListener('webkitendfullscreen', onWebkitEndFs)
}
}
}, [videoHtmlEl])
}, [])
// Center pan is owned by `pan` state (set by auto-fit / gestures).
const scale = zoom
@@ -754,9 +959,34 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
const isMobile = useIsMobile()
if (!hasMedia) {
if (data.status === 'failed') {
const errorMessage = data.media?.error
return (
<div
data-testid="video-viewer-failed-state"
className="flex h-full w-full flex-col items-center justify-center gap-3 p-6 text-center"
>
<AlertCircle className="h-12 w-12 text-destructive/80" />
<div className="space-y-1 max-w-md">
<h3 className="text-base font-semibold text-foreground">
{m.media_processing_failed()}
</h3>
<p className="text-sm text-muted-foreground">{m.media_processing_failed_desc()}</p>
{errorMessage && (
<div
title={errorMessage}
className="mt-3 rounded-lg border border-destructive/20 bg-destructive/10 p-3 text-xs text-destructive text-left line-clamp-3 break-words"
>
{errorMessage}
</div>
)}
</div>
</div>
)
}
return (
<div className="flex h-full w-full items-center justify-center">
<p className="text-muted-foreground">Media is not available.</p>
<p className="text-muted-foreground">{m.media_not_available()}</p>
</div>
)
}
@@ -783,7 +1013,6 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
'flex-1 bg-black cursor-pointer relative flex items-center justify-center overflow-hidden min-h-0 touch-none',
)}
onClick={togglePlay}
data-vjs-player
data-testid="video-area"
>
{/* Native Video Layer (Hardware-accelerated, full HDR EDR) */}
@@ -799,16 +1028,21 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
transformOrigin: '0 0',
}}
>
<div
ref={videoContainerRef}
className="w-full h-full [&_.video-js]:!w-full [&_.video-js]:!h-full [&_video]:!w-full [&_video]:!h-full [&_video]:!block [&_video]:!object-contain"
<video
ref={videoRef}
className="w-full h-full block object-contain pointer-events-none"
playsInline
preload="auto"
poster={data.preview?.thumbnailUrl || undefined}
/>
</div>
)}
{isAudio && (
<div
ref={videoContainerRef}
<video
ref={videoRef}
className="absolute inset-0 pointer-events-none opacity-0"
playsInline
preload="auto"
/>
)}
@@ -847,9 +1081,24 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
)
)}
{/* Loading Spinner Overlay */}
{isLoading && (
<div
className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200"
data-testid="video-loading-spinner"
>
<div className="w-16 h-16 bg-white/10 backdrop-blur-sm rounded-full flex items-center justify-center border-2 border-white/30">
<Loader2 className="w-8 h-8 text-white animate-spin" />
</div>
</div>
)}
{/* Initial Big Play Button Overlay (shown only before playback starts) */}
{!hasStartedPlaying && !isDrawing && (
<div className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200">
{!isLoading && !hasStartedPlaying && !isDrawing && (
<div
className="absolute inset-0 flex items-center justify-center bg-black/20 pointer-events-none z-10 transition-opacity duration-200"
data-testid="video-play-overlay"
>
<div className="w-20 h-20 bg-white/10 backdrop-blur-sm rounded-full flex items-center justify-center border-2 border-white/30">
<Play className="w-10 h-10 text-white ml-1 fill-white" />
</div>
@@ -883,6 +1132,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
onMouseEnter={handleControlsMouseEnter}
onMouseLeave={handleControlsMouseLeave}
allowDownload={allowDownload}
isLoading={isLoading}
/>
) : (
<VideoControlBar
@@ -909,6 +1159,7 @@ const VideoViewer = React.forwardRef<MediaController, FileViewerProps>(
onMouseEnter={handleControlsMouseEnter}
onMouseLeave={handleControlsMouseLeave}
allowDownload={allowDownload}
isLoading={isLoading}
/>
)}
</div>
+13 -6
View File
@@ -19,7 +19,7 @@ Shumai's video player is a decoupled, frame-locked system that coordinates three
▼ │
┌────────────────────────────┴──┐
│ Native HTML5 Video Player │
│ (VideoJS Engine Wrapper) │
│ (Hls.js / Native MSE) │
└───────────────────────────────┘
▲ ▲
│ │
@@ -32,7 +32,7 @@ Shumai's video player is a decoupled, frame-locked system that coordinates three
└──────────────┘ └──────────────┘
```
1. **Native HTML5 Video Player (VideoJS Engine Wrapper):** Manages the underlying browser media stream, buffering, audio clock, and presentation.
1. **Native HTML5 Video Player (Hls.js / Native MSE):** Manages the underlying browser media stream, buffering, audio clock, and presentation. Direct `<video>` element with zero library wrappers for MP4, and lightweight `hls.js` for adaptive HLS streams.
2. **Seekbar (Scrubber & Progress Bar):** Visualizes playback progress and maps horizontal screen clicks/drags to playhead percentages.
3. **Comment Area (Sidebar):** Renders comment list, displays timestamp labels, captures drawings, and triggers seeks to target frames.
@@ -204,14 +204,21 @@ When playing, the playhead loop coordinates two mechanisms in parallel to preven
---
### G. Switch Resolution
#### 1. HLS Streams (`hls.js`)
* **Immediate Rendition Switch (`currentLevel`)**: When the user selects a specific rendition (e.g. `1080p`), the player assigns `hls.currentLevel = targetIndex` rather than `nextLevel`. This causes `hls.js` to immediately flush the MSE buffer around `currentTime` and fetch fragments for the target rendition (matching Frame.io's responsive behavior).
* **Auto Mode**: When selecting `Auto`, `hls.currentLevel = -1` is set to re-enable adaptive bitrate (ABR). The player listens to `Hls.Events.LEVEL_SWITCHED` to update the control bar label dynamically (e.g., `Auto (1080p)`).
* **Paused Frame-Center Re-Nudge**: If the user switches quality while paused on a frame (e.g. Frame 120), the player listens for the subsequent `seeked` event once the new fragment renders and re-nudges `video.currentTime` to `calculateFrameCenterTime(currentFrame, frameRate)`. This guarantees zero frame drift across quality switches.
#### 2. Normal MP4 Streams (Native Video)
1. User selects a different resolution (e.g. `720p` or `Original`).
2. The player intercepts the click:
* Captures `wasPlaying = !player.paused()` and the current playhead time: `currentT = player.currentTime()`.
* Updates the source URL: `player.src({ type: 'video/mp4', src: res.url })`.
* Captures `wasPlaying = !video.paused` and the current playhead time: `currentT = video.currentTime`.
* Updates the source URL: `video.src = res.url`.
3. The player registers a one-shot `loadedmetadata` event listener on the new source:
* Restores playhead time: `player.currentTime(currentT)`.
* Restores playhead time: `video.currentTime = currentT`.
* Restores playback state (`play()` if `wasPlaying` was true) and playback rate.
4. The seek triggered by `player.currentTime(currentT)` fires a `seeked` event, which automatically calls `handleExternalSeeked`, recalculating the frame index and nudging the playhead to the safe frame center on the new media stream.
4. The seek triggered by `video.currentTime = currentT` fires a `seeked` event, which automatically calls `handleExternalSeeked`, recalculating the frame index and nudging the playhead to the safe frame center on the new media stream.
---
-2
View File
@@ -5,8 +5,6 @@
@source "./node_modules/@ferrucc-io/emoji-picker";
@source "../../node_modules/@ferrucc-io/emoji-picker";
@import 'video.js/dist/video-js.css' layer(base);
@custom-variant dark (&:is(.dark *));
:root {
+46 -1
View File
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest'
import { isImageFileName, getBestTranscode } from './media'
import { isImageFileName, getBestTranscode, getVideoResolutionLabel } from './media'
describe('isImageFileName', () => {
it('identifies web-supported image formats as images', () => {
@@ -49,3 +49,48 @@ describe('getBestTranscode', () => {
expect(getBestTranscode(transcodes, 1500)?.width).toBe(1200)
})
})
describe('getVideoResolutionLabel', () => {
it('preserves explicit resolution when present', () => {
expect(getVideoResolutionLabel({ width: 854, height: 480, resolution: '480p' })).toBe('480p')
expect(getVideoResolutionLabel({ width: 480, height: 854, resolution: '480p' })).toBe('480p')
expect(getVideoResolutionLabel({ width: 1920, height: 1080, resolution: '1080p' })).toBe(
'1080p',
)
})
it('derives resolution from long side for portrait legacy transcodes without explicit resolution', () => {
// 4K portrait
expect(getVideoResolutionLabel({ width: 2160, height: 3840 })).toBe('2160p')
// 2K / 1440p portrait
expect(getVideoResolutionLabel({ width: 1440, height: 2560 })).toBe('1440p')
// 1080p portrait
expect(getVideoResolutionLabel({ width: 1080, height: 1920 })).toBe('1080p')
// 720p portrait
expect(getVideoResolutionLabel({ width: 720, height: 1280 })).toBe('720p')
// 480p portrait
expect(getVideoResolutionLabel({ width: 480, height: 854 })).toBe('480p')
// 360p portrait
expect(getVideoResolutionLabel({ width: 360, height: 640 })).toBe('360p')
// 180p portrait
expect(getVideoResolutionLabel({ width: 180, height: 320 })).toBe('180p')
})
it('derives resolution from long side for landscape legacy transcodes without explicit resolution', () => {
expect(getVideoResolutionLabel({ width: 3840, height: 2160 })).toBe('2160p')
expect(getVideoResolutionLabel({ width: 2560, height: 1440 })).toBe('1440p')
expect(getVideoResolutionLabel({ width: 1920, height: 1080 })).toBe('1080p')
expect(getVideoResolutionLabel({ width: 1280, height: 720 })).toBe('720p')
expect(getVideoResolutionLabel({ width: 854, height: 480 })).toBe('480p')
expect(getVideoResolutionLabel({ width: 640, height: 360 })).toBe('360p')
expect(getVideoResolutionLabel({ width: 320, height: 180 })).toBe('180p')
})
it('falls back to short side or single dimension for non-standard sizes', () => {
expect(getVideoResolutionLabel({ width: 200, height: 100 })).toBe('100p')
expect(getVideoResolutionLabel({ height: 720 })).toBe('720p')
expect(getVideoResolutionLabel({ width: 500 })).toBe('500p')
expect(getVideoResolutionLabel(null)).toBe('')
expect(getVideoResolutionLabel(undefined)).toBe('')
})
})
+40
View File
@@ -40,3 +40,43 @@ export function getBestTranscode(
// If no candidate >= screenWidth, simply return the largest available (first in original sorted)
return sorted[0]
}
export function getVideoResolutionLabel(
transcode?: {
width?: number
height?: number
resolution?: string
} | null,
): string {
if (!transcode) return ''
if (transcode.resolution) {
return transcode.resolution
}
const width = transcode.width ?? 0
const height = transcode.height ?? 0
if (width > 0 && height > 0) {
const longSide = Math.max(width, height)
if (longSide >= 3840) return '2160p'
if (longSide >= 2560) return '1440p'
if (longSide >= 1920) return '1080p'
if (longSide >= 1280) return '720p'
if (longSide >= 854) return '480p'
if (longSide >= 640) return '360p'
if (longSide >= 320) return '180p'
const shortSide = Math.min(width, height)
return `${shortSide}p`
}
if (height > 0) {
return `${height}p`
}
if (width > 0) {
return `${width}p`
}
return ''
}

Some files were not shown because too many files have changed in this diff Show More