Commit Graph
38 Commits
Author SHA1 Message Date
Yiling-J 941bf0fd1b feat(transcode): smart poster frame selection and decoupled generation (#428)
* feat(transcode): smart poster frame selection and decoupled generation

* fix(transcode): detect MPEG-TS extension from URL pathname for signed remote sources
2026-09-25 16:37:16 +08:00
Yiling-J c3375ee935 feat(transcode): add HDR video proxy support and playback (#424)
* feat(transcode): add HDR output proxy support

* fix(transcode): auto-detect MoltenVK Vulkan ICD on macOS for libplacebo filters

* fix(transcode): split filter in generateSprite to ensure poster uses tone-mapped stream

* docs(changelog): update docs/changelog.mdx and ignore in prettier

* fix(transcode): respect video rotation metadata to preserve display aspect ratio

* fix(webui): render native video with hardware EDR acceleration instead of 2D canvas

* fix(webui): show HDR label on mobile video control bar

* fix(share): preserve hdr flag on watermarked video transcodes

* fix(webui): differentiate HDR download options in video player

* fix(transcode): handle Dolby Vision Profile 5 via standard HDR pipeline without libplacebo

* fix(transcode): prepend setparams filter to prevent zscale colorspace path error for DV P5

* refactor(transcode): simplify HDR proxies to always produce HDR without SDR setting
2026-09-24 14:32:47 +08:00
Yiling-J b52b2ce7d8 fix(transcode): preserve trashed status during background jobs and handle permanent purge (#416)
* fix(transcode): preserve trashed status during background jobs and handle permanent purge

* fix(workflow): clear heartbeat and add info logs on task cancellation

* fix(transcode): terminate media processes and guard task activities on cancellation

- Pass AbortSignal to ffmpeg and pdftoppm child processes in transcodeService to terminate processes on task cancellation
- Add local task AbortController context and cancellation guards in executeActivity to prevent subsequent workflow activities
- Forward taskId and cancellation signals in transcode, screenshot, and PDF activities
- Use updateMany instead of update in task activities to eliminate Prisma P2025 errors on purged task records

* fix(workflow): allow cleanup activities in finally blocks during task cancellation

Remove blanket cancellation check in executeActivity so that workflows can execute cleanup and failure handling activities in catch and finally blocks upon cancellation, while relying on AbortSignal in active activities to abort execution.

* docs(changelog): improve user friendliness of changelog entries and formatting example

* docs(agents): update changelog guidelines to emphasize end-user friendly descriptions

* fix(transcode): handle queued cancellation and delete races in media workflow
2026-09-19 10:40:49 +08:00
Yiling-J a91d8c4521 feat(settings): add FFmpeg threads setting for media processing (#415)
* feat(settings): add FFmpeg threads setting for media processing

* style(settings): add bottom padding to media processing page
2026-09-18 13:30:33 +08:00
Yiling-J 1796de8eda feat(premiere-uxp): introduce Adobe Premiere Pro UXP plugin for Shumai (#407)
* feat(premiere-uxp): use native Bun bundler and build script instead of Vite

* feat(premiere-uxp): switch FileListView to use Search API consistent with WebUI

* feat(premiere-uxp): add on-demand jpeg preview conversion for uxp compatibility

* feat(premiere-uxp): integrate spectrum web components (swc)

* fix(premiere-uxp): align swc packages with 0.37.0 and fix auth layout

* feat(premiere-uxp): use horizontal 2-column card view for projects and files

* style(premiere-uxp): enlarge file preview by 20% and add row padding

* fix(premiere-uxp): prevent card squishing and overflow with large item counts

* fix(premiere-uxp): change host in manifest to single object

* docs(premiere-uxp): add AGENTS.md with SWC preferences and UXP layout rules

* feat(premiere-uxp): integrate spectrum web components across plugin views

* fix(premiere-uxp): remove breadcrumb icons and use slash separator

* feat(premiere-uxp): add collapsible folder and file sections with infinite scroll

* feat(premiere-uxp): update file card layout with right column info and clean preview

* fix(premiere-uxp): use raw text for file card details and fix black icons

* style(premiere-uxp): use contain object-fit and black background for file card preview

* fix(premiere-uxp): refine header logo, sticky title bars, circular badges, and controls

* fix(premiere-uxp): replace stroke icons with spectrum workflow icons

* fix(premiere-uxp): switch icon buttons to swc sp-button with icon-only and slot=icon

* fix(premiere-uxp): vertically center endpoint text in header

* feat(premiere-uxp): add asset and proxy import support into Premiere Pro

- Implement context menu on file cards with left-click, right-click, and more action button triggers
- Support raw file import for non-video assets and video assets
- Support video proxy submenu listing ready transcode resolutions
- Add UXP native file picker save dialog and binary download orchestration
- Call Premiere Pro project.importFiles API to import downloaded assets into active project
- Add non-blocking toast notification banner for progress and status reporting
- Add unit tests for premiere host service and import orchestration

* fix(premiere-uxp): anchor import menu directly to file card using sp-action-menu

- Replace custom fixed-position ContextMenu with native SWC sp-action-menu
- Anchor menu directly to the triggered file card rather than the top-left window corner
- Support programmatic opening on file card click/contextmenu via imperative handle
- Support video submenu with Original and ready proxy options inside sp-action-menu
- Lazy load video proxies on menu open or hover
- Clean up obsolete fixed-position styles and unused backdrop overlays

* fix(premiere-uxp): align SWC packages to 0.37.0 and guard custom element registry

- Downgrade action-menu and menu SWC packages to 0.37.0 to match the rest of the workspace
- Eliminate duplicate icons-ui and sp-icon-chevron100 customElements.define calls
- Add polyfill guard to customElements.define to prevent duplicate registration errors during UXP reload

* fix(premiere-uxp): polyfill window.matchMedia for Adobe UXP

- Add window.matchMedia polyfill in polyfills.ts
- Satisfies SWC MatchMediaController used by ActionMenu and PickerBase in UXP
- Prevents TypeError when opening files page

* feat(premiere-uxp): replace context menu with card download action and video import dialog

* fix(premiere-uxp): resolve download url before save, pass null targetBin, and center download icon

* fix(premiere-uxp): resolve raw download url via download-url endpoint with fallback

* feat(premiere-uxp): use native folder picker and auto-increment conflict suffixes

* feat(premiere-uxp): add sequence linking, comment-marker sync, and navigation tabs

- Add sequence-to-asset linking with persistent project metadata storage

- Support one-way comment-to-marker timeline synchronization

- Add two-tab navigation (Sequences and Browse) using Spectrum buttons

- Add LinkSequenceDialog, SequencesView, and autoSyncService background engine

- Enforce Spectrum Workflow Icons and UXP layout rules

* feat(premiere-uxp): refine UI layouts, card interactions, and search box activation

* fix(premiere-uxp): set compact fixed width for file list search box

* fix(premiere-uxp): ensure search box click activation across full width in UXP

* fix(premiere-uxp): revert search box hacks and restore simple default sp-search layout

* feat(premiere-uxp): sync non-timestamped comments as markers at time 0

* feat(premiere-uxp): change link icon to unlink when asset is linked in file card

* fix(premiere-uxp): sync linked sequences between host project and browse view

* feat(premiere-uxp): remove synced markers on sequence unlink matching Frame.io

* fix(premiere-uxp): ensure reliable marker GUID capture, normalization, and fallback on unlink

* fix(premiere-uxp): use official Shumai logo for plugin manifest icons

* fix(premiere-uxp): use official PNG icons for manifest and plugin entrypoints

* refactor(premiere-uxp): use single 114x114 icon for plugin and panel

* feat(premiere-uxp): show file version in file list page file card

* build(premiere-uxp): check in built ccx package and update packaging configuration

* fix(premiere-uxp): address code review findings for media conversion, marker sync, and link scoping

- Reset hasJpegCover on project cover update and hasJpegPreview on asset media transcode
- Only advance synced comment metadata after confirmed marker transaction
- Normalize GUID keys and lookups in AutoSyncService map
- Restrict cached sequence link lookups to project-specific keys and prevent cross-project leakage on empty sequences
- Update built CCX distribution package
2026-09-17 11:51:03 +08:00
Yiling-J b37b2172e7 fix(transcode): co-locate screenshots and derived artifacts in asset storage directory (#399)
* fix(transcode): co-locate screenshots and derived artifacts in asset storage directory

* refactor(transcode): centralize derived artifact directory resolution and strengthen upload assertions
2026-09-09 11:03:48 +08:00
Yiling-J 04fd571bd9 feat(agent): enable read_asset tool for autofill agent (#396)
* feat(agent): enable read_asset tool for autofill agent

* fix(transcode): trigger autofill task for pdf assets

* fix(transcode): allow autofill task creation on all file assets

* fix(agent): require targetAssetId and teamId for autofill read_asset authorization

* feat(agent): instruct autofill agent to report failure directly if read_asset fails
2026-09-08 14:19:03 +08:00
Yiling-J b35c824a27 feat(transcode): optimize video operations using ffmpeg http range requests (#393)
* feat(transcode): optimize video operations using ffmpeg http range requests

* fix(transcode): preserve non-retryable errors and ensure mapConcurrent settles on rejection
2026-09-08 10:17:15 +08:00
Yiling-J 8f55ba1d37 feat(transcode): add hardware-accelerated video transcoding support (#352)
* feat(transcode): add hardware-accelerated video transcoding support

* feat(transcode): use encoder quality modes and source-capped maxrate for video transcode

* fix(transcode): scale max bitrate for downsampled preview framerates

* fix(transcode): propagate videoBitRate in getMediaInfoActivity and update auto description
2026-08-26 16:28:14 +08:00
Yiling-J aa6d5ed626 feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies (#283)
* feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies

* fix(transcode): use Gotenberg chromium markdown endpoint with index.html wrapper for markdown conversion

* test(transcode): add complete unit test coverage for HTML, CSV landscape/portrait, TXT, and Gotenberg fallback branches

* fix(gotenberg): force .md extension for Gotenberg markdown conversion endpoint compatibility

* refactor(gotenberg): standardize basic auth environment variables on GOTENBERG_BASIC_AUTH_USERNAME and GOTENBERG_BASIC_AUTH_PASSWORD

* test(e2e): add real Gotenberg testcontainers workflow E2E test for docx, pptx, and xlsx

* docs: add Gotenberg document PDF conversion documentation and update features in README

* docs(gotenberg): add Docker quickstart commands and remove obsolete GOTENBERG_USERNAME note
2026-08-09 14:44:13 +08:00
Yiling-J ee4933a4f8 feat(transcode): update preview thumbnail resolution to 300p with max length constraint (#281)
* feat(transcode): update preview thumbnail resolution to 300p with max length constraint

* fix(transcode): address code review feedback on preview thumbnail transcoding
2026-08-08 15:17:51 +08:00
Yiling-J 624a3a6a54 feat(webui): remove original-file display fallbacks, use proxies only (#273)
* feat(webui): remove original-file display fallbacks, use proxies only

WebUI now never uses the raw original file for display. All viewers
(image, video, compare, pdf) render exclusively from transcoded proxies;
when no proxy exists the viewer shows an unavailable placeholder instead
of falling back to media.original.

- getBestTranscode: drop the isRaw fallback candidates
- video-viewer / compare-video-pane: remove the synthetic 'Original'
  resolution entry and the original.downloadUrl hasMedia gate
- image-viewer / compare-image-pane: remove original.downloadUrl default
- pdf-viewer: require pdfTranscode.url (no original fallback)
- download menus (video-control-bar, breadcrumb-nav): use
  media.original.key for the 'Original' download option instead of the
  isRaw transcode marker entry

Backend:
- transcode-video workflow no longer pushes the isRaw:true marker entry
  into videoTranscodes (it pointed at the original file key)
- remove media.original.downloadUrl from the DTO, asset service mapping,
  MediaInfo DB type, and transcode workflows/activities (it was only
  consumed by the removed display fallbacks; downloads already use key
  via the download-url endpoints)

Tests/fixtures updated; docs updated to state the proxy-only display
policy.

* refactor: remove isRaw field via data migration, clean legacy media JSON

Add a Prisma data migration that cleans legacy media JSONB on both
assets and watermark_files:
- drop the raw-original marker entries (isRaw: true) from
  videoTranscodes / imageTranscodes — they pointed at the original file
  key and were only used by the removed display fallbacks / download
  menus, which now use media.original.key directly
- strip the isRaw field from remaining transcode entries
- drop original.downloadUrl (removed from the AssetInfo DTO)

The migration is generated via prisma migrate dev --create-only and
applied automatically by migrate deploy in production. It is guarded by
a test (packages/db/src/clean-media-migration.test.ts) that seeds legacy
media JSON, runs the migration SQL, and asserts the transform on both
tables.

With the data cleaned, remove the isRaw field entirely from the codebase:
- DTO schemas + ImageTranscode/VideoTranscode interfaces
- PrismaJson MediaInfo transcode types
- watermark activity isRaw skip (and its test), watermark proxy mapping,
  agent-embedding transcode selection, webui transcode filters, and all
  test fixtures

AGENTS.md: clarify that the no-manual-migration rule applies to DDL;
data migrations (pure SQL transforms, no schema change) are allowed
inside a --create-only generated migration and must be tested.
2026-08-07 22:40:06 +08:00
yiling 9a4f8d67c3 feat(transcode): stream uploads and transcode output to prevent memory crashes on large files 2026-07-29 21:22:49 +08:00
yiling e51d86e8f0 feat(transcode): trigger ai metadata autofill post transcode 2026-07-28 20:19:10 +08:00
yiling 8d1c51abca feat(transcode): add PSD support via ImageMagick and enforce sRGB conversion 2026-07-23 16:19:25 +08:00
yiling 208eb6898a feat(transcode): add markdown file to pdf proxy support 2026-07-20 18:35:16 +08:00
yiling 3afe0fc3c3 feat(api): remove mediaType and mimeType from DTO responses and DB MediaInfo interface 2026-07-20 14:37:17 +08:00
yiling f158d04686 feat(asset): replace mediaType/mimeType logic with proxyType 2026-07-20 10:22:36 +08:00
yiling 081dbf6d39 feat(transcode): add PDF proxy support for TXT and CSV files 2026-07-19 21:19:43 +08:00
yiling f68ba70656 feat(agent): add read_pdf_pages tool and PDF page context support 2026-07-19 09:45:14 +08:00
yiling bc1491bed5 feat(transcode): add PDF preview, sprite generation, and annotation support 2026-07-18 10:32:38 +08:00
yiling 8ad25215dd refactor(agent): prevent creating autofill and embedding tasks for audio assets 2026-07-05 16:31:14 +08:00
yiling a2e982f2e6 feat(transcode): add audio file support with AAC stereo mp4 proxy and styled viewer 2026-07-05 15:11:19 +08:00
yiling b7367bb129 refactor(storage): use original filenames and fix download UX
- Store uploaded files as files/{ulid}/{original-name} instead of files/{ulid}/raw
- Store transcoded videos as {stem}-{resolution}.mp4 (e.g. foo-720p.mp4)
- Store transcoded images as {stem}-{width}p.webp (e.g. foo-1920p.webp)
- Add download?: boolean param to presign - only sets Content-Disposition: attachment when true
- Local storage handler only sets Content-Disposition when ?download=1 query param is present
- Add POST /api/files/download-url endpoint for key-based single file downloads
  - Uses ResourceType.Asset for permission check
  - Validates key belongs to the asset's storage directory
- Fix existing download-links API to pass download: true to presign
- Fix breadcrumb navbar download: video shows resolution submenu, others download directly
- Fix video player download: calls download-url API, shows correct file type label
- Remove filename query param hack from presign URLs
- Add sanitizeFilename() and stemFromKey() utility helpers
- Add key field to VideoTranscode DTO interface
2026-07-02 17:39:23 +08:00
yiling 95f91cd4de feat(db): add custom migration to backfill totalFrames and startTimecode 2026-06-29 11:27:32 +08:00
yiling eee7b5a677 feat(video): add SMPTE timecode support and format selection dropdown 2026-06-28 21:04:40 +08:00
yiling 837fe5bd7a feat(video): add frame-accurate video player and CFR proxy generation 2026-06-28 20:35:47 +08:00
yiling 718d9d5b51 feat(transcode): optimize video chunks for embedding by forcing 1 FPS and dropping audio 2026-06-18 19:53:19 +08:00
yiling d84950d372 feat(transcode): set loglevel to warning for all ffmpeg commands 2026-06-18 11:57:38 +08:00
yiling 46815736e2 feat(embedding): decouple video chunk transcode and embedding generation 2026-06-18 11:54:30 +08:00
yiling f399d2c469 feat(embedding): defer embedding workflow, use transcoded assets, and execute chunks in separate activities 2026-06-18 10:57:22 +08:00
yiling 1ad5720f42 fix(transcode): handle high-precision FPS using rational fractions 2026-06-17 22:55:01 +08:00
yiling 5111fcd6a5 refactor(transcode): address code review feedback on imports and dataFormatNames typing 2026-06-16 09:53:28 +08:00
yiling f16a8a7069 refactor(transcode): relocate TranscodeService to core and flatten API package 2026-06-16 09:47:27 +08:00
yiling d687ea495b fix(transcode): make deterministic transcode activity failures non-retryable in Temporal 2026-06-14 10:00:13 +08:00
yiling 71f5881228 fix(transcode): automatically populate file_type metadata field 2026-06-07 17:27:47 +08:00
yiling 25d12d1574 feat(agent): improve agent chat workflow and add media tools 2026-06-05 21:25:08 +08:00
yiling e3f4fa219e refactor(monorepo): decouple and split background workflow workers into packages
Splits monolithic workflow workers into four decoupled workspaces under the Bun workspaces monorepo:
- @shumai/workflow-core containing the core execution engine and utilities.
- @shumai/agent containing agent-specific logic, workflows, and activities.
- @shumai/transcode containing transcoding workflows, transcoder, and transcode activities.
- @shumai/worker-db containing database activities.

Introduces a dynamic workflow and activity registration pattern on LocalExecutor to eliminate circular dependencies. Updates worker bootstrapping and test suites to cleanly leverage generic workspace modules.
2026-06-01 23:29:11 +08:00