Commit Graph
34 Commits
Author SHA1 Message Date
Yiling-J 1796de8eda feat(premiere-uxp): introduce Adobe Premiere Pro UXP plugin for Shumai (#407)
* feat(premiere-uxp): use native Bun bundler and build script instead of Vite

* feat(premiere-uxp): switch FileListView to use Search API consistent with WebUI

* feat(premiere-uxp): add on-demand jpeg preview conversion for uxp compatibility

* feat(premiere-uxp): integrate spectrum web components (swc)

* fix(premiere-uxp): align swc packages with 0.37.0 and fix auth layout

* feat(premiere-uxp): use horizontal 2-column card view for projects and files

* style(premiere-uxp): enlarge file preview by 20% and add row padding

* fix(premiere-uxp): prevent card squishing and overflow with large item counts

* fix(premiere-uxp): change host in manifest to single object

* docs(premiere-uxp): add AGENTS.md with SWC preferences and UXP layout rules

* feat(premiere-uxp): integrate spectrum web components across plugin views

* fix(premiere-uxp): remove breadcrumb icons and use slash separator

* feat(premiere-uxp): add collapsible folder and file sections with infinite scroll

* feat(premiere-uxp): update file card layout with right column info and clean preview

* fix(premiere-uxp): use raw text for file card details and fix black icons

* style(premiere-uxp): use contain object-fit and black background for file card preview

* fix(premiere-uxp): refine header logo, sticky title bars, circular badges, and controls

* fix(premiere-uxp): replace stroke icons with spectrum workflow icons

* fix(premiere-uxp): switch icon buttons to swc sp-button with icon-only and slot=icon

* fix(premiere-uxp): vertically center endpoint text in header

* feat(premiere-uxp): add asset and proxy import support into Premiere Pro

- Implement context menu on file cards with left-click, right-click, and more action button triggers
- Support raw file import for non-video assets and video assets
- Support video proxy submenu listing ready transcode resolutions
- Add UXP native file picker save dialog and binary download orchestration
- Call Premiere Pro project.importFiles API to import downloaded assets into active project
- Add non-blocking toast notification banner for progress and status reporting
- Add unit tests for premiere host service and import orchestration

* fix(premiere-uxp): anchor import menu directly to file card using sp-action-menu

- Replace custom fixed-position ContextMenu with native SWC sp-action-menu
- Anchor menu directly to the triggered file card rather than the top-left window corner
- Support programmatic opening on file card click/contextmenu via imperative handle
- Support video submenu with Original and ready proxy options inside sp-action-menu
- Lazy load video proxies on menu open or hover
- Clean up obsolete fixed-position styles and unused backdrop overlays

* fix(premiere-uxp): align SWC packages to 0.37.0 and guard custom element registry

- Downgrade action-menu and menu SWC packages to 0.37.0 to match the rest of the workspace
- Eliminate duplicate icons-ui and sp-icon-chevron100 customElements.define calls
- Add polyfill guard to customElements.define to prevent duplicate registration errors during UXP reload

* fix(premiere-uxp): polyfill window.matchMedia for Adobe UXP

- Add window.matchMedia polyfill in polyfills.ts
- Satisfies SWC MatchMediaController used by ActionMenu and PickerBase in UXP
- Prevents TypeError when opening files page

* feat(premiere-uxp): replace context menu with card download action and video import dialog

* fix(premiere-uxp): resolve download url before save, pass null targetBin, and center download icon

* fix(premiere-uxp): resolve raw download url via download-url endpoint with fallback

* feat(premiere-uxp): use native folder picker and auto-increment conflict suffixes

* feat(premiere-uxp): add sequence linking, comment-marker sync, and navigation tabs

- Add sequence-to-asset linking with persistent project metadata storage

- Support one-way comment-to-marker timeline synchronization

- Add two-tab navigation (Sequences and Browse) using Spectrum buttons

- Add LinkSequenceDialog, SequencesView, and autoSyncService background engine

- Enforce Spectrum Workflow Icons and UXP layout rules

* feat(premiere-uxp): refine UI layouts, card interactions, and search box activation

* fix(premiere-uxp): set compact fixed width for file list search box

* fix(premiere-uxp): ensure search box click activation across full width in UXP

* fix(premiere-uxp): revert search box hacks and restore simple default sp-search layout

* feat(premiere-uxp): sync non-timestamped comments as markers at time 0

* feat(premiere-uxp): change link icon to unlink when asset is linked in file card

* fix(premiere-uxp): sync linked sequences between host project and browse view

* feat(premiere-uxp): remove synced markers on sequence unlink matching Frame.io

* fix(premiere-uxp): ensure reliable marker GUID capture, normalization, and fallback on unlink

* fix(premiere-uxp): use official Shumai logo for plugin manifest icons

* fix(premiere-uxp): use official PNG icons for manifest and plugin entrypoints

* refactor(premiere-uxp): use single 114x114 icon for plugin and panel

* feat(premiere-uxp): show file version in file list page file card

* build(premiere-uxp): check in built ccx package and update packaging configuration

* fix(premiere-uxp): address code review findings for media conversion, marker sync, and link scoping

- Reset hasJpegCover on project cover update and hasJpegPreview on asset media transcode
- Only advance synced comment metadata after confirmed marker transaction
- Normalize GUID keys and lookups in AutoSyncService map
- Restrict cached sequence link lookups to project-specific keys and prevent cross-project leakage on empty sequences
- Update built CCX distribution package
2026-09-17 11:51:03 +08:00
Yiling-J b37b2172e7 fix(transcode): co-locate screenshots and derived artifacts in asset storage directory (#399)
* fix(transcode): co-locate screenshots and derived artifacts in asset storage directory

* refactor(transcode): centralize derived artifact directory resolution and strengthen upload assertions
2026-09-09 11:03:48 +08:00
Yiling-J 04fd571bd9 feat(agent): enable read_asset tool for autofill agent (#396)
* feat(agent): enable read_asset tool for autofill agent

* fix(transcode): trigger autofill task for pdf assets

* fix(transcode): allow autofill task creation on all file assets

* fix(agent): require targetAssetId and teamId for autofill read_asset authorization

* feat(agent): instruct autofill agent to report failure directly if read_asset fails
2026-09-08 14:19:03 +08:00
Yiling-J b35c824a27 feat(transcode): optimize video operations using ffmpeg http range requests (#393)
* feat(transcode): optimize video operations using ffmpeg http range requests

* fix(transcode): preserve non-retryable errors and ensure mapConcurrent settles on rejection
2026-09-08 10:17:15 +08:00
Yiling-J 8f55ba1d37 feat(transcode): add hardware-accelerated video transcoding support (#352)
* feat(transcode): add hardware-accelerated video transcoding support

* feat(transcode): use encoder quality modes and source-capped maxrate for video transcode

* fix(transcode): scale max bitrate for downsampled preview framerates

* fix(transcode): propagate videoBitRate in getMediaInfoActivity and update auto description
2026-08-26 16:28:14 +08:00
Yiling-J aa6d5ed626 feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies (#283)
* feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies

* fix(transcode): use Gotenberg chromium markdown endpoint with index.html wrapper for markdown conversion

* test(transcode): add complete unit test coverage for HTML, CSV landscape/portrait, TXT, and Gotenberg fallback branches

* fix(gotenberg): force .md extension for Gotenberg markdown conversion endpoint compatibility

* refactor(gotenberg): standardize basic auth environment variables on GOTENBERG_BASIC_AUTH_USERNAME and GOTENBERG_BASIC_AUTH_PASSWORD

* test(e2e): add real Gotenberg testcontainers workflow E2E test for docx, pptx, and xlsx

* docs: add Gotenberg document PDF conversion documentation and update features in README

* docs(gotenberg): add Docker quickstart commands and remove obsolete GOTENBERG_USERNAME note
2026-08-09 14:44:13 +08:00
Yiling-J ee4933a4f8 feat(transcode): update preview thumbnail resolution to 300p with max length constraint (#281)
* feat(transcode): update preview thumbnail resolution to 300p with max length constraint

* fix(transcode): address code review feedback on preview thumbnail transcoding
2026-08-08 15:17:51 +08:00
Yiling-J 624a3a6a54 feat(webui): remove original-file display fallbacks, use proxies only (#273)
* feat(webui): remove original-file display fallbacks, use proxies only

WebUI now never uses the raw original file for display. All viewers
(image, video, compare, pdf) render exclusively from transcoded proxies;
when no proxy exists the viewer shows an unavailable placeholder instead
of falling back to media.original.

- getBestTranscode: drop the isRaw fallback candidates
- video-viewer / compare-video-pane: remove the synthetic 'Original'
  resolution entry and the original.downloadUrl hasMedia gate
- image-viewer / compare-image-pane: remove original.downloadUrl default
- pdf-viewer: require pdfTranscode.url (no original fallback)
- download menus (video-control-bar, breadcrumb-nav): use
  media.original.key for the 'Original' download option instead of the
  isRaw transcode marker entry

Backend:
- transcode-video workflow no longer pushes the isRaw:true marker entry
  into videoTranscodes (it pointed at the original file key)
- remove media.original.downloadUrl from the DTO, asset service mapping,
  MediaInfo DB type, and transcode workflows/activities (it was only
  consumed by the removed display fallbacks; downloads already use key
  via the download-url endpoints)

Tests/fixtures updated; docs updated to state the proxy-only display
policy.

* refactor: remove isRaw field via data migration, clean legacy media JSON

Add a Prisma data migration that cleans legacy media JSONB on both
assets and watermark_files:
- drop the raw-original marker entries (isRaw: true) from
  videoTranscodes / imageTranscodes — they pointed at the original file
  key and were only used by the removed display fallbacks / download
  menus, which now use media.original.key directly
- strip the isRaw field from remaining transcode entries
- drop original.downloadUrl (removed from the AssetInfo DTO)

The migration is generated via prisma migrate dev --create-only and
applied automatically by migrate deploy in production. It is guarded by
a test (packages/db/src/clean-media-migration.test.ts) that seeds legacy
media JSON, runs the migration SQL, and asserts the transform on both
tables.

With the data cleaned, remove the isRaw field entirely from the codebase:
- DTO schemas + ImageTranscode/VideoTranscode interfaces
- PrismaJson MediaInfo transcode types
- watermark activity isRaw skip (and its test), watermark proxy mapping,
  agent-embedding transcode selection, webui transcode filters, and all
  test fixtures

AGENTS.md: clarify that the no-manual-migration rule applies to DDL;
data migrations (pure SQL transforms, no schema change) are allowed
inside a --create-only generated migration and must be tested.
2026-08-07 22:40:06 +08:00
yiling 9a4f8d67c3 feat(transcode): stream uploads and transcode output to prevent memory crashes on large files 2026-07-29 21:22:49 +08:00
yiling e51d86e8f0 feat(transcode): trigger ai metadata autofill post transcode 2026-07-28 20:19:10 +08:00
yiling 8d1c51abca feat(transcode): add PSD support via ImageMagick and enforce sRGB conversion 2026-07-23 16:19:25 +08:00
yiling 208eb6898a feat(transcode): add markdown file to pdf proxy support 2026-07-20 18:35:16 +08:00
yiling 3afe0fc3c3 feat(api): remove mediaType and mimeType from DTO responses and DB MediaInfo interface 2026-07-20 14:37:17 +08:00
yiling f158d04686 feat(asset): replace mediaType/mimeType logic with proxyType 2026-07-20 10:22:36 +08:00
yiling 081dbf6d39 feat(transcode): add PDF proxy support for TXT and CSV files 2026-07-19 21:19:43 +08:00
yiling f68ba70656 feat(agent): add read_pdf_pages tool and PDF page context support 2026-07-19 09:45:14 +08:00
yiling bc1491bed5 feat(transcode): add PDF preview, sprite generation, and annotation support 2026-07-18 10:32:38 +08:00
yiling 8ad25215dd refactor(agent): prevent creating autofill and embedding tasks for audio assets 2026-07-05 16:31:14 +08:00
yiling a2e982f2e6 feat(transcode): add audio file support with AAC stereo mp4 proxy and styled viewer 2026-07-05 15:11:19 +08:00
yiling b7367bb129 refactor(storage): use original filenames and fix download UX
- Store uploaded files as files/{ulid}/{original-name} instead of files/{ulid}/raw
- Store transcoded videos as {stem}-{resolution}.mp4 (e.g. foo-720p.mp4)
- Store transcoded images as {stem}-{width}p.webp (e.g. foo-1920p.webp)
- Add download?: boolean param to presign - only sets Content-Disposition: attachment when true
- Local storage handler only sets Content-Disposition when ?download=1 query param is present
- Add POST /api/files/download-url endpoint for key-based single file downloads
  - Uses ResourceType.Asset for permission check
  - Validates key belongs to the asset's storage directory
- Fix existing download-links API to pass download: true to presign
- Fix breadcrumb navbar download: video shows resolution submenu, others download directly
- Fix video player download: calls download-url API, shows correct file type label
- Remove filename query param hack from presign URLs
- Add sanitizeFilename() and stemFromKey() utility helpers
- Add key field to VideoTranscode DTO interface
2026-07-02 17:39:23 +08:00
yiling 95f91cd4de feat(db): add custom migration to backfill totalFrames and startTimecode 2026-06-29 11:27:32 +08:00
yiling eee7b5a677 feat(video): add SMPTE timecode support and format selection dropdown 2026-06-28 21:04:40 +08:00
yiling 837fe5bd7a feat(video): add frame-accurate video player and CFR proxy generation 2026-06-28 20:35:47 +08:00
yiling 718d9d5b51 feat(transcode): optimize video chunks for embedding by forcing 1 FPS and dropping audio 2026-06-18 19:53:19 +08:00
yiling d84950d372 feat(transcode): set loglevel to warning for all ffmpeg commands 2026-06-18 11:57:38 +08:00
yiling 46815736e2 feat(embedding): decouple video chunk transcode and embedding generation 2026-06-18 11:54:30 +08:00
yiling f399d2c469 feat(embedding): defer embedding workflow, use transcoded assets, and execute chunks in separate activities 2026-06-18 10:57:22 +08:00
yiling 1ad5720f42 fix(transcode): handle high-precision FPS using rational fractions 2026-06-17 22:55:01 +08:00
yiling 5111fcd6a5 refactor(transcode): address code review feedback on imports and dataFormatNames typing 2026-06-16 09:53:28 +08:00
yiling f16a8a7069 refactor(transcode): relocate TranscodeService to core and flatten API package 2026-06-16 09:47:27 +08:00
yiling d687ea495b fix(transcode): make deterministic transcode activity failures non-retryable in Temporal 2026-06-14 10:00:13 +08:00
yiling 71f5881228 fix(transcode): automatically populate file_type metadata field 2026-06-07 17:27:47 +08:00
yiling 25d12d1574 feat(agent): improve agent chat workflow and add media tools 2026-06-05 21:25:08 +08:00
yiling e3f4fa219e refactor(monorepo): decouple and split background workflow workers into packages
Splits monolithic workflow workers into four decoupled workspaces under the Bun workspaces monorepo:
- @shumai/workflow-core containing the core execution engine and utilities.
- @shumai/agent containing agent-specific logic, workflows, and activities.
- @shumai/transcode containing transcoding workflows, transcoder, and transcode activities.
- @shumai/worker-db containing database activities.

Introduces a dynamic workflow and activity registration pattern on LocalExecutor to eliminate circular dependencies. Updates worker bootstrapping and test suites to cleanly leverage generic workspace modules.
2026-06-01 23:29:11 +08:00